2024-09-21T05:29:34.434 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T05:29:34.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:29:34.551 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7913499' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected - object missing on disk - Invalid tag char sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP9CeIu7ANUoFBktppMpQJ6Gc/Gn96RqKc73kOHErFLQVeiGB367EjKHU/uKk4pcLgHlWZjPqkrx828Ptm+3vD4= smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATMHMnY0uoDaGO8C6hujcaU6+qoC9ev28eIKzG9ui6NCoIlB9EBr2cO5xO5nJ/pOyKm3fA7PZ/+TPV9qGYdOUE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T05:29:34.552 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T05:29:34.552 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T05:29:34.552 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T05:29:34.554 INFO:teuthology.task.internal:Checking packages... 2024-09-21T05:29:34.575 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T05:29:34.575 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T05:29:34.575 INFO:teuthology.packaging:ref: None 2024-09-21T05:29:34.575 INFO:teuthology.packaging:tag: None 2024-09-21T05:29:34.575 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:29:34.576 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:29:34.576 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T05:29:34.731 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T05:29:34.732 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T05:29:34.739 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T05:29:34.739 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T05:29:34.764 INFO:teuthology.task.internal:Saving configuration 2024-09-21T05:29:34.781 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T05:29:34.791 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T05:29:34.817 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:24:46.295564', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP9CeIu7ANUoFBktppMpQJ6Gc/Gn96RqKc73kOHErFLQVeiGB367EjKHU/uKk4pcLgHlWZjPqkrx828Ptm+3vD4='} 2024-09-21T05:29:34.837 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 05:24:46.293571', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATMHMnY0uoDaGO8C6hujcaU6+qoC9ev28eIKzG9ui6NCoIlB9EBr2cO5xO5nJ/pOyKm3fA7PZ/+TPV9qGYdOUE='} 2024-09-21T05:29:34.837 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T05:29:34.844 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T05:29:34.844 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T05:29:34.844 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T05:29:34.908 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe1e97c0160>, signals=[15]) 2024-09-21T05:29:34.908 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T05:29:34.915 INFO:teuthology.task.internal:Opening connections... 2024-09-21T05:29:34.915 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:29:34.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:29:34.997 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:29:34.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:29:35.074 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T05:29:35.081 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-09-21T05:29:35.099 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-09-21T05:29:35.100 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-09-21T05:29:35.157 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:29:35.158 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:29:35.159 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:29:35.159 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-09-21T05:29:35.181 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-21T05:29:35.200 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-21T05:29:35.200 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-21T05:29:35.257 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="CentOS Stream" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="9" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:ID="centos" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE="rhel fedora" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="9" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:PLATFORM_ID="platform:el9" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:ANSI_COLOR="0;31" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:LOGO="fedora-logo-icon" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://centos.org/" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T05:29:35.258 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T05:29:35.259 INFO:teuthology.orchestra.run.smithi134.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T05:29:35.259 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-21T05:29:35.281 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T05:29:35.290 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T05:29:35.298 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T05:29:35.298 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:29:35.300 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-21T05:29:35.317 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T05:29:35.322 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T05:29:35.322 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:29:35.357 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-21T05:29:35.476 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T05:29:35.533 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T05:29:35.547 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T05:29:35.547 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-09-21T05:29:35.547 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-09-21T05:29:35.548 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:29:35.548 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T05:29:35.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:29:35.565 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-21T05:29:35.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:29:35.566 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-21T05:29:35.623 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-503.el9.x86_64 2024-09-21T05:29:35.623 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-503.el9.x86_64 2024-09-21T05:29:35.624 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-09-21T05:29:35.625 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-503.el9.x86_64 2024-09-21T05:29:35.625 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.14.0-503.el9.x86_64 2024-09-21T05:29:35.625 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum install -y kernel 2024-09-21T05:29:37.896 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T05:29:38.152 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-21T05:29:39.722 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-21T05:29:40.403 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-21T05:29:44.215 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T05:29:45.736 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-21T05:29:45.923 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-21T05:29:46.669 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 37 MB/s | 23 MB 00:00 2024-09-21T05:29:47.279 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-21T05:29:48.191 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-21T05:29:52.334 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T05:29:53.622 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-21T05:29:54.332 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T05:29:54.332 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T05:29:54.432 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:29:54.443 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-21T05:29:54.444 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-09-21T05:29:54.444 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-09-21T05:29:54.444 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:29:54.444 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 79 M 2024-09-21T05:29:54.444 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 126 M 2024-09-21T05:29:54.444 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-21T05:29:54.560 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 382 kB/s | 36 kB 00:00 2024-09-21T05:29:54.935 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-21T05:29:55.377 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-09-21T05:29:55.528 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T05:29:55.528 INFO:teuthology.orchestra.run.smithi134.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T05:29:55.632 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout: Package Architecture Version Repository Size 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-21T05:29:55.636 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout:================================================================================ 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout:Install 4 Packages 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:29:55.637 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 79 M 2024-09-21T05:29:55.638 INFO:teuthology.orchestra.run.smithi134.stdout:Installed size: 126 M 2024-09-21T05:29:55.638 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-21T05:29:55.669 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 26 MB/s | 29 MB 00:01 2024-09-21T05:29:55.669 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:29:55.669 INFO:teuthology.orchestra.run.smithi022.stdout:Total 64 MB/s | 79 MB 00:01 2024-09-21T05:29:56.049 INFO:teuthology.orchestra.run.smithi134.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 271 kB/s | 36 kB 00:00 2024-09-21T05:29:56.378 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-21T05:29:56.751 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-21T05:29:56.751 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-21T05:29:57.590 INFO:teuthology.orchestra.run.smithi134.stdout:(2/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-21T05:29:57.726 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-21T05:29:57.726 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-21T05:29:58.790 INFO:teuthology.orchestra.run.smithi134.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-09-21T05:29:59.257 INFO:teuthology.orchestra.run.smithi134.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 8.9 MB/s | 29 MB 00:03 2024-09-21T05:29:59.257 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:29:59.258 INFO:teuthology.orchestra.run.smithi134.stdout:Total 22 MB/s | 79 MB 00:03 2024-09-21T05:29:59.942 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-21T05:30:00.266 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-21T05:30:00.266 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-21T05:30:00.275 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-21T05:30:00.949 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:30:01.028 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:30:01.200 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-21T05:30:01.200 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-21T05:30:03.630 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-21T05:30:03.769 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:30:03.899 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:30:04.339 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:30:04.418 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:30:07.177 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:30:07.338 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:30:09.024 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:30:09.066 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:12.152 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:30:12.197 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:14.208 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:17.118 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:53.295 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:53.335 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:53.691 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:53.691 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:30:53.691 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:30:53.692 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:30:54.542 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:30:54.543 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-21T05:30:54.901 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:30:54.982 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:30:54.982 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:30:54.982 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T05:30:54.982 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T05:30:54.983 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T05:30:54.983 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T05:30:54.983 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T05:30:54.983 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:30:55.040 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:30:55.041 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:30:55.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-09-21T05:30:55.079 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 82.el9 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8274670 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-09-21T05:30:55.080 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:30:55.081 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T05:30:55.082 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T05:30:55.082 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-508.el9.x86_64 2024-09-21T05:30:55.082 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T05:30:55.378 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:55.418 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:55.739 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-09-21T05:30:55.803 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:55.804 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T05:30:55.804 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T05:30:55.804 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:30:56.637 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-21T05:30:57.010 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:30:57.054 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T05:30:57.095 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-09-21T05:30:57.096 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-09-21T05:30:57.103 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:30:57.104 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:30:57.104 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T05:30:57.104 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T05:30:57.105 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T05:30:57.105 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-09-21T05:30:57.105 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T05:30:57.105 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T05:30:57.111 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.Y0GvbqT1bE 2024-09-21T05:30:57.112 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y0GvbqT1bE 2024-09-21T05:30:57.165 INFO:teuthology.orchestra.run.smithi134.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T05:30:57.165 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T05:30:57.165 DEBUG:teuthology.orchestra.run.smithi134:> sudo rpm -qi grub2-tools 2024-09-21T05:30:57.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.Y0GvbqT1bE 2024-09-21T05:30:57.203 INFO:teuthology.orchestra.run.smithi134.stdout:Name : grub2-tools 2024-09-21T05:30:57.203 INFO:teuthology.orchestra.run.smithi134.stdout:Epoch : 1 2024-09-21T05:30:57.203 INFO:teuthology.orchestra.run.smithi134.stdout:Version : 2.06 2024-09-21T05:30:57.203 INFO:teuthology.orchestra.run.smithi134.stdout:Release : 82.el9 2024-09-21T05:30:57.203 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: x86_64 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Group : Unspecified 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Size : 8274670 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:License : GPLv3+ 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Packager : builder@centos.org 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Vendor : CentOS 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Summary : Support tools for GRUB. 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:Description : 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T05:30:57.204 INFO:teuthology.orchestra.run.smithi134.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T05:30:57.205 INFO:teuthology.orchestra.run.smithi134.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T05:30:57.205 INFO:teuthology.orchestra.run.smithi134.stdout:hardware devices. 2024-09-21T05:30:57.205 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:30:57.205 INFO:teuthology.orchestra.run.smithi134.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T05:30:57.205 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T05:30:57.206 INFO:teuthology.task.kernel:Updating grub on smithi134 to boot 5.14.0-508.el9.x86_64 2024-09-21T05:30:57.206 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T05:30:57.316 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.Y0GvbqT1bE is 6KB 2024-09-21T05:30:57.365 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.Y0GvbqT1bE 2024-09-21T05:30:57.379 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T05:30:57.448 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T05:30:57.449 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T05:30:57.449 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T05:30:57.449 INFO:teuthology.orchestra.run.smithi022.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T05:30:57.450 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T05:30:57.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-09-21T05:30:57.854 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-09-21T05:30:59.248 INFO:teuthology.orchestra.run.smithi134.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T05:30:59.284 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-09-21T05:30:59.286 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-21T05:30:59.302 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.oo4ySIemxC 2024-09-21T05:30:59.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.oo4ySIemxC 2024-09-21T05:30:59.370 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.oo4ySIemxC 2024-09-21T05:30:59.509 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.oo4ySIemxC is 6KB 2024-09-21T05:30:59.556 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.oo4ySIemxC 2024-09-21T05:30:59.571 DEBUG:teuthology.orchestra.run.smithi134:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T05:30:59.634 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T05:30:59.634 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T05:30:59.634 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T05:30:59.634 INFO:teuthology.orchestra.run.smithi134.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T05:30:59.635 DEBUG:teuthology.orchestra.run.smithi134:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T05:30:59.782 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-09-21T05:31:27.615 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T05:31:27.616 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-21T05:31:27.617 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:31:29.787 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T05:31:29.787 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T05:31:29.788 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:32:27.655 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T05:32:29.793 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T05:32:36.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-21T05:32:36.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:32:37.291 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-21T05:32:37.925 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-21T05:32:37.926 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T05:32:37.926 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-09-21T05:32:37.973 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-508.el9.x86_64 2024-09-21T05:32:37.973 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T05:32:37.974 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:32:37.974 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T05:32:38.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T05:32:38.794 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:32:38.975 DEBUG:teuthology.parallel:result is None 2024-09-21T05:32:39.207 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-21T05:32:39.669 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T05:32:39.670 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-508.el9.x86_64"... 2024-09-21T05:32:39.670 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-21T05:32:39.684 INFO:teuthology.orchestra.run.smithi134.stdout:5.14.0-508.el9.x86_64 2024-09-21T05:32:39.684 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T05:32:39.684 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T05:32:39.684 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T05:32:40.686 DEBUG:teuthology.parallel:result is None 2024-09-21T05:32:40.686 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T05:32:40.695 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T05:32:40.695 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:32:40.698 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T05:32:40.716 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T05:32:40.727 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T05:32:40.741 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T05:32:40.741 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:32:40.758 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T05:32:40.800 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T05:32:40.808 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T05:32:40.809 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:32:40.836 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T05:32:40.906 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:32:40.915 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:32:40.936 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:32:40.936 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T05:32:40.938 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T05:32:40.946 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T05:32:40.946 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:32:40.949 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T05:32:40.977 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T05:32:40.986 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T05:32:40.986 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:32:41.021 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T05:32:41.037 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:32:41.119 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:32:41.177 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:32:41.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:32:41.307 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:32:41.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:32:41.375 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:32:41.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T05:32:41.464 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:32:41.548 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T05:32:41.594 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:32:41.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T05:32:41.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-09-21T05:32:41.664 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-21T05:32:41.707 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:32:41.756 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:32:42.119 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T05:32:42.129 INFO:teuthology.task.internal:Starting timer... 2024-09-21T05:32:42.129 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T05:32:42.150 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T05:32:42.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-09-21T05:32:42.199 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-09-21T05:32:42.201 DEBUG:teuthology.orchestra.run.smithi134:> sudo service auditd rotate 2024-09-21T05:32:42.294 INFO:teuthology.orchestra.run.smithi134.stdout:Rotating logs: 2024-09-21T05:32:42.296 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T05:32:42.296 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-09-21T05:32:42.323 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-09-21T05:32:42.324 DEBUG:teuthology.orchestra.run.smithi134:> /usr/sbin/getenforce 2024-09-21T05:32:42.343 INFO:teuthology.orchestra.run.smithi134.stdout:Permissive 2024-09-21T05:32:42.343 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi134.front.sepia.ceph.com': 'permissive'} 2024-09-21T05:32:42.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T05:32:42.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:32:42.392 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T05:32:42.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:32:42.417 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T05:32:42.417 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:32:42.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo /usr/sbin/setenforce permissive 2024-09-21T05:32:42.579 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T05:32:42.588 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T05:32:42.597 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T05:32:42.598 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T05:36:27.761 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-09-21T05:36:27.762 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-21T05:36:27.762 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:36:27.849 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-09-21T05:36:27.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-09-21T05:36:27.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T05:36:27.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T05:36:28.002 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-21T05:36:28.086 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-21T05:36:28.086 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T05:36:28.098 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T05:36:28.098 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:36:28.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:36:28.101 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T05:36:28.101 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:36:28.130 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:36:28.146 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:36:28.156 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T05:36:28.171 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T05:36:28.172 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-09-21T05:36:28.184 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-09-21T05:36:28.196 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: ntpd: command not found 2024-09-21T05:36:28.199 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:36:28.207 INFO:teuthology.orchestra.run.smithi134.stdout:506 Cannot talk to daemon 2024-09-21T05:36:28.214 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:36:28.221 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T05:36:28.235 INFO:teuthology.orchestra.run.smithi134.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T05:36:28.279 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:36:28.283 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:36:28.283 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-21T05:36:28.283 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.283 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.283 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.283 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.290 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:36:28.293 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:36:28.293 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-21T05:36:28.293 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.293 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.293 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.294 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:36:28.294 INFO:teuthology.run_tasks:Running task install... 2024-09-21T05:36:28.302 DEBUG:teuthology.task.install:project ceph 2024-09-21T05:36:28.302 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T05:36:28.303 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T05:36:28.303 INFO:teuthology.task.install:Using flavor: default 2024-09-21T05:36:28.308 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T05:36:28.308 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T05:36:28.308 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:36:28.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:36:28.310 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T05:36:28.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:36:28.448 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:36:28.448 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:36:28.457 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T05:36:28.457 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T05:36:28.560 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:36:28.560 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:36:28.560 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:36:28.564 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T05:36:28.565 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:36:28.565 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T05:36:28.587 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:36:28.587 DEBUG:teuthology.orchestra.run.smithi022:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:36:28.592 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T05:36:28.592 DEBUG:teuthology.orchestra.run.smithi134:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T05:36:28.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:36:28.657 DEBUG:teuthology.orchestra.run.smithi134:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T05:36:28.729 DEBUG:teuthology.orchestra.run.smithi022:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:36:28.731 DEBUG:teuthology.orchestra.run.smithi134:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T05:36:28.794 INFO:teuthology.orchestra.run.smithi022.stdout:check_obsoletes = 1 2024-09-21T05:36:28.796 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum clean all 2024-09-21T05:36:28.796 INFO:teuthology.orchestra.run.smithi134.stdout:check_obsoletes = 1 2024-09-21T05:36:28.797 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum clean all 2024-09-21T05:36:29.034 INFO:teuthology.orchestra.run.smithi022.stdout:45 files removed 2024-09-21T05:36:29.035 INFO:teuthology.orchestra.run.smithi134.stdout:45 files removed 2024-09-21T05:36:29.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:36:29.063 DEBUG:teuthology.orchestra.run.smithi134:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T05:36:29.603 INFO:teuthology.orchestra.run.smithi134.stdout:ceph packages for x86_64 331 kB/s | 84 kB 00:00 2024-09-21T05:36:29.639 INFO:teuthology.orchestra.run.smithi022.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-21T05:36:29.879 INFO:teuthology.orchestra.run.smithi134.stdout:ceph noarch packages 105 kB/s | 24 kB 00:00 2024-09-21T05:36:29.939 INFO:teuthology.orchestra.run.smithi022.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T05:36:30.171 INFO:teuthology.orchestra.run.smithi134.stdout:ceph source packages 7.5 kB/s | 1.9 kB 00:00 2024-09-21T05:36:30.215 INFO:teuthology.orchestra.run.smithi022.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-21T05:36:31.940 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.3 MB 00:01 2024-09-21T05:36:32.163 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.3 MB 00:01 2024-09-21T05:36:36.224 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 20 MB 00:03 2024-09-21T05:36:41.724 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-21T05:36:43.184 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 120 kB/s | 19 kB 00:00 2024-09-21T05:36:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-21T05:36:49.696 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T05:36:50.304 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 20 MB 00:17 2024-09-21T05:36:51.974 INFO:teuthology.orchestra.run.smithi022.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:36:51.974 INFO:teuthology.orchestra.run.smithi022.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:36:51.986 INFO:teuthology.orchestra.run.smithi022.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:36:51.987 INFO:teuthology.orchestra.run.smithi022.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:36:51.988 INFO:teuthology.orchestra.run.smithi022.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:36:51.989 INFO:teuthology.orchestra.run.smithi022.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:36:51.991 INFO:teuthology.orchestra.run.smithi022.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:36:51.993 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:36:51.994 INFO:teuthology.orchestra.run.smithi022.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:36:52.062 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-09-21T05:36:52.077 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-21T05:36:52.077 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-09-21T05:36:52.077 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-21T05:36:52.077 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-09-21T05:36:52.077 INFO:teuthology.orchestra.run.smithi022.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:36:52.078 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:36:52.079 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrading: 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:36:52.080 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:36:52.081 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:36:52.082 INFO:teuthology.orchestra.run.smithi022.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:36:52.083 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:36:52.084 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:36:52.085 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:36:52.086 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:36:52.087 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:36:52.088 INFO:teuthology.orchestra.run.smithi022.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout:Installing weak dependencies: 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout:======================================================================================= 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout:Install 149 Packages 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout:Upgrade 8 Packages 2024-09-21T05:36:52.089 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:36:52.090 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 228 M 2024-09-21T05:36:52.090 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-09-21T05:36:52.522 INFO:teuthology.orchestra.run.smithi022.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-21T05:36:52.697 INFO:teuthology.orchestra.run.smithi022.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 4.9 MB/s | 873 kB 00:00 2024-09-21T05:36:52.764 INFO:teuthology.orchestra.run.smithi022.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T05:36:52.914 INFO:teuthology.orchestra.run.smithi022.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T05:36:52.988 INFO:teuthology.orchestra.run.smithi022.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 10 MB/s | 2.3 MB 00:00 2024-09-21T05:36:53.040 INFO:teuthology.orchestra.run.smithi022.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 12 MB/s | 1.5 MB 00:00 2024-09-21T05:36:53.365 INFO:teuthology.orchestra.run.smithi022.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.0 MB 00:00 2024-09-21T05:36:53.565 INFO:teuthology.orchestra.run.smithi022.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T05:36:53.607 INFO:teuthology.orchestra.run.smithi022.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 606 kB/s | 25 kB 00:00 2024-09-21T05:36:53.999 INFO:teuthology.orchestra.run.smithi022.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 16 MB/s | 10 MB 00:00 2024-09-21T05:36:54.191 INFO:teuthology.orchestra.run.smithi022.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T05:36:54.216 INFO:teuthology.orchestra.run.smithi022.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.0 MB/s | 226 kB 00:00 2024-09-21T05:36:54.241 INFO:teuthology.orchestra.run.smithi022.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 540 kB/s | 27 kB 00:00 2024-09-21T05:36:54.267 INFO:teuthology.orchestra.run.smithi022.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 633 kB/s | 32 kB 00:00 2024-09-21T05:36:54.300 INFO:teuthology.orchestra.run.smithi022.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 13 MB/s | 766 kB 00:00 2024-09-21T05:36:54.325 INFO:teuthology.orchestra.run.smithi022.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.6 MB/s | 158 kB 00:00 2024-09-21T05:36:54.351 INFO:teuthology.orchestra.run.smithi022.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T05:36:54.384 INFO:teuthology.orchestra.run.smithi022.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T05:36:54.443 INFO:teuthology.orchestra.run.smithi022.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 775 kB/s | 45 kB 00:00 2024-09-21T05:36:54.519 INFO:teuthology.orchestra.run.smithi022.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.9 MB/s | 148 kB 00:00 2024-09-21T05:36:54.577 INFO:teuthology.orchestra.run.smithi022.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.7 MB/s | 161 kB 00:00 2024-09-21T05:36:54.644 INFO:teuthology.orchestra.run.smithi022.stdout:(22/157): librgw2-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.3 MB 00:00 2024-09-21T05:36:54.669 INFO:teuthology.orchestra.run.smithi022.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.4 MB/s | 321 kB 00:00 2024-09-21T05:36:54.703 INFO:teuthology.orchestra.run.smithi022.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T05:36:54.720 INFO:teuthology.orchestra.run.smithi022.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T05:36:54.745 INFO:teuthology.orchestra.run.smithi022.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 86 kB 00:00 2024-09-21T05:36:54.804 INFO:teuthology.orchestra.run.smithi022.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T05:36:54.896 INFO:teuthology.orchestra.run.smithi022.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 18 MB/s | 3.1 MB 00:00 2024-09-21T05:36:54.921 INFO:teuthology.orchestra.run.smithi022.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 293 kB/s | 34 kB 00:00 2024-09-21T05:36:54.946 INFO:teuthology.orchestra.run.smithi022.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T05:36:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.5 MB 00:01 2024-09-21T05:36:55.405 INFO:teuthology.orchestra.run.smithi022.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 16 MB/s | 7.4 MB 00:00 2024-09-21T05:36:55.472 INFO:teuthology.orchestra.run.smithi022.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 4.3 MB/s | 292 kB 00:00 2024-09-21T05:36:55.856 INFO:teuthology.orchestra.run.smithi022.stdout:(33/157): ceph-test-19.3.0-5120.g18432218.el9.x 18 MB/s | 40 MB 00:02 2024-09-21T05:36:55.872 INFO:teuthology.orchestra.run.smithi022.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 123 kB/s | 49 kB 00:00 2024-09-21T05:36:55.890 INFO:teuthology.orchestra.run.smithi022.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 494 kB/s | 16 kB 00:00 2024-09-21T05:36:55.923 INFO:teuthology.orchestra.run.smithi022.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 5.6 MB/s | 285 kB 00:00 2024-09-21T05:36:56.090 INFO:teuthology.orchestra.run.smithi022.stdout:(37/157): ceph-mgr-dashboard-19.3.0-5120.g18432 12 MB/s | 14 MB 00:01 2024-09-21T05:36:56.115 INFO:teuthology.orchestra.run.smithi022.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 96 kB/s | 18 kB 00:00 2024-09-21T05:36:56.140 INFO:teuthology.orchestra.run.smithi022.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.1 MB/s | 784 kB 00:00 2024-09-21T05:36:56.257 INFO:teuthology.orchestra.run.smithi022.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-21T05:36:56.308 INFO:teuthology.orchestra.run.smithi022.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-21T05:36:56.358 INFO:teuthology.orchestra.run.smithi022.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-21T05:36:56.391 INFO:teuthology.orchestra.run.smithi022.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-21T05:36:56.417 INFO:teuthology.orchestra.run.smithi022.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T05:36:56.450 INFO:teuthology.orchestra.run.smithi022.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-21T05:36:56.476 INFO:teuthology.orchestra.run.smithi022.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-21T05:36:56.518 INFO:teuthology.orchestra.run.smithi022.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-21T05:36:56.543 INFO:teuthology.orchestra.run.smithi022.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-09-21T05:36:56.847 INFO:teuthology.orchestra.run.smithi134.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-21T05:36:57.210 INFO:teuthology.orchestra.run.smithi022.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 160 kB/s | 107 kB 00:00 2024-09-21T05:36:57.419 INFO:teuthology.orchestra.run.smithi022.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-21T05:36:57.444 INFO:teuthology.orchestra.run.smithi022.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 712 kB/s | 689 kB 00:00 2024-09-21T05:36:57.769 INFO:teuthology.orchestra.run.smithi022.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-21T05:36:57.960 INFO:teuthology.orchestra.run.smithi134.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T05:36:58.028 INFO:teuthology.orchestra.run.smithi022.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 57 kB/s | 33 kB 00:00 2024-09-21T05:36:58.194 INFO:teuthology.orchestra.run.smithi022.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 404 kB/s | 313 kB 00:00 2024-09-21T05:36:58.536 INFO:teuthology.orchestra.run.smithi022.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-21T05:36:58.670 INFO:teuthology.orchestra.run.smithi022.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-21T05:36:58.804 INFO:teuthology.orchestra.run.smithi022.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-21T05:36:59.354 INFO:teuthology.orchestra.run.smithi022.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 209 kB/s | 171 kB 00:00 2024-09-21T05:36:59.379 INFO:teuthology.orchestra.run.smithi022.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 226 kB/s | 160 kB 00:00 2024-09-21T05:36:59.438 INFO:teuthology.orchestra.run.smithi022.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-21T05:36:59.979 INFO:teuthology.orchestra.run.smithi022.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-21T05:37:00.322 INFO:teuthology.orchestra.run.smithi022.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 685 kB/s | 662 kB 00:00 2024-09-21T05:37:00.355 INFO:teuthology.orchestra.run.smithi022.stdout:(63/157): libtool-2.4.6-46.el9.x86_64.rpm 596 kB/s | 581 kB 00:00 2024-09-21T05:37:00.564 INFO:teuthology.orchestra.run.smithi022.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-21T05:37:01.089 INFO:teuthology.orchestra.run.smithi022.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 381 kB/s | 292 kB 00:00 2024-09-21T05:37:01.114 INFO:teuthology.orchestra.run.smithi022.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 248 kB/s | 188 kB 00:00 2024-09-21T05:37:01.139 INFO:teuthology.orchestra.run.smithi022.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-21T05:37:02.106 INFO:teuthology.orchestra.run.smithi022.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 1.6 MB/s | 1.6 MB 00:00 2024-09-21T05:37:02.315 INFO:teuthology.orchestra.run.smithi022.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.0 MB/s | 4.9 MB 00:01 2024-09-21T05:37:02.398 INFO:teuthology.orchestra.run.smithi022.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-21T05:37:03.090 INFO:teuthology.orchestra.run.smithi022.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 925 kB/s | 910 kB 00:00 2024-09-21T05:37:03.115 INFO:teuthology.orchestra.run.smithi022.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 172 kB/s | 137 kB 00:00 2024-09-21T05:37:03.399 INFO:teuthology.orchestra.run.smithi134.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-21T05:37:03.557 INFO:teuthology.orchestra.run.smithi022.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-21T05:37:03.757 INFO:teuthology.orchestra.run.smithi022.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 136 kB/s | 87 kB 00:00 2024-09-21T05:37:04.033 INFO:teuthology.orchestra.run.smithi022.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-21T05:37:04.433 INFO:teuthology.orchestra.run.smithi022.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 91 kB/s | 61 kB 00:00 2024-09-21T05:37:04.750 INFO:teuthology.orchestra.run.smithi022.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.0 MB/s | 6.0 MB 00:01 2024-09-21T05:37:04.842 INFO:teuthology.orchestra.run.smithi022.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 303 kB/s | 245 kB 00:00 2024-09-21T05:37:05.234 INFO:teuthology.orchestra.run.smithi022.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 310 kB/s | 248 kB 00:00 2024-09-21T05:37:05.309 INFO:teuthology.orchestra.run.smithi022.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 46 kB/s | 26 kB 00:00 2024-09-21T05:37:05.359 INFO:teuthology.orchestra.run.smithi022.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 37 kB/s | 19 kB 00:00 2024-09-21T05:37:05.595 INFO:teuthology.orchestra.run.smithi134.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:37:05.596 INFO:teuthology.orchestra.run.smithi134.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T05:37:05.609 INFO:teuthology.orchestra.run.smithi134.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T05:37:05.610 INFO:teuthology.orchestra.run.smithi134.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T05:37:05.611 INFO:teuthology.orchestra.run.smithi134.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T05:37:05.612 INFO:teuthology.orchestra.run.smithi134.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T05:37:05.614 INFO:teuthology.orchestra.run.smithi134.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T05:37:05.617 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T05:37:05.618 INFO:teuthology.orchestra.run.smithi134.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T05:37:05.681 INFO:teuthology.orchestra.run.smithi134.stdout:Dependencies resolved. 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: Package Arch Version Repository Size 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout:Installing: 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T05:37:05.704 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T05:37:05.705 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T05:37:05.706 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrading: 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T05:37:05.707 INFO:teuthology.orchestra.run.smithi134.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout:Installing dependencies: 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T05:37:05.708 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T05:37:05.709 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T05:37:05.710 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T05:37:05.711 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T05:37:05.712 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T05:37:05.713 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T05:37:05.714 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T05:37:05.715 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:Installing weak dependencies: 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction Summary 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:======================================================================================= 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:Install 149 Packages 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:Upgrade 8 Packages 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:Total download size: 228 M 2024-09-21T05:37:05.716 INFO:teuthology.orchestra.run.smithi134.stdout:Downloading Packages: 2024-09-21T05:37:05.926 INFO:teuthology.orchestra.run.smithi022.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-09-21T05:37:05.952 INFO:teuthology.orchestra.run.smithi022.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 240 kB/s | 172 kB 00:00 2024-09-21T05:37:06.225 INFO:teuthology.orchestra.run.smithi134.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 52 kB/s | 6.6 kB 00:00 2024-09-21T05:37:06.467 INFO:teuthology.orchestra.run.smithi134.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.5 MB/s | 873 kB 00:00 2024-09-21T05:37:06.577 INFO:teuthology.orchestra.run.smithi022.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-21T05:37:06.592 INFO:teuthology.orchestra.run.smithi134.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-21T05:37:06.617 INFO:teuthology.orchestra.run.smithi134.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 987 kB/s | 148 kB 00:00 2024-09-21T05:37:06.702 INFO:teuthology.orchestra.run.smithi022.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 296 kB/s | 222 kB 00:00 2024-09-21T05:37:06.743 INFO:teuthology.orchestra.run.smithi134.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 15 MB/s | 2.3 MB 00:00 2024-09-21T05:37:06.761 INFO:teuthology.orchestra.run.smithi022.stdout:(86/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 300 kB/s | 250 kB 00:00 2024-09-21T05:37:06.860 INFO:teuthology.orchestra.run.smithi134.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-09-21T05:37:07.144 INFO:teuthology.orchestra.run.smithi134.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 19 MB/s | 20 MB 00:01 2024-09-21T05:37:07.203 INFO:teuthology.orchestra.run.smithi022.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 123 kB/s | 77 kB 00:00 2024-09-21T05:37:07.210 INFO:teuthology.orchestra.run.smithi134.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-21T05:37:07.270 INFO:teuthology.orchestra.run.smithi022.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 74 kB/s | 42 kB 00:00 2024-09-21T05:37:07.270 INFO:teuthology.orchestra.run.smithi134.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 431 kB/s | 25 kB 00:00 2024-09-21T05:37:07.536 INFO:teuthology.orchestra.run.smithi022.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 347 kB/s | 269 kB 00:00 2024-09-21T05:37:07.544 INFO:teuthology.orchestra.run.smithi134.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 26 MB/s | 10 MB 00:00 2024-09-21T05:37:07.611 INFO:teuthology.orchestra.run.smithi134.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T05:37:07.686 INFO:teuthology.orchestra.run.smithi134.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 361 kB/s | 27 kB 00:00 2024-09-21T05:37:07.762 INFO:teuthology.orchestra.run.smithi134.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 424 kB/s | 32 kB 00:00 2024-09-21T05:37:07.862 INFO:teuthology.orchestra.run.smithi134.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 7.5 MB/s | 766 kB 00:00 2024-09-21T05:37:07.945 INFO:teuthology.orchestra.run.smithi134.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 1.9 MB/s | 158 kB 00:00 2024-09-21T05:37:07.970 INFO:teuthology.orchestra.run.smithi022.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 207 kB/s | 159 kB 00:00 2024-09-21T05:37:08.070 INFO:teuthology.orchestra.run.smithi022.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 348 kB/s | 279 kB 00:00 2024-09-21T05:37:08.145 INFO:teuthology.orchestra.run.smithi022.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-21T05:37:08.146 INFO:teuthology.orchestra.run.smithi134.stdout:(16/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 14 MB/s | 18 MB 00:01 2024-09-21T05:37:08.171 INFO:teuthology.orchestra.run.smithi134.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 562 kB/s | 127 kB 00:00 2024-09-21T05:37:08.204 INFO:teuthology.orchestra.run.smithi134.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T05:37:08.272 INFO:teuthology.orchestra.run.smithi134.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 676 kB/s | 45 kB 00:00 2024-09-21T05:37:08.388 INFO:teuthology.orchestra.run.smithi134.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 24 MB/s | 5.3 MB 00:00 2024-09-21T05:37:08.414 INFO:teuthology.orchestra.run.smithi134.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 1.0 MB/s | 148 kB 00:00 2024-09-21T05:37:08.439 INFO:teuthology.orchestra.run.smithi134.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.2 MB/s | 161 kB 00:00 2024-09-21T05:37:08.518 INFO:teuthology.orchestra.run.smithi134.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 3.0 MB/s | 321 kB 00:00 2024-09-21T05:37:08.571 INFO:teuthology.orchestra.run.smithi022.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 89 kB/s | 54 kB 00:00 2024-09-21T05:37:08.589 INFO:teuthology.orchestra.run.smithi134.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 2.0 MB/s | 307 kB 00:00 2024-09-21T05:37:08.615 INFO:teuthology.orchestra.run.smithi134.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.0 MB/s | 99 kB 00:00 2024-09-21T05:37:08.640 INFO:teuthology.orchestra.run.smithi134.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T05:37:08.712 INFO:teuthology.orchestra.run.smithi022.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-21T05:37:08.716 INFO:teuthology.orchestra.run.smithi134.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.2 MB/s | 170 kB 00:00 2024-09-21T05:37:08.782 INFO:teuthology.orchestra.run.smithi134.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 19 MB/s | 3.1 MB 00:00 2024-09-21T05:37:08.807 INFO:teuthology.orchestra.run.smithi134.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 374 kB/s | 34 kB 00:00 2024-09-21T05:37:08.833 INFO:teuthology.orchestra.run.smithi134.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T05:37:09.204 INFO:teuthology.orchestra.run.smithi022.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 70 kB/s | 44 kB 00:00 2024-09-21T05:37:09.525 INFO:teuthology.orchestra.run.smithi134.stdout:(31/157): ceph-test-19.3.0-5120.g18432218.el9.x 18 MB/s | 40 MB 00:02 2024-09-21T05:37:09.625 INFO:teuthology.orchestra.run.smithi134.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 9.3 MB/s | 7.4 MB 00:00 2024-09-21T05:37:09.650 INFO:teuthology.orchestra.run.smithi134.stdout:(33/157): ceph-mgr-modules-core-19.3.0-5120.g18 2.3 MB/s | 292 kB 00:00 2024-09-21T05:37:09.676 INFO:teuthology.orchestra.run.smithi134.stdout:(34/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 980 kB/s | 49 kB 00:00 2024-09-21T05:37:09.701 INFO:teuthology.orchestra.run.smithi134.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 327 kB/s | 16 kB 00:00 2024-09-21T05:37:09.735 INFO:teuthology.orchestra.run.smithi134.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T05:37:09.769 INFO:teuthology.orchestra.run.smithi022.stdout:(96/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.6 MB/s | 16 MB 00:01 2024-09-21T05:37:09.796 INFO:teuthology.orchestra.run.smithi022.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 204 kB/s | 221 kB 00:01 2024-09-21T05:37:09.894 INFO:teuthology.orchestra.run.smithi134.stdout:(37/157): ceph-mgr-dashboard-19.3.0-5120.g18432 13 MB/s | 14 MB 00:01 2024-09-21T05:37:09.919 INFO:teuthology.orchestra.run.smithi134.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 100 kB/s | 18 kB 00:00 2024-09-21T05:37:09.952 INFO:teuthology.orchestra.run.smithi134.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.1 MB/s | 784 kB 00:00 2024-09-21T05:37:10.044 INFO:teuthology.orchestra.run.smithi134.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-21T05:37:10.072 INFO:teuthology.orchestra.run.smithi022.stdout:(98/157): qatlib-service-24.02.0-1.el9.x86_64.r 41 kB/s | 36 kB 00:00 2024-09-21T05:37:10.086 INFO:teuthology.orchestra.run.smithi134.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-21T05:37:10.128 INFO:teuthology.orchestra.run.smithi134.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T05:37:10.162 INFO:teuthology.orchestra.run.smithi134.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-21T05:37:10.187 INFO:teuthology.orchestra.run.smithi134.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-21T05:37:10.237 INFO:teuthology.orchestra.run.smithi134.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-21T05:37:10.262 INFO:teuthology.orchestra.run.smithi134.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-09-21T05:37:10.288 INFO:teuthology.orchestra.run.smithi134.stdout:(47/157): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-09-21T05:37:10.321 INFO:teuthology.orchestra.run.smithi134.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-21T05:37:10.355 INFO:teuthology.orchestra.run.smithi022.stdout:(99/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-21T05:37:10.447 INFO:teuthology.orchestra.run.smithi134.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 107 kB 00:00 2024-09-21T05:37:10.480 INFO:teuthology.orchestra.run.smithi134.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-21T05:37:10.505 INFO:teuthology.orchestra.run.smithi134.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-21T05:37:10.539 INFO:teuthology.orchestra.run.smithi134.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 943 kB 00:00 2024-09-21T05:37:10.564 INFO:teuthology.orchestra.run.smithi134.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-09-21T05:37:10.572 INFO:teuthology.orchestra.run.smithi022.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-21T05:37:10.598 INFO:teuthology.orchestra.run.smithi134.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-21T05:37:10.623 INFO:teuthology.orchestra.run.smithi134.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-21T05:37:10.648 INFO:teuthology.orchestra.run.smithi134.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T05:37:10.674 INFO:teuthology.orchestra.run.smithi134.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 3.3 MB/s | 171 kB 00:00 2024-09-21T05:37:10.731 INFO:teuthology.orchestra.run.smithi022.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-09-21T05:37:10.732 INFO:teuthology.orchestra.run.smithi134.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-21T05:37:10.766 INFO:teuthology.orchestra.run.smithi134.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-21T05:37:10.783 INFO:teuthology.orchestra.run.smithi134.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 416 kB/s | 45 kB 00:00 2024-09-21T05:37:10.816 INFO:teuthology.orchestra.run.smithi134.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-21T05:37:10.842 INFO:teuthology.orchestra.run.smithi134.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-21T05:37:10.858 INFO:teuthology.orchestra.run.smithi134.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T05:37:10.892 INFO:teuthology.orchestra.run.smithi134.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-21T05:37:10.918 INFO:teuthology.orchestra.run.smithi134.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-21T05:37:10.943 INFO:teuthology.orchestra.run.smithi134.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-21T05:37:10.968 INFO:teuthology.orchestra.run.smithi134.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-21T05:37:11.023 INFO:teuthology.orchestra.run.smithi022.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 144 kB/s | 96 kB 00:00 2024-09-21T05:37:11.060 INFO:teuthology.orchestra.run.smithi134.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 35 MB/s | 4.9 MB 00:00 2024-09-21T05:37:11.144 INFO:teuthology.orchestra.run.smithi134.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-09-21T05:37:11.186 INFO:teuthology.orchestra.run.smithi134.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-21T05:37:11.206 INFO:teuthology.orchestra.run.smithi022.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-21T05:37:11.211 INFO:teuthology.orchestra.run.smithi134.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T05:37:11.244 INFO:teuthology.orchestra.run.smithi134.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 4.8 MB/s | 910 kB 00:00 2024-09-21T05:37:11.278 INFO:teuthology.orchestra.run.smithi134.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-21T05:37:11.303 INFO:teuthology.orchestra.run.smithi134.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T05:37:11.323 INFO:teuthology.orchestra.run.smithi022.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-09-21T05:37:11.345 INFO:teuthology.orchestra.run.smithi134.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T05:37:11.357 INFO:teuthology.orchestra.run.smithi022.stdout:(105/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-09-21T05:37:11.399 INFO:teuthology.orchestra.run.smithi022.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-21T05:37:11.421 INFO:teuthology.orchestra.run.smithi134.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-21T05:37:11.432 INFO:teuthology.orchestra.run.smithi022.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 622 kB/s | 19 kB 00:00 2024-09-21T05:37:11.454 INFO:teuthology.orchestra.run.smithi134.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-21T05:37:11.466 INFO:teuthology.orchestra.run.smithi022.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-21T05:37:11.488 INFO:teuthology.orchestra.run.smithi134.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-21T05:37:11.492 INFO:teuthology.orchestra.run.smithi022.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T05:37:11.513 INFO:teuthology.orchestra.run.smithi134.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-21T05:37:11.516 INFO:teuthology.orchestra.run.smithi022.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T05:37:11.538 INFO:teuthology.orchestra.run.smithi134.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T05:37:11.542 INFO:teuthology.orchestra.run.smithi022.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T05:37:11.567 INFO:teuthology.orchestra.run.smithi022.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-21T05:37:11.572 INFO:teuthology.orchestra.run.smithi134.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-21T05:37:11.597 INFO:teuthology.orchestra.run.smithi134.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 595 kB/s | 35 kB 00:00 2024-09-21T05:37:11.617 INFO:teuthology.orchestra.run.smithi022.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-21T05:37:11.659 INFO:teuthology.orchestra.run.smithi022.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T05:37:11.672 INFO:teuthology.orchestra.run.smithi134.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T05:37:11.685 INFO:teuthology.orchestra.run.smithi022.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T05:37:11.698 INFO:teuthology.orchestra.run.smithi134.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-09-21T05:37:11.710 INFO:teuthology.orchestra.run.smithi022.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:37:11.723 INFO:teuthology.orchestra.run.smithi134.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-21T05:37:11.727 INFO:teuthology.orchestra.run.smithi022.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 854 kB/s | 14 kB 00:00 2024-09-21T05:37:11.748 INFO:teuthology.orchestra.run.smithi134.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T05:37:11.752 INFO:teuthology.orchestra.run.smithi022.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-21T05:37:11.774 INFO:teuthology.orchestra.run.smithi134.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T05:37:11.807 INFO:teuthology.orchestra.run.smithi134.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T05:37:11.812 INFO:teuthology.orchestra.run.smithi022.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T05:37:11.832 INFO:teuthology.orchestra.run.smithi134.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-21T05:37:11.836 INFO:teuthology.orchestra.run.smithi022.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T05:37:11.861 INFO:teuthology.orchestra.run.smithi022.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-21T05:37:11.866 INFO:teuthology.orchestra.run.smithi134.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-21T05:37:11.887 INFO:teuthology.orchestra.run.smithi022.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T05:37:11.891 INFO:teuthology.orchestra.run.smithi134.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 871 kB/s | 51 kB 00:00 2024-09-21T05:37:11.912 INFO:teuthology.orchestra.run.smithi022.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-21T05:37:11.916 INFO:teuthology.orchestra.run.smithi134.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-21T05:37:11.937 INFO:teuthology.orchestra.run.smithi022.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-21T05:37:11.962 INFO:teuthology.orchestra.run.smithi022.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-21T05:37:11.988 INFO:teuthology.orchestra.run.smithi022.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 926 kB/s | 23 kB 00:00 2024-09-21T05:37:11.992 INFO:teuthology.orchestra.run.smithi134.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T05:37:12.013 INFO:teuthology.orchestra.run.smithi022.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-21T05:37:12.017 INFO:teuthology.orchestra.run.smithi134.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-09-21T05:37:12.042 INFO:teuthology.orchestra.run.smithi134.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-21T05:37:12.046 INFO:teuthology.orchestra.run.smithi022.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 589 kB/s | 20 kB 00:00 2024-09-21T05:37:12.067 INFO:teuthology.orchestra.run.smithi134.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T05:37:12.101 INFO:teuthology.orchestra.run.smithi134.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 612 kB/s | 36 kB 00:00 2024-09-21T05:37:12.105 INFO:teuthology.orchestra.run.smithi022.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-21T05:37:12.127 INFO:teuthology.orchestra.run.smithi134.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-09-21T05:37:12.130 INFO:teuthology.orchestra.run.smithi022.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T05:37:12.152 INFO:teuthology.orchestra.run.smithi134.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T05:37:12.155 INFO:teuthology.orchestra.run.smithi022.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:37:12.177 INFO:teuthology.orchestra.run.smithi134.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T05:37:12.181 INFO:teuthology.orchestra.run.smithi022.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T05:37:12.202 INFO:teuthology.orchestra.run.smithi134.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T05:37:12.214 INFO:teuthology.orchestra.run.smithi022.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-21T05:37:12.227 INFO:teuthology.orchestra.run.smithi134.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T05:37:12.240 INFO:teuthology.orchestra.run.smithi022.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-21T05:37:12.253 INFO:teuthology.orchestra.run.smithi134.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-21T05:37:12.265 INFO:teuthology.orchestra.run.smithi022.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T05:37:12.290 INFO:teuthology.orchestra.run.smithi022.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-21T05:37:12.316 INFO:teuthology.orchestra.run.smithi022.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-21T05:37:12.341 INFO:teuthology.orchestra.run.smithi022.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-21T05:37:12.345 INFO:teuthology.orchestra.run.smithi134.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-21T05:37:12.383 INFO:teuthology.orchestra.run.smithi022.stdout:(139/157): protobuf-compiler-3.14.0-13.el9.x86_ 734 kB/s | 863 kB 00:01 2024-09-21T05:37:12.417 INFO:teuthology.orchestra.run.smithi022.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-09-21T05:37:12.450 INFO:teuthology.orchestra.run.smithi022.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-21T05:37:12.476 INFO:teuthology.orchestra.run.smithi022.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-21T05:37:12.501 INFO:teuthology.orchestra.run.smithi022.stdout:(143/157): python3-tempora-5.0.0-2.el9.noarch.r 304 kB/s | 36 kB 00:00 2024-09-21T05:37:12.512 INFO:teuthology.orchestra.run.smithi134.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-21T05:37:12.526 INFO:teuthology.orchestra.run.smithi022.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T05:37:12.560 INFO:teuthology.orchestra.run.smithi022.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T05:37:12.562 INFO:teuthology.orchestra.run.smithi134.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-09-21T05:37:12.585 INFO:teuthology.orchestra.run.smithi022.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 340 kB/s | 20 kB 00:00 2024-09-21T05:37:12.610 INFO:teuthology.orchestra.run.smithi022.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T05:37:12.612 INFO:teuthology.orchestra.run.smithi134.stdout:(107/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-09-21T05:37:12.635 INFO:teuthology.orchestra.run.smithi022.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T05:37:12.637 INFO:teuthology.orchestra.run.smithi134.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-21T05:37:12.663 INFO:teuthology.orchestra.run.smithi134.stdout:(109/157): grpc-data-1.46.7-10.el9.noarch.rpm 197 kB/s | 19 kB 00:00 2024-09-21T05:37:12.686 INFO:teuthology.orchestra.run.smithi022.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T05:37:12.688 INFO:teuthology.orchestra.run.smithi134.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-09-21T05:37:12.713 INFO:teuthology.orchestra.run.smithi134.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-09-21T05:37:12.738 INFO:teuthology.orchestra.run.smithi134.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-21T05:37:12.764 INFO:teuthology.orchestra.run.smithi134.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-21T05:37:12.797 INFO:teuthology.orchestra.run.smithi134.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-21T05:37:12.814 INFO:teuthology.orchestra.run.smithi134.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-21T05:37:12.840 INFO:teuthology.orchestra.run.smithi134.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T05:37:12.856 INFO:teuthology.orchestra.run.smithi134.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 853 kB/s | 14 kB 00:00 2024-09-21T05:37:12.870 INFO:teuthology.orchestra.run.smithi022.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 13 MB/s | 3.4 MB 00:00 2024-09-21T05:37:12.897 INFO:teuthology.orchestra.run.smithi134.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.1 MB/s | 172 kB 00:00 2024-09-21T05:37:12.928 INFO:teuthology.orchestra.run.smithi022.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 11 MB/s | 3.2 MB 00:00 2024-09-21T05:37:12.939 INFO:teuthology.orchestra.run.smithi134.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-09-21T05:37:12.965 INFO:teuthology.orchestra.run.smithi134.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T05:37:12.990 INFO:teuthology.orchestra.run.smithi134.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-21T05:37:13.015 INFO:teuthology.orchestra.run.smithi134.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-21T05:37:13.049 INFO:teuthology.orchestra.run.smithi134.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T05:37:13.107 INFO:teuthology.orchestra.run.smithi134.stdout:(124/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-09-21T05:37:13.128 INFO:teuthology.orchestra.run.smithi022.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 204 kB/s | 41 kB 00:00 2024-09-21T05:37:13.132 INFO:teuthology.orchestra.run.smithi134.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-09-21T05:37:13.158 INFO:teuthology.orchestra.run.smithi134.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-21T05:37:13.183 INFO:teuthology.orchestra.run.smithi134.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-21T05:37:13.208 INFO:teuthology.orchestra.run.smithi134.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T05:37:13.234 INFO:teuthology.orchestra.run.smithi134.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T05:37:13.259 INFO:teuthology.orchestra.run.smithi134.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T05:37:13.278 INFO:teuthology.orchestra.run.smithi022.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.6 MB/s | 249 kB 00:00 2024-09-21T05:37:13.284 INFO:teuthology.orchestra.run.smithi134.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T05:37:13.310 INFO:teuthology.orchestra.run.smithi134.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T05:37:13.343 INFO:teuthology.orchestra.run.smithi134.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T05:37:13.360 INFO:teuthology.orchestra.run.smithi134.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-21T05:37:13.394 INFO:teuthology.orchestra.run.smithi134.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-09-21T05:37:13.444 INFO:teuthology.orchestra.run.smithi134.stdout:(136/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-21T05:37:13.461 INFO:teuthology.orchestra.run.smithi134.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 420 kB/s | 28 kB 00:00 2024-09-21T05:37:13.495 INFO:teuthology.orchestra.run.smithi134.stdout:(138/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 669 kB/s | 90 kB 00:00 2024-09-21T05:37:13.520 INFO:teuthology.orchestra.run.smithi134.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-21T05:37:13.545 INFO:teuthology.orchestra.run.smithi134.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T05:37:13.573 INFO:teuthology.orchestra.run.smithi134.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 681 kB/s | 36 kB 00:00 2024-09-21T05:37:13.654 INFO:teuthology.orchestra.run.smithi134.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 685 kB/s | 74 kB 00:00 2024-09-21T05:37:13.670 INFO:teuthology.orchestra.run.smithi022.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 1.4 MB 00:00 2024-09-21T05:37:13.688 INFO:teuthology.orchestra.run.smithi134.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 834 kB/s | 188 kB 00:00 2024-09-21T05:37:13.713 INFO:teuthology.orchestra.run.smithi134.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-21T05:37:13.738 INFO:teuthology.orchestra.run.smithi134.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-09-21T05:37:13.745 INFO:teuthology.orchestra.run.smithi022.stdout:(155/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-09-21T05:37:13.763 INFO:teuthology.orchestra.run.smithi134.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-21T05:37:13.789 INFO:teuthology.orchestra.run.smithi134.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T05:37:13.814 INFO:teuthology.orchestra.run.smithi134.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T05:37:13.864 INFO:teuthology.orchestra.run.smithi134.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T05:37:13.921 INFO:teuthology.orchestra.run.smithi022.stdout:(156/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.7 MB/s | 1.8 MB 00:01 2024-09-21T05:37:14.048 INFO:teuthology.orchestra.run.smithi134.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 13 MB/s | 3.4 MB 00:00 2024-09-21T05:37:14.168 INFO:teuthology.orchestra.run.smithi022.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.8 MB/s | 2.4 MB 00:00 2024-09-21T05:37:14.175 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:37:14.175 INFO:teuthology.orchestra.run.smithi022.stdout:Total 10 MB/s | 228 MB 00:22 2024-09-21T05:37:14.181 INFO:teuthology.orchestra.run.smithi134.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.6 MB/s | 3.2 MB 00:00 2024-09-21T05:37:14.265 INFO:teuthology.orchestra.run.smithi134.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 491 kB/s | 41 kB 00:00 2024-09-21T05:37:14.358 INFO:teuthology.orchestra.run.smithi134.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.7 MB/s | 249 kB 00:00 2024-09-21T05:37:14.457 INFO:teuthology.orchestra.run.smithi134.stdout:(154/157): NetworkManager-1.51.0-1.el9.x86_64.r 4.0 MB/s | 2.3 MB 00:00 2024-09-21T05:37:14.524 INFO:teuthology.orchestra.run.smithi134.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 3.8 MB/s | 1.8 MB 00:00 2024-09-21T05:37:14.574 INFO:teuthology.orchestra.run.smithi134.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-09-21T05:37:15.024 INFO:teuthology.orchestra.run.smithi134.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.2 MB/s | 2.4 MB 00:00 2024-09-21T05:37:15.031 INFO:teuthology.orchestra.run.smithi134.stdout:-------------------------------------------------------------------------------- 2024-09-21T05:37:15.031 INFO:teuthology.orchestra.run.smithi134.stdout:Total 24 MB/s | 228 MB 00:09 2024-09-21T05:37:15.457 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-09-21T05:37:15.629 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-09-21T05:37:15.629 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-09-21T05:37:16.255 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction check 2024-09-21T05:37:16.407 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction check succeeded. 2024-09-21T05:37:16.407 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction test 2024-09-21T05:37:17.927 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-09-21T05:37:17.928 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-09-21T05:37:18.655 INFO:teuthology.orchestra.run.smithi134.stdout:Transaction test succeeded. 2024-09-21T05:37:18.656 INFO:teuthology.orchestra.run.smithi134.stdout:Running transaction 2024-09-21T05:37:24.641 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-09-21T05:37:24.808 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:37:24.848 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:37:24.901 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:37:24.951 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:37:24.964 INFO:teuthology.orchestra.run.smithi134.stdout: Preparing : 1/1 2024-09-21T05:37:24.976 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:37:25.029 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:37:25.103 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T05:37:25.249 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T05:37:25.363 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T05:37:25.399 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:37:25.404 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T05:37:25.438 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T05:37:25.491 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T05:37:25.511 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:37:25.641 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:37:25.662 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:37:25.755 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:37:25.784 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:37:25.814 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:37:25.830 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T05:37:25.955 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:37:25.993 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:37:26.049 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:37:26.076 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:37:26.081 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:37:26.106 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:37:26.167 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:37:26.193 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:37:26.197 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:37:26.227 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:37:26.244 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:37:26.257 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T05:37:26.269 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:37:26.291 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:37:26.423 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T05:37:26.483 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T05:37:26.512 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:37:26.514 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:37:26.620 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:37:26.645 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:37:26.694 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:37:26.708 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T05:37:26.738 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:37:26.760 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T05:37:26.814 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:37:26.935 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:37:26.957 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:37:27.019 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:37:27.119 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T05:37:27.253 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T05:37:27.259 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:37:27.370 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:37:27.386 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:37:27.398 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:37:27.434 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:37:27.479 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:37:27.510 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:37:27.561 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:37:27.600 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:37:27.643 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:37:27.676 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:37:27.703 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T05:37:27.715 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:37:27.793 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:37:27.813 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T05:37:27.844 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:37:27.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T05:37:27.873 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:37:27.893 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T05:37:27.917 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:37:27.974 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T05:37:28.001 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:37:28.019 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T05:37:28.036 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:37:28.076 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T05:37:28.080 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:37:28.108 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:37:28.119 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T05:37:28.161 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T05:37:28.182 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:37:28.196 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T05:37:28.231 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:37:28.265 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T05:37:28.310 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:37:28.319 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T05:37:28.356 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T05:37:28.394 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T05:37:28.454 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:37:28.473 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T05:37:28.510 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:37:28.511 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T05:37:28.555 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T05:37:28.592 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T05:37:28.669 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T05:37:28.732 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T05:37:28.805 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T05:37:28.938 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T05:37:28.992 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T05:37:29.068 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:37:29.149 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:37:29.183 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:37:29.218 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:37:29.256 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:37:29.298 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:37:29.464 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:37:29.510 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T05:37:29.525 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:37:29.596 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T05:37:29.622 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:37:29.625 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T05:37:29.660 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T05:37:29.684 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:37:29.697 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T05:37:29.749 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T05:37:29.758 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:37:29.793 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:37:29.830 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:37:29.878 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:37:29.913 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:37:29.924 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T05:37:29.956 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:37:29.992 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T05:37:29.998 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:37:30.083 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T05:37:30.143 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:37:30.148 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T05:37:30.191 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:37:30.220 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T05:37:30.241 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:37:30.255 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T05:37:30.292 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T05:37:30.338 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T05:37:30.349 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:37:30.382 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T05:37:30.383 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:37:30.424 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:37:30.434 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T05:37:30.456 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:37:30.480 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:37:30.484 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T05:37:30.575 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:37:30.706 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:37:30.727 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T05:37:30.746 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:37:30.769 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T05:37:30.774 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:37:30.809 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:37:30.810 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T05:37:30.848 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:37:30.888 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:37:30.913 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T05:37:30.917 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:37:30.945 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T05:37:30.952 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:37:30.986 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:37:30.994 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T05:37:31.027 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T05:37:31.028 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:37:31.051 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T05:37:31.061 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:37:31.143 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T05:37:31.155 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:37:31.204 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:37:31.235 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:37:31.268 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T05:37:31.308 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T05:37:31.335 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:37:31.336 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T05:37:31.371 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T05:37:31.387 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:37:31.418 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T05:37:31.475 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T05:37:31.504 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T05:37:31.539 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T05:37:31.579 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T05:37:31.621 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T05:37:31.655 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T05:37:31.741 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T05:37:31.793 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T05:37:31.833 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T05:37:31.935 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T05:37:31.984 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T05:37:32.158 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:37:32.200 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:37:32.242 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:37:32.346 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:37:32.406 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:37:32.459 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:37:32.551 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:37:32.586 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:37:32.701 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T05:37:32.748 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T05:37:32.776 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:37:32.782 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T05:37:32.800 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:37:32.879 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:37:32.888 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T05:37:32.969 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T05:37:33.014 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T05:37:33.101 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T05:37:33.153 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T05:37:33.335 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T05:37:33.367 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T05:37:33.445 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T05:37:33.581 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:37:33.693 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:37:34.084 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T05:37:34.188 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T05:37:35.027 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:37:35.113 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:37:35.141 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:37:35.174 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:37:35.450 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T05:37:35.526 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:37:35.536 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T05:37:35.564 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T05:37:35.570 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:37:35.597 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T05:37:35.655 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:37:35.683 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:37:35.720 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:37:36.144 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T05:37:36.185 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:37:36.247 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:37:36.273 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:37:36.276 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T05:37:36.307 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T05:37:36.328 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:37:36.342 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T05:37:36.358 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:37:36.848 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T05:37:36.879 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:37:36.951 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T05:37:36.981 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T05:37:38.735 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:37:38.813 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:37:39.234 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:37:39.434 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:37:39.458 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:37:39.466 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:37:39.542 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:37:39.542 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:37:39.543 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:37:39.543 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:37:39.584 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:37:40.049 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T05:37:40.089 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:37:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T05:37:40.176 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T05:37:40.176 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T05:37:40.176 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:40.215 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /sys 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /proc 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /mnt 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /var/tmp 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /home 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /root 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout:skipping the directory /tmp 2024-09-21T05:37:51.825 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:37:52.006 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:37:52.024 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:37:52.024 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:52.024 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:37:52.025 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:37:52.187 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /sys 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /proc 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /mnt 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /var/tmp 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /home 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /root 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout:skipping the directory /tmp 2024-09-21T05:37:52.188 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:52.374 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:37:52.392 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T05:37:52.392 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:52.392 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T05:37:52.392 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:53.675 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:37:53.704 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:37:53.704 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:53.704 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:37:53.705 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:37:53.705 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:37:53.705 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:37:53.943 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:37:53.968 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:37:53.968 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:53.969 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:37:53.969 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:37:53.969 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:37:53.969 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:37:53.977 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:37:54.004 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T05:37:54.004 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:54.004 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T05:37:54.004 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:37:54.004 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T05:37:54.005 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:54.228 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:37:54.251 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T05:37:54.251 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:54.251 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T05:37:54.252 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:37:54.252 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T05:37:54.252 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:54.468 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:37:54.494 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:37:54.494 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:54.494 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:37:54.494 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:37:54.494 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:37:54.494 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:37:54.600 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:37:54.658 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:37:54.722 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:37:54.728 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:37:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T05:37:54.752 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:37:54.752 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T05:37:54.752 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:37:54.752 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T05:37:54.752 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:37:54.850 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T05:37:54.907 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T05:37:54.917 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:37:54.962 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:37:54.971 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T05:37:55.043 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:37:55.132 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:37:55.163 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T05:37:55.204 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T05:37:55.275 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:37:55.281 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T05:37:55.329 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:37:55.360 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:37:55.371 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T05:37:55.388 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:37:55.534 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T05:37:55.579 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T05:37:55.611 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T05:37:55.639 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T05:37:55.841 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:37:55.870 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:37:55.902 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:37:55.972 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:37:56.009 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:37:56.090 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:37:56.121 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T05:37:56.153 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T05:37:56.218 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T05:37:56.244 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:37:59.392 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:37:59.404 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T05:37:59.458 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:37:59.465 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:38:00.418 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:38:00.450 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:38:00.479 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T05:38:00.507 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:38:00.564 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:38:00.633 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T05:38:00.742 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:38:00.776 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:38:00.799 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T05:38:00.800 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:38:00.801 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:00.801 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:38:00.801 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:38:00.801 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:38:00.801 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:00.834 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:38:00.852 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:38:00.860 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T05:38:00.860 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:00.860 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T05:38:00.860 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:38:00.861 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T05:38:00.861 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:00.864 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:38:00.910 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:38:00.922 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T05:38:00.962 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:38:01.018 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:38:01.034 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T05:38:01.096 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T05:38:01.984 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:38:02.013 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:38:02.037 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:38:02.038 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.038 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:38:02.038 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:38:02.038 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:38:02.038 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:02.088 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:38:02.113 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T05:38:02.113 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:38:02.113 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.113 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:38:02.113 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:02.146 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:38:02.172 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T05:38:02.172 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.172 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T05:38:02.172 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:38:02.173 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T05:38:02.173 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:02.187 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:38:02.211 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:38:02.211 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.211 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:38:02.211 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:38:02.212 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:38:02.212 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:02.213 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:38:02.238 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T05:38:02.239 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.239 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T05:38:02.239 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:02.289 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:38:02.314 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T05:38:02.314 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.314 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T05:38:02.315 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:38:02.315 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T05:38:02.315 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:02.531 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:38:02.554 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:38:02.554 INFO:teuthology.orchestra.run.smithi134.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.554 INFO:teuthology.orchestra.run.smithi134.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:38:02.555 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:38:02.555 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:38:02.555 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:02.648 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:38:02.673 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T05:38:02.673 INFO:teuthology.orchestra.run.smithi022.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T05:38:02.673 INFO:teuthology.orchestra.run.smithi022.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T05:38:02.673 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:38:02.674 INFO:teuthology.orchestra.run.smithi022.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T05:38:02.674 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:06.571 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:38:06.618 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:38:06.658 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:38:06.691 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:38:06.718 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:38:06.912 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T05:38:06.916 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:38:06.953 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T05:38:06.956 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:38:06.986 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:38:07.002 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T05:38:07.016 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:38:07.033 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T05:38:07.060 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T05:38:07.062 INFO:teuthology.orchestra.run.smithi134.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:38:07.284 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T05:38:07.324 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T05:38:07.354 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T05:38:07.384 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T05:38:07.422 INFO:teuthology.orchestra.run.smithi022.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T05:38:07.667 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:38:07.710 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:38:07.730 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:38:08.056 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:38:08.094 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:38:08.116 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T05:38:08.561 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:38:08.656 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:38:08.700 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:38:08.794 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:38:08.825 INFO:teuthology.orchestra.run.smithi134.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:38:08.850 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:38:08.876 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:38:08.877 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:38:08.967 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:38:08.967 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:38:09.028 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T05:38:09.061 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:38:09.086 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:38:09.091 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:38:09.091 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:38:09.124 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T05:38:09.175 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T05:38:09.272 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T05:38:09.279 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:38:09.279 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:38:09.302 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T05:38:09.328 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T05:38:09.353 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T05:38:09.354 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:38:09.364 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:38:09.428 INFO:teuthology.orchestra.run.smithi134.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:38:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T05:38:09.446 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:38:09.514 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T05:38:09.547 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T05:38:09.553 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:38:09.553 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:38:09.749 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T05:38:09.749 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:38:09.833 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T05:38:09.903 INFO:teuthology.orchestra.run.smithi022.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:38:15.853 INFO:teuthology.orchestra.run.smithi134.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:38:15.853 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T05:38:15.853 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T05:38:15.853 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T05:38:15.853 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T05:38:15.853 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T05:38:15.854 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T05:38:15.855 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T05:38:15.871 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T05:38:15.872 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T05:38:15.873 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T05:38:15.874 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T05:38:15.875 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T05:38:15.876 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T05:38:15.877 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T05:38:15.878 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T05:38:15.879 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T05:38:15.880 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T05:38:16.174 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T05:38:16.175 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T05:38:16.176 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T05:38:16.176 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T05:38:16.176 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T05:38:16.176 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T05:38:16.187 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T05:38:16.188 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T05:38:16.189 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T05:38:16.190 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T05:38:16.191 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T05:38:16.192 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T05:38:16.193 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T05:38:16.194 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T05:38:16.195 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T05:38:16.196 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T05:38:16.197 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T05:38:18.687 INFO:teuthology.orchestra.run.smithi134.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout:Upgraded: 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T05:38:18.688 INFO:teuthology.orchestra.run.smithi134.stdout:Installed: 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.689 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.690 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.691 INFO:teuthology.orchestra.run.smithi134.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T05:38:18.692 INFO:teuthology.orchestra.run.smithi134.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T05:38:18.693 INFO:teuthology.orchestra.run.smithi134.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T05:38:18.694 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T05:38:18.695 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T05:38:18.696 INFO:teuthology.orchestra.run.smithi134.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.697 INFO:teuthology.orchestra.run.smithi134.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T05:38:18.698 INFO:teuthology.orchestra.run.smithi134.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T05:38:18.699 INFO:teuthology.orchestra.run.smithi134.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T05:38:18.700 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:18.700 INFO:teuthology.orchestra.run.smithi134.stdout:Complete! 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout:Upgraded: 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T05:38:18.774 INFO:teuthology.orchestra.run.smithi022.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.775 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.776 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T05:38:18.777 INFO:teuthology.orchestra.run.smithi022.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T05:38:18.778 INFO:teuthology.orchestra.run.smithi022.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T05:38:18.779 INFO:teuthology.orchestra.run.smithi022.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T05:38:18.780 INFO:teuthology.orchestra.run.smithi022.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T05:38:18.781 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T05:38:18.782 INFO:teuthology.orchestra.run.smithi022.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T05:38:18.783 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T05:38:18.784 INFO:teuthology.orchestra.run.smithi022.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T05:38:18.785 INFO:teuthology.orchestra.run.smithi022.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T05:38:18.786 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T05:38:18.786 INFO:teuthology.orchestra.run.smithi022.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T05:38:18.786 INFO:teuthology.orchestra.run.smithi022.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T05:38:18.786 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.786 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.786 INFO:teuthology.orchestra.run.smithi022.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T05:38:18.787 INFO:teuthology.orchestra.run.smithi022.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T05:38:18.787 INFO:teuthology.orchestra.run.smithi022.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T05:38:18.787 INFO:teuthology.orchestra.run.smithi022.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T05:38:18.787 INFO:teuthology.orchestra.run.smithi022.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T05:38:18.787 INFO:teuthology.orchestra.run.smithi022.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T05:38:18.787 INFO:teuthology.orchestra.run.smithi022.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T05:38:18.788 INFO:teuthology.orchestra.run.smithi022.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T05:38:18.788 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:18.788 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-09-21T05:38:19.123 DEBUG:teuthology.parallel:result is None 2024-09-21T05:38:19.180 DEBUG:teuthology.parallel:result is None 2024-09-21T05:38:19.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:38:19.327 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T05:38:19.353 INFO:teuthology.orchestra.run.smithi022.stdout:19.3.0-5120.g18432218.el9 2024-09-21T05:38:19.353 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T05:38:19.353 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T05:38:19.356 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T05:38:19.509 DEBUG:teuthology.orchestra.run.smithi134:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T05:38:19.533 INFO:teuthology.orchestra.run.smithi134.stdout:19.3.0-5120.g18432218.el9 2024-09-21T05:38:19.533 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T05:38:19.533 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T05:38:19.535 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T05:38:19.535 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:19.535 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:38:19.564 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:19.564 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T05:38:19.602 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T05:38:19.603 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:19.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:38:19.631 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:38:19.698 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:19.698 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T05:38:19.725 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T05:38:19.791 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T05:38:19.792 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:19.792 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:38:19.818 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:38:19.884 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:19.884 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T05:38:19.910 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T05:38:19.975 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T05:38:19.975 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:19.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:38:20.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:38:20.068 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:20.068 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T05:38:20.092 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T05:38:20.157 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T05:38:20.235 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T05:38:20.235 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /var/log/ceph 2024-09-21T05:38:20.237 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-09-21T05:38:20.261 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T05:38:20.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T05:38:20.304 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T05:38:20.328 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T05:38:20.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T05:38:20.370 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T05:38:20.400 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T05:38:20.400 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T05:38:20.401 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T05:38:20.401 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:20.401 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T05:38:20.451 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T05:38:20.562 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T05:38:20.588 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:20.588 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T05:38:20.604 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T05:38:20.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T05:38:20.752 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T05:38:20.752 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T05:38:20.752 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913499', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi022.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP9CeIu7ANUoFBktppMpQJ6Gc/Gn96RqKc73kOHErFLQVeiGB367EjKHU/uKk4pcLgHlWZjPqkrx828Ptm+3vD4=', 'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBATMHMnY0uoDaGO8C6hujcaU6+qoC9ev28eIKzG9ui6NCoIlB9EBr2cO5xO5nJ/pOyKm3fA7PZ/+TPV9qGYdOUE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T05:38:20.753 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T05:38:20.756 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T05:38:20.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T05:38:20.812 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T05:38:20.861 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:20.861 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:38:20.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:38:20.892 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-21 05:38:10.287743908 +0000 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-21 05:36:09.707342101 +0000 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-21 05:36:09.707342101 +0000 2024-09-21T05:38:20.949 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-21 05:36:09.707342101 +0000 2024-09-21T05:38:20.949 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:38:21.013 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-21T05:38:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-21T05:38:21.014 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000105415 s, 4.9 MB/s 2024-09-21T05:38:21.015 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:38:21.072 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-21 05:38:10.287743908 +0000 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-21 05:36:10.041337709 +0000 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-21 05:36:10.041337709 +0000 2024-09-21T05:38:21.128 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-21 05:36:10.041337709 +0000 2024-09-21T05:38:21.129 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:38:21.192 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-21T05:38:21.192 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-21T05:38:21.192 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 8.5499e-05 s, 6.0 MB/s 2024-09-21T05:38:21.193 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:38:21.250 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-09-21T05:38:21.305 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:38:21.305 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-21 05:38:10.288743895 +0000 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-21 05:36:10.368333408 +0000 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-21 05:36:10.368333408 +0000 2024-09-21T05:38:21.306 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-21 05:36:10.368333408 +0000 2024-09-21T05:38:21.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:38:21.369 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-21T05:38:21.369 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-21T05:38:21.369 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.9072e-05 s, 5.2 MB/s 2024-09-21T05:38:21.370 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:38:21.427 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-09-21T05:38:21.483 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:38:21.483 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-09-21 05:38:10.288743895 +0000 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-09-21 05:36:10.704328990 +0000 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-09-21 05:36:10.704328990 +0000 2024-09-21T05:38:21.484 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-09-21 05:36:10.704328990 +0000 2024-09-21T05:38:21.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:38:21.548 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-09-21T05:38:21.548 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-09-21T05:38:21.548 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 7.8667e-05 s, 6.5 MB/s 2024-09-21T05:38:21.549 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:38:21.605 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:21.605 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:21.606 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T05:38:21.622 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T05:38:21.623 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 05:38:09.795491991 +0000 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 05:36:04.832429106 +0000 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 05:36:04.832429106 +0000 2024-09-21T05:38:21.679 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 05:36:04.832429106 +0000 2024-09-21T05:38:21.679 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T05:38:21.744 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T05:38:21.744 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T05:38:21.744 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.0002948 s, 1.7 MB/s 2024-09-21T05:38:21.745 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T05:38:21.802 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-21T05:38:21.858 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T05:38:21.858 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-21T05:38:21.858 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:21.859 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:21.859 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 05:38:09.795491991 +0000 2024-09-21T05:38:21.859 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 05:36:05.156421603 +0000 2024-09-21T05:38:21.859 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 05:36:05.156421603 +0000 2024-09-21T05:38:21.859 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 05:36:05.156421603 +0000 2024-09-21T05:38:21.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T05:38:21.922 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T05:38:21.922 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T05:38:21.922 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000164722 s, 3.1 MB/s 2024-09-21T05:38:21.923 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T05:38:21.980 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 05:38:09.796491968 +0000 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 05:36:05.476414191 +0000 2024-09-21T05:38:22.036 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 05:36:05.476414191 +0000 2024-09-21T05:38:22.037 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 05:36:05.476414191 +0000 2024-09-21T05:38:22.037 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T05:38:22.100 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T05:38:22.100 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T05:38:22.100 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000189588 s, 2.7 MB/s 2024-09-21T05:38:22.101 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T05:38:22.157 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-21T05:38:22.212 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-21 05:38:09.796491968 +0000 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-21 05:36:05.812406408 +0000 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-21 05:36:05.812406408 +0000 2024-09-21T05:38:22.213 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: 2024-09-21 05:36:05.812406408 +0000 2024-09-21T05:38:22.213 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T05:38:22.275 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-21T05:38:22.275 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-21T05:38:22.275 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.00017756 s, 2.9 MB/s 2024-09-21T05:38:22.276 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T05:38:22.332 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:22.332 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T05:38:22.332 INFO:tasks.ceph:Generating config... 2024-09-21T05:38:22.333 INFO:tasks.ceph:[client] pid file = 2024-09-21T05:38:22.334 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T05:38:22.334 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T05:38:22.334 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T05:38:22.334 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T05:38:22.334 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T05:38:22.334 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T05:38:22.334 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T05:38:22.334 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T05:38:22.334 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T05:38:22.334 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T05:38:22.334 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T05:38:22.334 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T05:38:22.334 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T05:38:22.335 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T05:38:22.335 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T05:38:22.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T05:38:22.404 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T05:38:22.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T05:38:22.468 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T05:38:22.495 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.22'), ('mon.b', '172.21.15.134'), ('mon.c', '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]')] 2024-09-21T05:38:22.495 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T05:38:22.496 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:22.496 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T05:38:22.551 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.22 --add b 172.21.15.134 --addv c '[v2:172.21.15.134:3301,v1:172.21.15.134:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: generated fsid cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:epoch 0 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:fsid cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:last_changed 2024-09-21T05:38:22.651135+0000 2024-09-21T05:38:22.652 INFO:teuthology.orchestra.run.smithi022.stdout:created 2024-09-21T05:38:22.651135+0000 2024-09-21T05:38:22.653 INFO:teuthology.orchestra.run.smithi022.stdout:min_mon_release 17 (quincy) 2024-09-21T05:38:22.653 INFO:teuthology.orchestra.run.smithi022.stdout:election_strategy: 1 2024-09-21T05:38:22.653 INFO:teuthology.orchestra.run.smithi022.stdout:0: [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon.a 2024-09-21T05:38:22.653 INFO:teuthology.orchestra.run.smithi022.stdout:1: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.b 2024-09-21T05:38:22.653 INFO:teuthology.orchestra.run.smithi022.stdout:2: [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] mon.c 2024-09-21T05:38:22.653 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T05:38:22.654 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T05:38:22.670 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cdac58e0-7b6b-4531-9caa-dad309e0d36b... 2024-09-21T05:38:22.671 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:38:22.712 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:38:22.754 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-21T05:38:22.754 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-09-21T05:38:22.754 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:38:22.754 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-09-21T05:38:22.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:38:22.756 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on skipped message = False 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.757 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-09-21T05:38:22.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-09-21T05:38:22.759 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:38:22.760 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:38:22.761 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:38:22.762 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:38:22.763 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-21T05:38:22.764 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:38:22.765 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-21T05:38:22.766 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-09-21T05:38:22.767 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-21T05:38:22.768 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-21T05:38:22.769 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:38:22.770 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:38:22.771 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:38:22.772 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-21T05:38:22.773 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-21T05:38:22.774 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-21T05:38:22.774 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-21T05:38:22.774 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-21T05:38:22.774 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-21T05:38:22.774 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-21T05:38:22.775 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T05:38:22.775 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T05:38:22.881 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T05:38:22.882 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:22.882 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T05:38:22.897 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:22.897 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T05:38:22.952 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:38:22.953 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:22.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T05:38:22.953 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T05:38:23.029 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:23.029 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:23.084 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:38:23.085 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:23.085 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T05:38:23.085 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T05:38:23.119 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:23.119 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:23.174 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T05:38:23.174 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T05:38:23.174 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T05:38:23.236 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T05:38:23.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T05:38:23.308 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T05:38:23.311 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T05:38:23.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T05:38:23.373 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T05:38:23.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T05:38:23.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T05:38:23.504 INFO:teuthology.orchestra.run.smithi022.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T05:38:23.507 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T05:38:23.532 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T05:38:23.595 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T05:38:23.598 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T05:38:23.622 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T05:38:23.723 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T05:38:23.726 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T05:38:23.750 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T05:38:23.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T05:38:23.804 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T05:38:23.817 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T05:38:23.868 INFO:teuthology.orchestra.run.smithi022.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T05:38:23.882 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T05:38:23.932 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T05:38:23.945 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T05:38:23.995 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T05:38:24.008 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T05:38:24.009 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi022.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T05:38:24.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T05:38:24.035 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:24.035 INFO:tasks.ceph:role: osd.0 2024-09-21T05:38:24.035 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:38:24.035 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T05:38:24.100 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:24.100 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:24.100 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:24.101 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:24.133 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-09-21T05:38:24.135 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-09-21T05:38:24.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T05:38:24.206 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T05:38:24.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T05:38:24.346 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:24.346 INFO:tasks.ceph:role: osd.1 2024-09-21T05:38:24.346 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:38:24.346 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T05:38:24.411 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:24.411 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:24.411 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:24.412 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:24.412 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:24.412 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:24.412 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:24.443 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-09-21T05:38:24.446 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-09-21T05:38:24.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T05:38:24.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T05:38:24.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T05:38:24.656 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:24.656 INFO:tasks.ceph:role: osd.2 2024-09-21T05:38:24.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:38:24.656 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T05:38:24.721 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:24.722 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:24.722 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:24.722 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:24.723 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:24.723 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:24.723 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:24.753 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-09-21T05:38:24.755 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-09-21T05:38:24.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T05:38:24.825 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T05:38:24.895 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T05:38:24.961 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:24.961 INFO:tasks.ceph:role: osd.3 2024-09-21T05:38:24.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:38:24.962 DEBUG:teuthology.orchestra.run.smithi022:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T05:38:25.026 INFO:teuthology.orchestra.run.smithi022.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:25.026 INFO:teuthology.orchestra.run.smithi022.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:25.026 INFO:teuthology.orchestra.run.smithi022.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:25.027 INFO:teuthology.orchestra.run.smithi022.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:25.027 INFO:teuthology.orchestra.run.smithi022.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:25.027 INFO:teuthology.orchestra.run.smithi022.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:25.027 INFO:teuthology.orchestra.run.smithi022.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:25.058 INFO:teuthology.orchestra.run.smithi022.stdout:Discarding blocks...Done. 2024-09-21T05:38:25.060 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi022.front.sepia.ceph.com -o noatime 2024-09-21T05:38:25.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T05:38:25.130 DEBUG:teuthology.orchestra.run.smithi022:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T05:38:25.200 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:25.301+0000 7f599fa44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T05:38:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:25.301+0000 7f599fa44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T05:38:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:25.301+0000 7f599fa44640 -1 bdev(0x555b5271d900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:25.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:25.301+0000 7f599fa44640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T05:38:28.244 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T05:38:28.275 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:28.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:28.378+0000 7fe26f630640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T05:38:28.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:28.379+0000 7fe26f630640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T05:38:28.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:28.379+0000 7fe26f630640 -1 bdev(0x55d2d8f45900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:28.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:28.379+0000 7fe26f630640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T05:38:31.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T05:38:31.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:31.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:31.422+0000 7f2dd421f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T05:38:31.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:31.422+0000 7f2dd421f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T05:38:31.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:31.422+0000 7f2dd421f640 -1 bdev(0x55bed99eb900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:31.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:31.422+0000 7f2dd421f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T05:38:34.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T05:38:34.866 DEBUG:teuthology.orchestra.run.smithi022:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:34.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:34.969+0000 7f362e620640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T05:38:34.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:34.969+0000 7f362e620640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T05:38:34.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:34.969+0000 7f362e620640 -1 bdev(0x55843318f900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:34.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:34.969+0000 7f362e620640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T05:38:38.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T05:38:38.171 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T05:38:38.201 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:38.201 INFO:tasks.ceph:role: osd.4 2024-09-21T05:38:38.201 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:38:38.201 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T05:38:38.277 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:38.278 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:38.767 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-21T05:38:38.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-21T05:38:38.769 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T05:38:38.889 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T05:38:38.920 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T05:38:38.985 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:38.986 INFO:tasks.ceph:role: osd.5 2024-09-21T05:38:38.986 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:38:38.986 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T05:38:39.058 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:39.058 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:39.058 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:39.058 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:39.058 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:39.059 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:39.059 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:39.059 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:39.059 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:39.059 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:39.521 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-21T05:38:39.523 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-21T05:38:39.523 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T05:38:39.616 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T05:38:39.645 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T05:38:39.709 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:39.710 INFO:tasks.ceph:role: osd.6 2024-09-21T05:38:39.710 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:38:39.710 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T05:38:39.781 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:39.781 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:39.781 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:39.781 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:39.782 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:39.782 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:39.782 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:39.782 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:39.782 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:39.782 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:40.262 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-21T05:38:40.264 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-21T05:38:40.264 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T05:38:40.403 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T05:38:40.432 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T05:38:40.497 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T05:38:40.497 INFO:tasks.ceph:role: osd.7 2024-09-21T05:38:40.497 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:38:40.497 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T05:38:40.569 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T05:38:40.570 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T05:38:41.070 INFO:teuthology.orchestra.run.smithi134.stdout:Discarding blocks...Done. 2024-09-21T05:38:41.072 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-21T05:38:41.072 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T05:38:41.204 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T05:38:41.236 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:41.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:41.362+0000 7f1221bdd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T05:38:41.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:41.362+0000 7f1221bdd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T05:38:41.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:41.362+0000 7f1221bdd640 -1 bdev(0x5600cadbb900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:41.366 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:41.362+0000 7f1221bdd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T05:38:44.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T05:38:44.307 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:44.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:44.408+0000 7f8904f56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T05:38:44.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:44.408+0000 7f8904f56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T05:38:44.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:44.408+0000 7f8904f56640 -1 bdev(0x563ec5031900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:44.411 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:44.408+0000 7f8904f56640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T05:38:47.565 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T05:38:47.594 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:47.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:47.693+0000 7f6079414640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T05:38:47.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:47.694+0000 7f6079414640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T05:38:47.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:47.694+0000 7f6079414640 -1 bdev(0x56532db25900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:47.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:47.694+0000 7f6079414640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T05:38:50.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:38:50.758 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:38:51.109 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T05:38:51.138 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:51.236+0000 7f6740244640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T05:38:51.239 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:51.237+0000 7f6740244640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T05:38:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:51.237+0000 7f6740244640 -1 bdev(0x556f26adf900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T05:38:51.240 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:51.237+0000 7f6740244640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T05:38:54.155 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T05:38:54.184 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T05:38:54.184 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T05:38:54.216 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T05:38:54.285 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.286 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T05:38:54.354 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T05:38:54.422 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T05:38:54.491 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T05:38:54.558 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:54.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T05:38:54.626 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.626 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T05:38:54.656 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.656 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T05:38:54.724 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T05:38:54.792 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.792 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T05:38:54.859 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.859 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T05:38:54.925 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.925 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T05:38:54.991 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:54.991 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T05:38:55.056 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:55.056 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T05:38:55.077 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:55.078 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T05:38:55.137 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:55.137 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T05:38:55.152 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:55.153 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T05:38:55.208 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T05:38:55.208 DEBUG:teuthology.orchestra.run.smithi022:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T05:38:55.210 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T05:38:55.251 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.y] 2024-09-21T05:38:55.251 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.0] 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.1] 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.2] 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.3] 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA== 2024-09-21T05:38:55.252 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr.x] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.4] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.5] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.6] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[osd.7] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ== 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:38:55.253 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:38:55.254 INFO:teuthology.orchestra.run.smithi022.stdout:[client.2] 2024-09-21T05:38:55.254 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ== 2024-09-21T05:38:55.254 INFO:teuthology.orchestra.run.smithi022.stdout:[client.1] 2024-09-21T05:38:55.254 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ== 2024-09-21T05:38:55.254 INFO:teuthology.orchestra.run.smithi022.stdout:[client.3] 2024-09-21T05:38:55.254 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw== 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.y] 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw== 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA== 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g== 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-09-21T05:38:55.271 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw== 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-09-21T05:38:55.272 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[client.2] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[client.1] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ== 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout:[client.3] 2024-09-21T05:38:55.273 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw== 2024-09-21T05:38:55.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:38:55.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:38:55.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.377 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.513 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.515 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.579 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.642 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.702 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.705 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:38:55.768 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T05:38:55.829 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.831 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.898 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T05:38:55.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:55.965 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.028 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.089 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.091 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.154 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T05:38:56.216 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.218 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.285 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.352 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.418 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T05:38:56.483 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T05:38:56.483 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T05:38:56.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T05:38:56.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T05:38:56.641 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T05:38:56.666 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T05:38:56.773 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T05:38:56.798 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T05:38:56.863 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T05:38:56.970 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T05:38:56.994 DEBUG:teuthology.orchestra.run.smithi022:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:56.996 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T05:38:57.049 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T05:38:57.049 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T05:38:57.049 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T05:38:57.052 INFO:tasks.ceph.mon.a:Started 2024-09-21T05:38:57.052 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T05:38:57.052 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T05:38:57.091 INFO:tasks.ceph.mon.b:Started 2024-09-21T05:38:57.091 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T05:38:57.091 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T05:38:57.093 INFO:tasks.ceph.mon.c:Started 2024-09-21T05:38:57.093 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T05:38:57.094 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T05:38:57.094 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T05:38:57.096 INFO:tasks.ceph.mgr.y:Started 2024-09-21T05:38:57.096 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T05:38:57.096 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T05:38:57.098 INFO:tasks.ceph.mgr.x:Started 2024-09-21T05:38:57.098 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T05:38:57.098 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T05:38:57.100 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T05:38:57.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T05:38:57.142 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T05:38:57.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T05:38:57.186 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T05:38:57.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T05:38:57.230 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T05:38:57.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T05:38:57.273 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T05:38:57.273 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T05:38:57.316 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T05:38:57.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T05:38:57.359 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T05:38:57.359 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T05:38:57.402 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T05:38:57.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T05:38:57.446 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T05:38:57.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T05:38:57.490 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T05:38:57.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T05:38:57.533 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T05:38:57.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T05:38:57.564 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T05:38:57.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T05:38:57.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.567+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28803f6d0 con 0x7fd29015b680 2024-09-21T05:38:57.575 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T05:38:57.576 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T05:38:57.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.572+0000 7f95ad99d640 1 -- 172.21.15.22:0/1211830787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a81505e0 msgr2=0x7f95a80b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.572+0000 7f95ad99d640 1 --2- 172.21.15.22:0/1211830787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a81505e0 0x7f95a80b6420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9594033000 tx=0x7f9594004730 comp rx=0 tx=0).stop 2024-09-21T05:38:57.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.573+0000 7f95ad99d640 1 -- 172.21.15.22:0/1211830787 shutdown_connections 2024-09-21T05:38:57.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.573+0000 7f95ad99d640 1 --2- 172.21.15.22:0/1211830787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95a80b6960 0x7f95a80c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.569+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2900b4490 con 0x7fd29015b680 2024-09-21T05:38:57.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.569+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2900b4a50 con 0x7fd29015b680 2024-09-21T05:38:57.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.572+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd290071950 con 0x7fd29015b680 2024-09-21T05:38:57.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.573+0000 7f95ad99d640 1 --2- 172.21.15.22:0/1211830787 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95a81505e0 0x7f95a80b6420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.574+0000 7f95ad99d640 1 --2- 172.21.15.22:0/1211830787 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9598004800 0x7f95a80cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.574+0000 7f95ad99d640 1 -- 172.21.15.22:0/1211830787 >> 172.21.15.22:0/1211830787 conn(0x7f95a80b3c50 msgr2=0x7f95a80cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.574+0000 7f95ad99d640 1 -- 172.21.15.22:0/1211830787 shutdown_connections 2024-09-21T05:38:57.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.574+0000 7f95ad99d640 1 -- 172.21.15.22:0/1211830787 wait complete. 2024-09-21T05:38:57.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.582+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd288046020 con 0x7fd29015b680 2024-09-21T05:38:57.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.582+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd2880338f0 con 0x7fd29015b680 2024-09-21T05:38:57.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.582+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd28804d070 con 0x7fd29015b680 2024-09-21T05:38:57.585 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T05:38:57.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T05:38:57.590 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T05:38:57.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T05:38:57.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.592+0000 7f80577fe640 1 -- 172.21.15.22:0/3278097288 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064037040 con 0x7f8068038c30 2024-09-21T05:38:57.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.592+0000 7f80577fe640 1 -- 172.21.15.22:0/3278097288 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8064044020 con 0x7f8068038c30 2024-09-21T05:38:57.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.592+0000 7f80577fe640 1 -- 172.21.15.22:0/3278097288 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f806403d790 con 0x7f8068038c30 2024-09-21T05:38:57.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.593+0000 7f80577fe640 1 -- 172.21.15.22:0/3278097288 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f806404b070 con 0x7f8068038c30 2024-09-21T05:38:57.598 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T05:38:57.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T05:38:57.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.597+0000 7ff452339640 1 -- 172.21.15.22:0/3926399137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ff424005100 con 0x7ff44c153bd0 2024-09-21T05:38:57.605 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T05:38:57.605 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T05:38:57.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.602+0000 7ff448ff9640 1 -- 172.21.15.22:0/3926399137 <== mon.0 v2:172.21.15.22:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7ff434013030 con 0x7ff44c153bd0 2024-09-21T05:38:57.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.602+0000 7ff448ff9640 1 -- 172.21.15.22:0/3926399137 <== mon.0 v2:172.21.15.22:3300/0 11 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ff434017d30 con 0x7ff44c153bd0 2024-09-21T05:38:57.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.601+0000 7f7a6dffb640 1 -- 172.21.15.22:0/3957401662 <== mon.1 v2:172.21.15.134:3300/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f7a7000f3e0 con 0x7f7a7815e920 2024-09-21T05:38:57.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.602+0000 7f80577fe640 1 -- 172.21.15.22:0/3278097288 <== mon.2 v2:172.21.15.134:3301/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f80640047c0 con 0x7f8068038c30 2024-09-21T05:38:57.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.603+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd288004810 con 0x7fd29015b680 2024-09-21T05:38:57.611 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T05:38:57.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.610+0000 7ff452339640 1 -- 172.21.15.22:0/3926399137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff44c153bd0 msgr2=0x7ff44c09d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.610+0000 7ff452339640 1 --2- 172.21.15.22:0/3926399137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff44c153bd0 0x7ff44c09d730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff43400b590 tx=0x7ff43400ba50 comp rx=0 tx=0).stop 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.611+0000 7ff452339640 1 -- 172.21.15.22:0/3926399137 shutdown_connections 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.611+0000 7ff452339640 1 --2- 172.21.15.22:0/3926399137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff44c153bd0 0x7ff44c09d730 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.611+0000 7ff452339640 1 --2- 172.21.15.22:0/3926399137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff44c162f20 0x7ff44c09d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.611+0000 7ff452339640 1 --2- 172.21.15.22:0/3926399137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff44c0cfcd0 0x7ff44c152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.611+0000 7ff452339640 1 -- 172.21.15.22:0/3926399137 >> 172.21.15.22:0/3926399137 conn(0x7ff44c06d410 msgr2=0x7ff44c0737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.615+0000 7ff452339640 1 -- 172.21.15.22:0/3926399137 shutdown_connections 2024-09-21T05:38:57.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.615+0000 7ff452339640 1 -- 172.21.15.22:0/3926399137 wait complete. 2024-09-21T05:38:57.626 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T05:38:57.626 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T05:38:57.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.623+0000 7fb9331da640 1 -- 172.21.15.22:0/2626028309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb91c0037a0 msgr2=0x7fb91c003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.623+0000 7fb9331da640 1 --2- 172.21.15.22:0/2626028309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb91c0037a0 0x7fb91c003bd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb918003c90 tx=0x7fb91c005830 comp rx=0 tx=0).stop 2024-09-21T05:38:57.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.625+0000 7fb931777640 1 -- 172.21.15.22:0/2626028309 reap_dead start 2024-09-21T05:38:57.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.625+0000 7fb9331da640 1 -- 172.21.15.22:0/2626028309 shutdown_connections 2024-09-21T05:38:57.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.625+0000 7fb9331da640 1 -- 172.21.15.22:0/2626028309 >> 172.21.15.22:0/2626028309 conn(0x7fb92401d9d0 msgr2=0x7fb924039690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.626+0000 7fb9331da640 1 -- 172.21.15.22:0/2626028309 shutdown_connections 2024-09-21T05:38:57.637 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T05:38:57.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T05:38:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.626+0000 7fb9331da640 1 -- 172.21.15.22:0/2626028309 wait complete. 2024-09-21T05:38:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.632+0000 7fb9331da640 1 Processor -- start 2024-09-21T05:38:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.632+0000 7fb9331da640 1 -- start start 2024-09-21T05:38:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.635+0000 7fb9331da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb924187230 0x7fb924185fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.635+0000 7fb9331da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 0x7fb9240e2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 0x7fb9240e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb92bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 0x7fb9240e2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42028/0 (socket says 172.21.15.22:42028) 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb92bfff640 1 -- 172.21.15.22:0/2156100705 learned_addr learned my addr 172.21.15.22:0/2156100705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb930f76640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb924187230 0x7fb924185fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb9331da640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9240e2a90 0x7fb92404c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240e35c0 con 0x7fb9240e3800 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240e3440 con 0x7fb9240e2a90 2024-09-21T05:38:57.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.638+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb9240f08d0 con 0x7fb924187230 2024-09-21T05:38:57.644 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T05:38:57.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.640+0000 7fb92bfff640 1 -- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb924187230 msgr2=0x7fb924185fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.640+0000 7fb92bfff640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb924187230 0x7fb924185fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.640+0000 7fb92bfff640 1 -- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9240e2a90 msgr2=0x7fb92404c060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.640+0000 7fb92bfff640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9240e2a90 0x7fb92404c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.641+0000 7fb92bfff640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9240e2ea0 con 0x7fb9240e3800 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.641+0000 7fb92bfff640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 0x7fb9240e2550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb918030040 tx=0x7fb918007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.641+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9180035f0 con 0x7fb9240e3800 2024-09-21T05:38:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.642+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9240e4030 con 0x7fb9240e3800 2024-09-21T05:38:57.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.644+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9240e45f0 con 0x7fb9240e3800 2024-09-21T05:38:57.649 DEBUG:tasks.ceph:set 22 configs 2024-09-21T05:38:57.649 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi022') 2024-09-21T05:38:57.649 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi022') 2024-09-21T05:38:57.649 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi022') 2024-09-21T05:38:57.649 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi022') 2024-09-21T05:38:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.646+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fb91802bac0 con 0x7fb9240e3800 2024-09-21T05:38:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.646+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918004620 con 0x7fb9240e3800 2024-09-21T05:38:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.647+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9180047c0 con 0x7fb9240e3800 2024-09-21T05:38:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.647+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9180070d0 con 0x7fb9240e3800 2024-09-21T05:38:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.648+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c005080 con 0x7fb9240e3800 2024-09-21T05:38:57.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.653+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb9240e2ea0 con 0x7fb9240e3800 2024-09-21T05:38:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.809+0000 7f96f4b64640 1 -- 172.21.15.22:0/2570695903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 msgr2=0x7f96f00d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.809+0000 7f96f4b64640 1 --2- 172.21.15.22:0/2570695903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 0x7f96f00d0340 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009830 tx=0x7f96e002a560 comp rx=0 tx=0).stop 2024-09-21T05:38:57.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.815+0000 7f96f4b64640 1 -- 172.21.15.22:0/2570695903 shutdown_connections 2024-09-21T05:38:57.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.815+0000 7f96f4b64640 1 --2- 172.21.15.22:0/2570695903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 0x7f96f00d0340 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.815+0000 7f96f4b64640 1 --2- 172.21.15.22:0/2570695903 >> v2:172.21.15.134:3300/0 conn(0x7f96f0162d00 0x7f96f015ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.815+0000 7f96f4b64640 1 -- 172.21.15.22:0/2570695903 >> 172.21.15.22:0/2570695903 conn(0x7f96f0070b20 msgr2=0x7f96f0070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.819+0000 7f96f4b64640 1 -- 172.21.15.22:0/2570695903 shutdown_connections 2024-09-21T05:38:57.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.823+0000 7f96f4b64640 1 -- 172.21.15.22:0/2570695903 wait complete. 2024-09-21T05:38:57.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.826+0000 7f96f4b64640 1 Processor -- start 2024-09-21T05:38:57.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.829+0000 7f96f4b64640 1 -- start start 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96f4b64640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 0x7f96f0153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96f4b64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96f00b3640 0x7f96f0156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96f4b64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 0x7f96f0156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96f4b64640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00d0880 con 0x7f96f0162d00 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96f4b64640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96f00aeec0 con 0x7f96f00b3640 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96f4b64640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f96f00d0a00 con 0x7f96f00b3230 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.831+0000 7f96effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 0x7f96f0156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96ef7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 0x7f96f0153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 0x7f96f0156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42044/0 (socket says 172.21.15.22:42044) 2024-09-21T05:38:57.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.833+0000 7f7a7f3ed640 1 -- 172.21.15.22:0/3957401662 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f7a500050e0 con 0x7f7a7815e920 2024-09-21T05:38:57.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 -- 172.21.15.22:0/1160835688 learned_addr learned my addr 172.21.15.22:0/1160835688 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:57.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 -- 172.21.15.22:0/1160835688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 msgr2=0x7f96f0153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 0x7f96f0153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 -- 172.21.15.22:0/1160835688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96f00b3640 msgr2=0x7f96f0156610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96f00b3640 0x7f96f0156610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 -- 172.21.15.22:0/1160835688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f0155320 con 0x7f96f0162d00 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.832+0000 7f96effff640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 0x7f96f0156b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96e800cae0 tx=0x7f96e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.834+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8027020 con 0x7f96f0162d00 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.834+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f96e800aab0 con 0x7f96f0162d00 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.834+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8023430 con 0x7f96f0162d00 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.834+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f00cd0b0 con 0x7f96f0162d00 2024-09-21T05:38:57.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.834+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f01555b0 con 0x7f96f0162d00 2024-09-21T05:38:57.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.838+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f96e802a020 con 0x7f96f0162d00 2024-09-21T05:38:57.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.838+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96e802cac0 con 0x7f96f0162d00 2024-09-21T05:38:57.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.838+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d4005080 con 0x7f96f0162d00 2024-09-21T05:38:57.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.845+0000 7f7a6dffb640 1 -- 172.21.15.22:0/3957401662 <== mon.1 v2:172.21.15.134:3300/0 11 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v6)=0 v6) ==== 108+0+0 (secure 0 0 0) 0x7f7a7000f630 con 0x7f7a7815e920 2024-09-21T05:38:57.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.846+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f96e8031030 con 0x7f96f0162d00 2024-09-21T05:38:57.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f47a009b640 1 -- 172.21.15.22:0/274382224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479808b2c0 msgr2=0x7f479808b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f47a009b640 1 --2- 172.21.15.22:0/274382224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479808b2c0 0x7f479808b6d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4790005fd0 tx=0x7f479002b390 comp rx=0 tx=0).stop 2024-09-21T05:38:57.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 -- 172.21.15.22:0/3957401662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7815e920 msgr2=0x7f7a781565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 --2- 172.21.15.22:0/3957401662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7815e920 0x7f7a781565f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000c880 tx=0x7f7a7000cd40 comp rx=0 tx=0).stop 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 -- 172.21.15.22:0/3957401662 shutdown_connections 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 --2- 172.21.15.22:0/3957401662 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7a7815e920 0x7f7a781565f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 --2- 172.21.15.22:0/3957401662 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7a780cec40 0x7f7a7815a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 --2- 172.21.15.22:0/3957401662 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7a780af500 0x7f7a78145560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f7a7f3ed640 1 -- 172.21.15.22:0/3957401662 >> 172.21.15.22:0/3957401662 conn(0x7f7a780cd4b0 msgr2=0x7f7a780af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.849+0000 7f7a7f3ed640 1 -- 172.21.15.22:0/3957401662 shutdown_connections 2024-09-21T05:38:57.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.849+0000 7f7a7f3ed640 1 -- 172.21.15.22:0/3957401662 wait complete. 2024-09-21T05:38:57.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f47a009b640 1 -- 172.21.15.22:0/274382224 shutdown_connections 2024-09-21T05:38:57.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f47a009b640 1 --2- 172.21.15.22:0/274382224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4798072e00 0x7f4798073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f47a009b640 1 --2- 172.21.15.22:0/274382224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f479808b2c0 0x7f479808b6d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.848+0000 7f47a009b640 1 -- 172.21.15.22:0/274382224 >> 172.21.15.22:0/274382224 conn(0x7f47980b1c20 msgr2=0x7f47980b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.849+0000 7f47a009b640 1 -- 172.21.15.22:0/274382224 shutdown_connections 2024-09-21T05:38:57.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.849+0000 7f47a009b640 1 -- 172.21.15.22:0/274382224 wait complete. 2024-09-21T05:38:57.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.854+0000 7f47a009b640 1 Processor -- start 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.855+0000 7f47a009b640 1 -- start start 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.857+0000 7f47a009b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 0x7f4798160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.857+0000 7f47a009b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479808b2c0 0x7f4798201110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.857+0000 7f47a009b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479816e750 0x7f47981698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.857+0000 7f47a009b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f479806f9c0 con 0x7f479816e750 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.857+0000 7f47a009b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f479806f840 con 0x7f4798072e00 2024-09-21T05:38:57.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.857+0000 7f47a009b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4798073750 con 0x7f479808b2c0 2024-09-21T05:38:57.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.860+0000 7f479de37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 0x7f4798160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.860+0000 7f479de37640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 0x7f4798160020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:35214/0 (socket says 172.21.15.22:35214) 2024-09-21T05:38:57.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.860+0000 7f479de37640 1 -- 172.21.15.22:0/31148096 learned_addr learned my addr 172.21.15.22:0/31148096 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:57.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.860+0000 7f479e638640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479816e750 0x7f47981698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.861+0000 7f479de37640 1 -- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479808b2c0 msgr2=0x7f4798201110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:57.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.861+0000 7f479de37640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479808b2c0 0x7f4798201110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.861+0000 7f479de37640 1 -- 172.21.15.22:0/31148096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479816e750 msgr2=0x7f47981698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.861+0000 7f479de37640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479816e750 0x7f47981698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.861+0000 7f479de37640 1 -- 172.21.15.22:0/31148096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479820ab00 con 0x7f4798072e00 2024-09-21T05:38:57.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.861+0000 7f479e638640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479816e750 0x7f47981698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:57.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.862+0000 7f479de37640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 0x7f4798160020 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f479400c7a0 tx=0x7f479400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:57.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.865+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794017070 con 0x7f4798072e00 2024-09-21T05:38:57.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.865+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f47940035f0 con 0x7f4798072e00 2024-09-21T05:38:57.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.865+0000 7f47a009b640 1 -- 172.21.15.22:0/31148096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4798160c60 con 0x7f4798072e00 2024-09-21T05:38:57.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.866+0000 7f47a009b640 1 -- 172.21.15.22:0/31148096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f479820ada0 con 0x7f4798072e00 2024-09-21T05:38:57.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.868+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47940039e0 con 0x7f4798072e00 2024-09-21T05:38:57.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.881+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f479401a070 con 0x7f4798072e00 2024-09-21T05:38:57.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.881+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f479401c780 con 0x7f4798072e00 2024-09-21T05:38:57.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.881+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4778005080 con 0x7f4798072e00 2024-09-21T05:38:57.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.885+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4794021020 con 0x7f4798072e00 2024-09-21T05:38:57.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.896+0000 7f89effff640 1 -- 172.21.15.22:0/3556405669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e80e7000 msgr2=0x7f89e8005de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.896+0000 7f89effff640 1 --2- 172.21.15.22:0/3556405669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e80e7000 0x7f89e8005de0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f89e800eec0 tx=0x7f89e003b750 comp rx=0 tx=0).stop 2024-09-21T05:38:57.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.897+0000 7f89effff640 1 -- 172.21.15.22:0/3556405669 shutdown_connections 2024-09-21T05:38:57.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.897+0000 7f89effff640 1 --2- 172.21.15.22:0/3556405669 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e80e7000 0x7f89e8005de0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.897+0000 7f89effff640 1 --2- 172.21.15.22:0/3556405669 >> v2:172.21.15.134:3300/0 conn(0x7f89e8052360 0x7f89e80e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.897+0000 7f89effff640 1 -- 172.21.15.22:0/3556405669 >> 172.21.15.22:0/3556405669 conn(0x7f89e801d9a0 msgr2=0x7f89e8039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.898+0000 7f89effff640 1 -- 172.21.15.22:0/3556405669 shutdown_connections 2024-09-21T05:38:57.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.898+0000 7f89effff640 1 -- 172.21.15.22:0/3556405669 wait complete. 2024-09-21T05:38:57.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.901+0000 7f89effff640 1 Processor -- start 2024-09-21T05:38:57.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.911+0000 7f89effff640 1 -- start start 2024-09-21T05:38:57.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.913+0000 7f89effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 0x7f89e80dc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.914+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 0x7f89e80dc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.914+0000 7f89eeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 0x7f89e80dc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42054/0 (socket says 172.21.15.22:42054) 2024-09-21T05:38:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.914+0000 7f89eeffd640 1 -- 172.21.15.22:0/2422296287 learned_addr learned my addr 172.21.15.22:0/2422296287 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.914+0000 7f89effff640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e80e7410 0x7f89e80d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.915+0000 7f89effff640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89e80d7120 0x7f89e80d93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.915+0000 7f89ee7fc640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e80e7410 0x7f89e80d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.915+0000 7f89ef7fe640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89e80d7120 0x7f89e80d93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.915+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89e8033ea0 con 0x7f89e8052360 2024-09-21T05:38:57.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.915+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89e80e3620 con 0x7f89e80e7410 2024-09-21T05:38:57.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.915+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89e80512e0 con 0x7f89e80d7120 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.916+0000 7f89eeffd640 1 -- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89e80d7120 msgr2=0x7f89e80d93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.916+0000 7f89eeffd640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89e80d7120 0x7f89e80d93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.916+0000 7f89eeffd640 1 -- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e80e7410 msgr2=0x7f89e80d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.916+0000 7f89eeffd640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e80e7410 0x7f89e80d8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.916+0000 7f89eeffd640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e80e2210 con 0x7f89e8052360 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.916+0000 7f89eeffd640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 0x7f89e80dc890 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f89f006bc20 tx=0x7f89f006a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:57.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.917+0000 7f89ee7fc640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e80e7410 0x7f89e80d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:57.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.917+0000 7f89ef7fe640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89e80d7120 0x7f89e80d93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:57.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.931+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0099070 con 0x7f89e8052360 2024-09-21T05:38:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.931+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f89f0074820 con 0x7f89e8052360 2024-09-21T05:38:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.932+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e80d7900 con 0x7f89e8052360 2024-09-21T05:38:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.932+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e80e1f30 con 0x7f89e8052360 2024-09-21T05:38:57.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.934+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f0074bd0 con 0x7f89e8052360 2024-09-21T05:38:57.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.936+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89f009c070 con 0x7f89e8052360 2024-09-21T05:38:57.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.936+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89f009e780 con 0x7f89e8052360 2024-09-21T05:38:57.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.936+0000 7f89d57fa640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d0005080 con 0x7f89e8052360 2024-09-21T05:38:57.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.941+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f89f0091070 con 0x7f89e8052360 2024-09-21T05:38:57.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.951+0000 7fb9331da640 1 -- 172.21.15.22:0/2156100705 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fb90c0053a0 con 0x7fb9240e3800 2024-09-21T05:38:57.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.967+0000 7ffafe0be640 1 -- 172.21.15.22:0/2752521679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf816e750 msgr2=0x7ffaf80cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.968+0000 7fb9297fa640 1 -- 172.21.15.22:0/2156100705 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v7)=0 v7) ==== 106+0+0 (secure 0 0 0) 0x7fb91803f460 con 0x7fb9240e3800 2024-09-21T05:38:57.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 -- 172.21.15.22:0/2156100705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 msgr2=0x7fb9240e2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 0x7fb9240e2550 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb918030040 tx=0x7fb918007e20 comp rx=0 tx=0).stop 2024-09-21T05:38:57.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 -- 172.21.15.22:0/2156100705 shutdown_connections 2024-09-21T05:38:57.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9240e2a90 0x7fb92404c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9240e3800 0x7fb9240e2550 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 --2- 172.21.15.22:0/2156100705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb924187230 0x7fb924185fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 -- 172.21.15.22:0/2156100705 >> 172.21.15.22:0/2156100705 conn(0x7fb92401d9d0 msgr2=0x7fb924054d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 -- 172.21.15.22:0/2156100705 shutdown_connections 2024-09-21T05:38:57.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.970+0000 7fb916ffd640 1 -- 172.21.15.22:0/2156100705 wait complete. 2024-09-21T05:38:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.967+0000 7ffafe0be640 1 --2- 172.21.15.22:0/2752521679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf816e750 0x7ffaf80cc940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8074cb0 tx=0x7ffae802a410 comp rx=0 tx=0).stop 2024-09-21T05:38:57.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.978+0000 7ffafe0be640 1 -- 172.21.15.22:0/2752521679 shutdown_connections 2024-09-21T05:38:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.978+0000 7ffafe0be640 1 --2- 172.21.15.22:0/2752521679 >> v2:172.21.15.134:3300/0 conn(0x7ffaf808b9d0 0x7ffaf8072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.978+0000 7ffafe0be640 1 --2- 172.21.15.22:0/2752521679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf816e750 0x7ffaf80cc940 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.978+0000 7ffafe0be640 1 -- 172.21.15.22:0/2752521679 >> 172.21.15.22:0/2752521679 conn(0x7ffaf80b1c20 msgr2=0x7ffaf80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.978+0000 7ffafe0be640 1 -- 172.21.15.22:0/2752521679 shutdown_connections 2024-09-21T05:38:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.978+0000 7ffafe0be640 1 -- 172.21.15.22:0/2752521679 wait complete. 2024-09-21T05:38:57.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.981+0000 7ffafe0be640 1 Processor -- start 2024-09-21T05:38:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.982+0000 7ffafe0be640 1 -- start start 2024-09-21T05:38:57.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.984+0000 7ffafe0be640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 0x7ffaf8212110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffaf77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 0x7ffaf8212110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffaf77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 0x7ffaf8212110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42064/0 (socket says 172.21.15.22:42064) 2024-09-21T05:38:57.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7f8074a2f640 1 -- 172.21.15.22:0/3278097288 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f80680063b0 con 0x7f8068038c30 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffaf77fe640 1 -- 172.21.15.22:0/814313590 learned_addr learned my addr 172.21.15.22:0/814313590 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffafe0be640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf81f8f80 0x7ffaf80b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffafe0be640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaf81f9390 0x7ffaf80b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffaf6ffd640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf81f8f80 0x7ffaf80b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.985+0000 7ffaf7fff640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaf81f9390 0x7ffaf80b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.986+0000 7ffafe0be640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf80734c0 con 0x7ffaf80cce80 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.986+0000 7ffafe0be640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8073340 con 0x7ffaf81f8f80 2024-09-21T05:38:57.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.986+0000 7ffafe0be640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffaf8073640 con 0x7ffaf81f9390 2024-09-21T05:38:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7f80577fe640 1 -- 172.21.15.22:0/3278097288 <== mon.2 v2:172.21.15.134:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f8064004990 con 0x7f8068038c30 2024-09-21T05:38:57.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf6ffd640 1 -- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaf81f9390 msgr2=0x7ffaf80b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf6ffd640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaf81f9390 0x7ffaf80b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf6ffd640 1 -- 172.21.15.22:0/814313590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 msgr2=0x7ffaf8212110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf6ffd640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 0x7ffaf8212110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf6ffd640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf81f7f00 con 0x7ffaf81f8f80 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf7fff640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaf81f9390 0x7ffaf80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf77fe640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 0x7ffaf8212110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffaf6ffd640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf81f8f80 0x7ffaf80b47a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000c870 tx=0x7ffaf000cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.990+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0017070 con 0x7ffaf81f8f80 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.991+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7ffaf0005070 con 0x7ffaf81f8f80 2024-09-21T05:38:57.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.991+0000 7ffafe0be640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf81f64b0 con 0x7ffaf81f8f80 2024-09-21T05:38:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.991+0000 7ffafe0be640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf81f9e40 con 0x7ffaf81f8f80 2024-09-21T05:38:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7f80557fa640 1 -- 172.21.15.22:0/3278097288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8068038c30 msgr2=0x7f806801e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7f80557fa640 1 --2- 172.21.15.22:0/3278097288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8068038c30 0x7f806801e310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8064031000 tx=0x7f8064003d70 comp rx=0 tx=0).stop 2024-09-21T05:38:57.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7f80557fa640 1 -- 172.21.15.22:0/3278097288 shutdown_connections 2024-09-21T05:38:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7f80557fa640 1 --2- 172.21.15.22:0/3278097288 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80680e3a90 0x7f806801e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7f80557fa640 1 --2- 172.21.15.22:0/3278097288 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8068038c30 0x7f806801e310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0005460 con 0x7ffaf81f8f80 2024-09-21T05:38:57.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.992+0000 7f80557fa640 1 --2- 172.21.15.22:0/3278097288 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8068038820 0x7f806801fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.993+0000 7f80557fa640 1 -- 172.21.15.22:0/3278097288 >> 172.21.15.22:0/3278097288 conn(0x7f8068051420 msgr2=0x7f80680e0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.993+0000 7f80557fa640 1 -- 172.21.15.22:0/3278097288 shutdown_connections 2024-09-21T05:38:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.993+0000 7f80557fa640 1 -- 172.21.15.22:0/3278097288 wait complete. 2024-09-21T05:38:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.994+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffaf001a070 con 0x7ffaf81f8f80 2024-09-21T05:38:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.994+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffaf00127f0 con 0x7ffaf81f8f80 2024-09-21T05:38:57.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.994+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad8005080 con 0x7ffaf81f8f80 2024-09-21T05:38:57.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:57.997+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ffaf0005600 con 0x7ffaf81f8f80 2024-09-21T05:38:58.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.004+0000 7f13f2b62640 1 -- 172.21.15.22:0/3563533759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec08b2c0 msgr2=0x7f13ec08b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.004+0000 7f13f2b62640 1 --2- 172.21.15.22:0/3563533759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec08b2c0 0x7f13ec08b6d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13e4005fd0 tx=0x7f13e402ac10 comp rx=0 tx=0).stop 2024-09-21T05:38:58.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.004+0000 7f13f2b62640 1 -- 172.21.15.22:0/3563533759 shutdown_connections 2024-09-21T05:38:58.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.005+0000 7f13f2b62640 1 --2- 172.21.15.22:0/3563533759 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec08b2c0 0x7f13ec08b6d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.005+0000 7f13f2b62640 1 -- 172.21.15.22:0/3563533759 >> 172.21.15.22:0/3563533759 conn(0x7f13ec0b1c20 msgr2=0x7f13ec0b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.005+0000 7f13f2b62640 1 -- 172.21.15.22:0/3563533759 shutdown_connections 2024-09-21T05:38:58.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.005+0000 7f13f2b62640 1 -- 172.21.15.22:0/3563533759 wait complete. 2024-09-21T05:38:58.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.008+0000 7f13f2b62640 1 Processor -- start 2024-09-21T05:38:58.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.009+0000 7f13f2b62640 1 -- start start 2024-09-21T05:38:58.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.011+0000 7f13f2b62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 0x7f13ec200d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.011+0000 7f13f08fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 0x7f13ec200d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 0x7f13ec200d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42078/0 (socket says 172.21.15.22:42078) 2024-09-21T05:38:58.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 -- 172.21.15.22:0/165850832 learned_addr learned my addr 172.21.15.22:0/165850832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f2b62640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ec16e750 0x7f13ec161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f2b62640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13ec162490 0x7f13ec1652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f2b62640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec073920 con 0x7f13ec072e00 2024-09-21T05:38:58.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.014+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fd290160080 con 0x7fd29015b680 2024-09-21T05:38:58.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f2b62640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec0cd160 con 0x7f13ec162490 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f2b62640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec073aa0 con 0x7f13ec16e750 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13ebfff640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ec16e750 0x7f13ec161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f10ff640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13ec162490 0x7f13ec1652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 -- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ec16e750 msgr2=0x7f13ec161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ec16e750 0x7f13ec161f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 -- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13ec162490 msgr2=0x7f13ec1652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13ec162490 0x7f13ec1652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.012+0000 7f13f08fe640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec1685d0 con 0x7f13ec072e00 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.013+0000 7f13f08fe640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 0x7f13ec200d50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b780 tx=0x7f13dc00bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.013+0000 7f13ebfff640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ec16e750 0x7f13ec161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:58.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.013+0000 7f13f10ff640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13ec162490 0x7f13ec1652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:58.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.018+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd288055020 con 0x7fd29015b680 2024-09-21T05:38:58.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.018+0000 7fd285ffb640 1 -- 172.21.15.22:0/3884461916 <== mon.2 v2:172.21.15.134:3301/0 9 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v9)=0 v9) ==== 123+0+0 (secure 0 0 0) 0x7fd288003e60 con 0x7fd29015b680 2024-09-21T05:38:58.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96e802c400 con 0x7f96f0162d00 2024-09-21T05:38:58.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.018+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f479400a5e0 con 0x7f4798072e00 2024-09-21T05:38:58.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89f0074d70 con 0x7f89e8052360 2024-09-21T05:38:58.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.018+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf00057e0 con 0x7ffaf81f8f80 2024-09-21T05:38:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc019070 con 0x7f13ec072e00 2024-09-21T05:38:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.020+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f13dc00c810 con 0x7f13ec072e00 2024-09-21T05:38:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.020+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13dc00cc40 con 0x7f13ec072e00 2024-09-21T05:38:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.020+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13dc014440 con 0x7f13ec072e00 2024-09-21T05:38:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd29015b680 msgr2=0x7fd2901514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 --2- 172.21.15.22:0/3884461916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd29015b680 0x7fd2901514d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2900c7d00 tx=0x7fd288004270 comp rx=0 tx=0).stop 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 shutdown_connections 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 --2- 172.21.15.22:0/3884461916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd29015b680 0x7fd2901514d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 --2- 172.21.15.22:0/3884461916 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2900d0d70 0x7fd290143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 --2- 172.21.15.22:0/3884461916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2900d00d0 0x7fd290149520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 >> 172.21.15.22:0/3884461916 conn(0x7fd2900cdc60 msgr2=0x7fd2900c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 shutdown_connections 2024-09-21T05:38:58.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.019+0000 7fd297439640 1 -- 172.21.15.22:0/3884461916 wait complete. 2024-09-21T05:38:58.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.020+0000 7f13f2b62640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ec15e830 con 0x7f13ec072e00 2024-09-21T05:38:58.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.020+0000 7f13f2b62640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ec15edf0 con 0x7f13ec072e00 2024-09-21T05:38:58.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.022+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f13dc01c070 con 0x7f13ec072e00 2024-09-21T05:38:58.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.022+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 6 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f13dc01dc20 con 0x7f13ec072e00 2024-09-21T05:38:58.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.025+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13cc005290 con 0x7f13ec072e00 2024-09-21T05:38:58.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.029+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f13dc01de20 con 0x7f13ec072e00 2024-09-21T05:38:58.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 -- 172.21.15.22:0/877850534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b80cce80 msgr2=0x7f34b808b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/877850534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b80cce80 0x7f34b808b2c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34b0005fd0 tx=0x7f34b002cbb0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 -- 172.21.15.22:0/877850534 shutdown_connections 2024-09-21T05:38:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/877850534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b80cce80 0x7f34b808b2c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/877850534 >> v2:172.21.15.22:3300/0 conn(0x7f34b816e750 0x7f34b80cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 -- 172.21.15.22:0/877850534 >> 172.21.15.22:0/877850534 conn(0x7f34b80b1c20 msgr2=0x7f34b80b2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.042+0000 7f76e1320640 1 -- 172.21.15.22:0/4219421236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 msgr2=0x7f76d40512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.042+0000 7f76e1320640 1 --2- 172.21.15.22:0/4219421236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d40512e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76dc06af90 tx=0x7f76dc0a6680 comp rx=0 tx=0).stop 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.037+0000 7f34bdfa1640 1 -- 172.21.15.22:0/877850534 shutdown_connections 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.038+0000 7f34bdfa1640 1 -- 172.21.15.22:0/877850534 wait complete. 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.041+0000 7f34bdfa1640 1 Processor -- start 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.041+0000 7f34bdfa1640 1 -- start start 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.043+0000 7f34bdfa1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 0x7f34b815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.043+0000 7f34bdfa1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b816e750 0x7f34b8167eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.043+0000 7f34bdfa1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b808b350 0x7f34b8163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.043+0000 7f34bdfa1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f34b816f510 con 0x7f34b808b800 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.043+0000 7f34bdfa1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f34b806fa50 con 0x7f34b816e750 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.043+0000 7f34bdfa1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f34b816f690 con 0x7f34b808b350 2024-09-21T05:38:58.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 0x7f34b815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 0x7f34b815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42106/0 (socket says 172.21.15.22:42106) 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 -- 172.21.15.22:0/3874329722 learned_addr learned my addr 172.21.15.22:0/3874329722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 -- 172.21.15.22:0/3874329722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b808b350 msgr2=0x7f34b8163c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b808b350 0x7f34b8163c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 -- 172.21.15.22:0/3874329722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b816e750 msgr2=0x7f34b8167eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b816e750 0x7f34b8167eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.044+0000 7f34b77fe640 1 -- 172.21.15.22:0/3874329722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b8162b80 con 0x7f34b808b800 2024-09-21T05:38:58.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.045+0000 7f34b77fe640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 0x7f34b815f830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34a800fe70 tx=0x7f34a800d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.047+0000 7f76e1320640 1 -- 172.21.15.22:0/4219421236 shutdown_connections 2024-09-21T05:38:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.047+0000 7f76e1320640 1 --2- 172.21.15.22:0/4219421236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d40512e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.047+0000 7f76e1320640 1 --2- 172.21.15.22:0/4219421236 >> v2:172.21.15.134:3300/0 conn(0x7f76d40e71d0 0x7f76d4005de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.047+0000 7f76e1320640 1 -- 172.21.15.22:0/4219421236 >> 172.21.15.22:0/4219421236 conn(0x7f76d401d9a0 msgr2=0x7f76d4039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.047+0000 7f76e1320640 1 -- 172.21.15.22:0/4219421236 shutdown_connections 2024-09-21T05:38:58.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.050+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8009d20 con 0x7f34b808b800 2024-09-21T05:38:58.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.047+0000 7f76e1320640 1 -- 172.21.15.22:0/4219421236 wait complete. 2024-09-21T05:38:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.050+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b8162e60 con 0x7f34b808b800 2024-09-21T05:38:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.050+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b8163420 con 0x7f34b808b800 2024-09-21T05:38:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.051+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a8004020 con 0x7f34b808b800 2024-09-21T05:38:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.051+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8004470 con 0x7f34b808b800 2024-09-21T05:38:58.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.053+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f34a80035f0 con 0x7f34b808b800 2024-09-21T05:38:58.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.053+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f34a80179b0 con 0x7f34b808b800 2024-09-21T05:38:58.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.053+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b80b22e0 con 0x7f34b808b800 2024-09-21T05:38:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.058+0000 7f76e1320640 1 Processor -- start 2024-09-21T05:38:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.058+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f34a8017410 con 0x7f34b808b800 2024-09-21T05:38:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.058+0000 7f76e1320640 1 -- start start 2024-09-21T05:38:58.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.063+0000 7fdefa6ec640 1 -- 172.21.15.22:0/580312941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 msgr2=0x7fdef40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.063+0000 7fdefa6ec640 1 --2- 172.21.15.22:0/580312941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef40cf010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdee40097b0 tx=0x7fdee402a300 comp rx=0 tx=0).stop 2024-09-21T05:38:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.064+0000 7fdefa6ec640 1 -- 172.21.15.22:0/580312941 shutdown_connections 2024-09-21T05:38:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.064+0000 7fdefa6ec640 1 --2- 172.21.15.22:0/580312941 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef40cf010 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.064+0000 7fdefa6ec640 1 -- 172.21.15.22:0/580312941 >> 172.21.15.22:0/580312941 conn(0x7fdef40b1cb0 msgr2=0x7fdef40b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.064+0000 7fdefa6ec640 1 -- 172.21.15.22:0/580312941 shutdown_connections 2024-09-21T05:38:58.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76e1320640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d4180c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.064+0000 7fdefa6ec640 1 -- 172.21.15.22:0/580312941 wait complete. 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76e1320640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76d4052360 0x7f76d40db870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76daffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d4180c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76daffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d4180c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42112/0 (socket says 172.21.15.22:42112) 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76daffd640 1 -- 172.21.15.22:0/1034924012 learned_addr learned my addr 172.21.15.22:0/1034924012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76e1320640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d40e71d0 0x7f76d40dfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.065+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40e3590 con 0x7f76d4006320 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f76d40e3410 con 0x7f76d40e71d0 2024-09-21T05:38:58.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f76d4006730 con 0x7f76d4052360 2024-09-21T05:38:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76daffd640 1 -- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76d4052360 msgr2=0x7f76d40db870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76daffd640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76d4052360 0x7f76d40db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76daffd640 1 -- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d40e71d0 msgr2=0x7f76d40dfa70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76daffd640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d40e71d0 0x7f76d40dfa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.066+0000 7f76daffd640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d4181140 con 0x7f76d4006320 2024-09-21T05:38:58.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.067+0000 7f76daffd640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d4180c00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f76cc001290 tx=0x7f76cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.070 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.070 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.070 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.070 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.071 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.068+0000 7fdefa6ec640 1 Processor -- start 2024-09-21T05:38:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.068+0000 7fdefa6ec640 1 -- start start 2024-09-21T05:38:58.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.068+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc011790 con 0x7f76d4006320 2024-09-21T05:38:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.068+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76cc017070 con 0x7f76d4006320 2024-09-21T05:38:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.068+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc011ce0 con 0x7f76d4006320 2024-09-21T05:38:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.068+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d4186d90 con 0x7f76d4006320 2024-09-21T05:38:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.069+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d40df270 con 0x7f76d4006320 2024-09-21T05:38:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.069+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f76cc01c460 con 0x7f76d4006320 2024-09-21T05:38:58.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.069+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f76cc012950 con 0x7f76d4006320 2024-09-21T05:38:58.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.073+0000 7fdefa6ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef4153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.073+0000 7fdefa6ec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdef40af630 0x7fdef41ebde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.073+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef4153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.073+0000 7fdef3fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef4153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:35316/0 (socket says 172.21.15.22:35316) 2024-09-21T05:38:58.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.077+0000 7fdef3fff640 1 -- 172.21.15.22:0/3752222945 learned_addr learned my addr 172.21.15.22:0/3752222945 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.077+0000 7fdefa6ec640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef40aebe0 0x7fdef41ec320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.077+0000 7fdef8c89640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef40aebe0 0x7fdef41ec320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdefa6ec640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdef40cfb90 con 0x7fdef40aebe0 2024-09-21T05:38:58.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdefa6ec640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdef40cf950 con 0x7fdef4070f20 2024-09-21T05:38:58.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdefa6ec640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdef40cfd10 con 0x7fdef40af630 2024-09-21T05:38:58.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdef8c89640 1 -- 172.21.15.22:0/3752222945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdef40af630 msgr2=0x7fdef41ebde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:38:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdef8c89640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdef40af630 0x7fdef41ebde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdef8c89640 1 -- 172.21.15.22:0/3752222945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 msgr2=0x7fdef4153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdef8c89640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef4153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.080+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d4005de0 con 0x7f76d4006320 2024-09-21T05:38:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.078+0000 7fdef8c89640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef4153f80 con 0x7fdef40aebe0 2024-09-21T05:38:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.079+0000 7fdef8c89640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef40aebe0 0x7fdef41ec320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00caf0 tx=0x7fdeec002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.080+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec017070 con 0x7fdef40aebe0 2024-09-21T05:38:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.081+0000 7fdefa6ec640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef4072f90 con 0x7fdef40aebe0 2024-09-21T05:38:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.081+0000 7fdefa6ec640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef4073250 con 0x7fdef40aebe0 2024-09-21T05:38:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.083+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdeec003c60 con 0x7fdef40aebe0 2024-09-21T05:38:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.084+0000 7fc1e5be6640 1 -- 172.21.15.22:0/3009115464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 msgr2=0x7fc1d8033cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.084+0000 7fc1e5be6640 1 --2- 172.21.15.22:0/3009115464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d8033cd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc1e006af90 tx=0x7fc1e0095520 comp rx=0 tx=0).stop 2024-09-21T05:38:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.085+0000 7fc1e5be6640 1 -- 172.21.15.22:0/3009115464 shutdown_connections 2024-09-21T05:38:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.085+0000 7fc1e5be6640 1 --2- 172.21.15.22:0/3009115464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d8033cd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.085+0000 7fc1e5be6640 1 --2- 172.21.15.22:0/3009115464 >> v2:172.21.15.22:3300/0 conn(0x7fc1d8054120 0x7fc1d8006300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.085+0000 7fc1e5be6640 1 -- 172.21.15.22:0/3009115464 >> 172.21.15.22:0/3009115464 conn(0x7fc1d801d990 msgr2=0x7fc1d8039650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.085+0000 7fc1e5be6640 1 -- 172.21.15.22:0/3009115464 shutdown_connections 2024-09-21T05:38:58.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.085+0000 7fc1e5be6640 1 -- 172.21.15.22:0/3009115464 wait complete. 2024-09-21T05:38:58.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.088+0000 7fc1e5be6640 1 Processor -- start 2024-09-21T05:38:58.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.088+0000 7fc1e5be6640 1 -- start start 2024-09-21T05:38:58.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.091+0000 7fc1e5be6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d804fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.084+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef40cf010 con 0x7fdef40aebe0 2024-09-21T05:38:58.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.091+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00aa20 con 0x7fdef40aebe0 2024-09-21T05:38:58.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.091+0000 7fc1e5be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 0x7fc1d8101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.091+0000 7fc1df7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d804fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.091+0000 7fc1df7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d804fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:57962/0 (socket says 172.21.15.22:57962) 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1deffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 0x7fc1d8101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1deffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 0x7fc1d8101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42138/0 (socket says 172.21.15.22:42138) 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1df7fe640 1 -- 172.21.15.22:0/2499982299 learned_addr learned my addr 172.21.15.22:0/2499982299 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1e5be6640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80550d0 0x7fc1d8103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdeec01a070 con 0x7fdef40aebe0 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdeec0128d0 con 0x7fdef40aebe0 2024-09-21T05:38:58.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8039fc0 con 0x7fc1d8054120 2024-09-21T05:38:58.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdeec012b90 con 0x7fdef40aebe0 2024-09-21T05:38:58.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1dffff640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80550d0 0x7fc1d8103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8039e40 con 0x7fc1d80550d0 2024-09-21T05:38:58.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.092+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d8034210 con 0x7fc1d80070a0 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1deffd640 1 -- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 msgr2=0x7fc1d804fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1deffd640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d804fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1deffd640 1 -- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80550d0 msgr2=0x7fc1d8103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1deffd640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80550d0 0x7fc1d8103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1deffd640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80509e0 con 0x7fc1d8054120 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1deffd640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 0x7fc1d8101480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0143a0 tx=0x7fc1cc014860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.093+0000 7fc1dffff640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80550d0 0x7fc1d8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.094+0000 7fad621da640 1 -- 172.21.15.22:0/2758683033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ced10 msgr2=0x7fad5c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.094+0000 7fad621da640 1 --2- 172.21.15.22:0/2758683033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ced10 0x7fad5c070550 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0d1a90 tx=0x7fad5002a480 comp rx=0 tx=0).stop 2024-09-21T05:38:58.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.095+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f76d4181140 con 0x7f76d4006320 2024-09-21T05:38:58.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.095+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc004020 con 0x7fc1d8054120 2024-09-21T05:38:58.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.095+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1cc0041c0 con 0x7fc1d8054120 2024-09-21T05:38:58.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.095+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d8050cc0 con 0x7fc1d8054120 2024-09-21T05:38:58.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.095+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d8051280 con 0x7fc1d8054120 2024-09-21T05:38:58.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.096+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00f9d0 con 0x7fc1d8054120 2024-09-21T05:38:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.098+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc1cc023020 con 0x7fc1d8054120 2024-09-21T05:38:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.098+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc1cc004a20 con 0x7fc1d8054120 2024-09-21T05:38:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.098+0000 7fad621da640 1 -- 172.21.15.22:0/2758683033 shutdown_connections 2024-09-21T05:38:58.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.098+0000 7fad621da640 1 --2- 172.21.15.22:0/2758683033 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ced10 0x7fad5c070550 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.098+0000 7fad621da640 1 -- 172.21.15.22:0/2758683033 >> 172.21.15.22:0/2758683033 conn(0x7fad5c0cd4b0 msgr2=0x7fad5c0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.100+0000 7fc982297640 1 -- 172.21.15.22:0/846345453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974005ea0 msgr2=0x7fc9740062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.100+0000 7fc982297640 1 --2- 172.21.15.22:0/846345453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974005ea0 0x7fc9740062b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc978013f80 tx=0x7fc978032ac0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.100+0000 7fc982297640 1 -- 172.21.15.22:0/846345453 shutdown_connections 2024-09-21T05:38:58.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.100+0000 7fc982297640 1 --2- 172.21.15.22:0/846345453 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc974034c10 0x7fc9740534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.100+0000 7fc982297640 1 --2- 172.21.15.22:0/846345453 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974005ea0 0x7fc9740062b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.100+0000 7fc982297640 1 -- 172.21.15.22:0/846345453 >> 172.21.15.22:0/846345453 conn(0x7fc97401d9a0 msgr2=0x7fc974039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.101+0000 7fad621da640 1 -- 172.21.15.22:0/2758683033 shutdown_connections 2024-09-21T05:38:58.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.098+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c0005290 con 0x7fc1d8054120 2024-09-21T05:38:58.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.106+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc1cc02a020 con 0x7fc1d8054120 2024-09-21T05:38:58.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.104+0000 7fad621da640 1 -- 172.21.15.22:0/2758683033 wait complete. 2024-09-21T05:38:58.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.106+0000 7fad621da640 1 Processor -- start 2024-09-21T05:38:58.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.107+0000 7fad621da640 1 -- start start 2024-09-21T05:38:58.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.103+0000 7fc982297640 1 -- 172.21.15.22:0/846345453 shutdown_connections 2024-09-21T05:38:58.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.103+0000 7fc982297640 1 -- 172.21.15.22:0/846345453 wait complete. 2024-09-21T05:38:58.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.106+0000 7fc982297640 1 Processor -- start 2024-09-21T05:38:58.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.107+0000 7fc982297640 1 -- start start 2024-09-21T05:38:58.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.109+0000 7fc982297640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 0x7fc9740fa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.109+0000 7fad621da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c06e100 0x7fad5c149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad621da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad5c0afd70 0x7fad5c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad621da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c14fbe0 0x7fad5c14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad621da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c070c10 con 0x7fad5c06e100 2024-09-21T05:38:58.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad5b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c06e100 0x7fad5c149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad5b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c06e100 0x7fad5c149c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42150/0 (socket says 172.21.15.22:42150) 2024-09-21T05:38:58.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad5b7fe640 1 -- 172.21.15.22:0/3114505303 learned_addr learned my addr 172.21.15.22:0/3114505303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad5affd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad5c0afd70 0x7fad5c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad5bfff640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c14fbe0 0x7fad5c14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad621da640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c070a90 con 0x7fad5c14fbe0 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.110+0000 7fad621da640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c070d90 con 0x7fad5c0afd70 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7fad5affd640 1 -- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c14fbe0 msgr2=0x7fad5c14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7fad5affd640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c14fbe0 0x7fad5c14fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7fad5affd640 1 -- 172.21.15.22:0/3114505303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c06e100 msgr2=0x7fad5c149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7fad5affd640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c06e100 0x7fad5c149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7fad5affd640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c1536a0 con 0x7fad5c0afd70 2024-09-21T05:38:58.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7f8eac320640 1 -- 172.21.15.22:0/1039176331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea406dd30 msgr2=0x7f8ea40ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.111+0000 7f8eac320640 1 --2- 172.21.15.22:0/1039176331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea406dd30 0x7f8ea40ce710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005fd0 tx=0x7f8e9c02a980 comp rx=0 tx=0).stop 2024-09-21T05:38:58.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7f8eac320640 1 -- 172.21.15.22:0/1039176331 shutdown_connections 2024-09-21T05:38:58.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fad5affd640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad5c0afd70 0x7fad5c154170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00b3e0 tx=0x7fad4c00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7f8eac320640 1 --2- 172.21.15.22:0/1039176331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea406dd30 0x7f8ea40ce710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7f8eac320640 1 --2- 172.21.15.22:0/1039176331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea40afeb0 0x7f8ea406d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7f8eac320640 1 -- 172.21.15.22:0/1039176331 >> 172.21.15.22:0/1039176331 conn(0x7f8ea40cd480 msgr2=0x7f8ea40cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.113+0000 7f8eac320640 1 -- 172.21.15.22:0/1039176331 shutdown_connections 2024-09-21T05:38:58.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c015020 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad4c011920 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c011d50 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.113+0000 7fad621da640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c0c4c20 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.114+0000 7fad621da640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c153970 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.114+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fad4c018020 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.114+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fad4c019990 con 0x7fad5c0afd70 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc981295640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 0x7fc9740fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc981295640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 0x7fc9740fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42148/0 (socket says 172.21.15.22:42148) 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc981295640 1 -- 172.21.15.22:0/4180637478 learned_addr learned my addr 172.21.15.22:0/4180637478 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc982297640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9740340f0 0x7fc9740511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc982297640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974034c10 0x7fc974051730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc982297640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc974007040 con 0x7fc974005ea0 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc982297640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc974006ec0 con 0x7fc974034c10 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc982297640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc974053a00 con 0x7fc9740340f0 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc980a94640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9740340f0 0x7fc9740511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.112+0000 7fc981a96640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974034c10 0x7fc974051730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.113+0000 7f8eac320640 1 -- 172.21.15.22:0/1039176331 wait complete. 2024-09-21T05:38:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.116+0000 7f8eac320640 1 Processor -- start 2024-09-21T05:38:58.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.116+0000 7f8eac320640 1 -- start start 2024-09-21T05:38:58.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.119+0000 7f8eac320640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 0x7f8ea4140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.119+0000 7f8eaa0bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 0x7f8ea4140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.119+0000 7f8eaa0bc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 0x7f8ea4140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42160/0 (socket says 172.21.15.22:42160) 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.119+0000 7f8eaa0bc640 1 -- 172.21.15.22:0/3700783422 learned_addr learned my addr 172.21.15.22:0/3700783422 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.119+0000 7f8eac320640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea40afeb0 0x7f8ea4156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.119+0000 7f8eac320640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea40d0350 0x7f8ea4142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.120+0000 7f8ea98bb640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea40afeb0 0x7f8ea4156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.120+0000 7f8eaa8bd640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea40d0350 0x7f8ea4142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.120+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea40cec50 con 0x7f8ea406dd30 2024-09-21T05:38:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.120+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea406e350 con 0x7f8ea40afeb0 2024-09-21T05:38:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.120+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea40cedd0 con 0x7f8ea40d0350 2024-09-21T05:38:58.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c15d0a0 con 0x7fad5c0afd70 2024-09-21T05:38:58.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7f8eaa0bc640 1 -- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea40d0350 msgr2=0x7f8ea4142b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7f8eaa0bc640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea40d0350 0x7f8ea4142b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7f8eaa0bc640 1 -- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea40afeb0 msgr2=0x7f8ea4156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7f8eaa0bc640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea40afeb0 0x7f8ea4156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.124+0000 7f8eaa0bc640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea4151190 con 0x7f8ea406dd30 2024-09-21T05:38:58.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.125+0000 7f8eaa0bc640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 0x7f8ea4140c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000b7b0 tx=0x7f8ea000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.125+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0019070 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ea000c830 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea41586b0 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea4156e70 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.128+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000cc80 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.128+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ea001c070 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.128+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ea001da10 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.128+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea40ce710 con 0x7f8ea406dd30 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.126+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fad5c15d0a0 con 0x7fad5c0afd70 2024-09-21T05:38:58.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.126+0000 7f3e31e47640 1 -- 172.21.15.22:0/93436787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e24052360 msgr2=0x7f3e240e6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.126+0000 7f3e31e47640 1 --2- 172.21.15.22:0/93436787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e24052360 0x7f3e240e6ac0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e24034920 tx=0x7f3e2002a4f0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f3e31e47640 1 -- 172.21.15.22:0/93436787 shutdown_connections 2024-09-21T05:38:58.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f3e31e47640 1 --2- 172.21.15.22:0/93436787 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e24052360 0x7f3e240e6ac0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f3e31e47640 1 -- 172.21.15.22:0/93436787 >> 172.21.15.22:0/93436787 conn(0x7f3e2401d9a0 msgr2=0x7f3e24039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7fc981295640 1 -- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9740340f0 msgr2=0x7fc9740511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7fc981295640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9740340f0 0x7fc9740511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7fc981295640 1 -- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974034c10 msgr2=0x7fc974051730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7fc981295640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974034c10 0x7fc974051730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.121+0000 7fc981295640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9741ba0b0 con 0x7fc974005ea0 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.124+0000 7fc981295640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 0x7fc9740fa830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc978009400 tx=0x7fc978034410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.130+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97803a040 con 0x7fc974005ea0 2024-09-21T05:38:58.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.130+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc978004220 con 0x7fc974005ea0 2024-09-21T05:38:58.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.131+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9780348d0 con 0x7fc974005ea0 2024-09-21T05:38:58.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.132+0000 7fc982297640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9741ba5b0 con 0x7fc974005ea0 2024-09-21T05:38:58.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.132+0000 7fc982297640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9741bab70 con 0x7fc974005ea0 2024-09-21T05:38:58.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.134+0000 7f123dfa1640 1 -- 172.21.15.22:0/2296441523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1220003aa0 msgr2=0x7f1220003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.134+0000 7f123dfa1640 1 --2- 172.21.15.22:0/2296441523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1220003aa0 0x7f1220003f10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f123000c610 tx=0x7f1220005830 comp rx=0 tx=0).stop 2024-09-21T05:38:58.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.134+0000 7f123dfa1640 1 -- 172.21.15.22:0/2296441523 shutdown_connections 2024-09-21T05:38:58.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.134+0000 7f123dfa1640 1 --2- 172.21.15.22:0/2296441523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1220003aa0 0x7f1220003f10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.134+0000 7f123dfa1640 1 -- 172.21.15.22:0/2296441523 >> 172.21.15.22:0/2296441523 conn(0x7f123806d410 msgr2=0x7f12380b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.135+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc974049260 con 0x7fc974005ea0 2024-09-21T05:38:58.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.138+0000 7f123dfa1640 1 -- 172.21.15.22:0/2296441523 shutdown_connections 2024-09-21T05:38:58.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.138+0000 7f123dfa1640 1 -- 172.21.15.22:0/2296441523 wait complete. 2024-09-21T05:38:58.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.141+0000 7f123dfa1640 1 Processor -- start 2024-09-21T05:38:58.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.143+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc97800c050 con 0x7fc974005ea0 2024-09-21T05:38:58.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.143+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc978003cc0 con 0x7fc974005ea0 2024-09-21T05:38:58.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.143+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f96d4005370 con 0x7f96f0162d00 2024-09-21T05:38:58.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.141+0000 7f123dfa1640 1 -- start start 2024-09-21T05:38:58.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.143+0000 7f123dfa1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 0x7f12381ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.143+0000 7f123dfa1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f123809aa30 0x7f12381504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.144+0000 7f123dfa1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1238150a20 0x7f1238205b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.144+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8ea000ce20 con 0x7f8ea406dd30 2024-09-21T05:38:58.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.144+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc9780035f0 con 0x7fc974005ea0 2024-09-21T05:38:58.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.148+0000 7f96e7fff640 1 -- 172.21.15.22:0/1160835688 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f96e8023b40 con 0x7f96f0162d00 2024-09-21T05:38:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.127+0000 7f3e31e47640 1 -- 172.21.15.22:0/93436787 shutdown_connections 2024-09-21T05:38:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.129+0000 7f3e31e47640 1 -- 172.21.15.22:0/93436787 wait complete. 2024-09-21T05:38:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.132+0000 7f3e31e47640 1 Processor -- start 2024-09-21T05:38:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.143+0000 7f3e31e47640 1 -- start start 2024-09-21T05:38:58.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31e47640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e24052360 0x7f3e2400b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31e47640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e240e7000 0x7f3e24050360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31e47640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 0x7f3e24032920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31e47640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e240f2a30 con 0x7f3e240e7000 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31e47640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e240f28b0 con 0x7f3e240e7410 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31e47640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e240f2bb0 con 0x7f3e24052360 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31646640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 0x7f3e24032920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31646640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 0x7f3e24032920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:35396/0 (socket says 172.21.15.22:35396) 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e31646640 1 -- 172.21.15.22:0/2222201158 learned_addr learned my addr 172.21.15.22:0/2222201158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.154+0000 7f3e2bfff640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e240e7000 0x7f3e24050360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e31646640 1 -- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e24052360 msgr2=0x7f3e2400b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e31646640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e24052360 0x7f3e2400b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e31646640 1 -- 172.21.15.22:0/2222201158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e240e7000 msgr2=0x7f3e24050360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e31646640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e240e7000 0x7f3e24050360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e31646640 1 -- 172.21.15.22:0/2222201158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e240508a0 con 0x7f3e240e7410 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e31646640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 0x7f3e24032920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c06af90 tx=0x7f3e2c06a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.155+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c069250 con 0x7f3e240e7410 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f1236ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f123809aa30 0x7f12381504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f12377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 0x7f12381ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f12377fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 0x7f12381ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42184/0 (socket says 172.21.15.22:42184) 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f12377fe640 1 -- 172.21.15.22:0/1238688467 learned_addr learned my addr 172.21.15.22:0/1238688467 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12380cfd20 con 0x7f1220003aa0 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f1237fff640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1238150a20 0x7f1238205b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12380cfba0 con 0x7f1238150a20 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12380cfea0 con 0x7f123809aa30 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f1236ffd640 1 -- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1238150a20 msgr2=0x7f1238205b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f1236ffd640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1238150a20 0x7f1238205b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f1236ffd640 1 -- 172.21.15.22:0/1238688467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 msgr2=0x7f12381ffb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f1236ffd640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 0x7f12381ffb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f1236ffd640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1238150e30 con 0x7f123809aa30 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f1236ffd640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f123809aa30 0x7f12381504e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f123806b8e0 tx=0x7f122c00fd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c01d070 con 0x7f123809aa30 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.157+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f122c00dcd0 con 0x7f123809aa30 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.159+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c0186a0 con 0x7f123809aa30 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.159+0000 7f12377fe640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 0x7f12381ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.159+0000 7f1237fff640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1238150a20 0x7f1238205b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.159+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1238202620 con 0x7f123809aa30 2024-09-21T05:38:58.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.159+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1238202be0 con 0x7f123809aa30 2024-09-21T05:38:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2400a570 con 0x7f3e240e7410 2024-09-21T05:38:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2400ab30 con 0x7f3e240e7410 2024-09-21T05:38:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e2c0694e0 con 0x7f3e240e7410 2024-09-21T05:38:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c069950 con 0x7f3e240e7410 2024-09-21T05:38:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e2c082400 con 0x7f3e240e7410 2024-09-21T05:38:58.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.156+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3e2c08b860 con 0x7f3e240e7410 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 msgr2=0x7f96f0156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 0x7f96f0156b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96e800cae0 tx=0x7f96e800a060 comp rx=0 tx=0).stop 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 shutdown_connections 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96f0162d00 0x7f96f0156b50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96f00b3640 0x7f96f0156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 --2- 172.21.15.22:0/1160835688 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96f00b3230 0x7f96f0153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 >> 172.21.15.22:0/1160835688 conn(0x7f96f0070b20 msgr2=0x7f96f00b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.160+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f122c00c040 con 0x7f123809aa30 2024-09-21T05:38:58.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.161+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f122c010930 con 0x7f123809aa30 2024-09-21T05:38:58.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.161+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123815fca0 con 0x7f123809aa30 2024-09-21T05:38:58.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.164+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f122c026080 con 0x7f123809aa30 2024-09-21T05:38:58.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.165+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0c005290 con 0x7f3e240e7410 2024-09-21T05:38:58.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.167+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 shutdown_connections 2024-09-21T05:38:58.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.167+0000 7f96f4b64640 1 -- 172.21.15.22:0/1160835688 wait complete. 2024-09-21T05:38:58.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.167+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3e2c090020 con 0x7f3e240e7410 2024-09-21T05:38:58.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.178+0000 7f89d57fa640 1 -- 172.21.15.22:0/2422296287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f89d0005640 con 0x7f89e8052360 2024-09-21T05:38:58.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.183+0000 7f89d77fe640 1 -- 172.21.15.22:0/2422296287 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7f89f00943f0 con 0x7f89e8052360 2024-09-21T05:38:58.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.184+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f4778005370 con 0x7f4798072e00 2024-09-21T05:38:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.184+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 msgr2=0x7f89e80dc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.184+0000 7f89effff640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 0x7f89e80dc890 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f89f006bc20 tx=0x7f89f006a000 comp rx=0 tx=0).stop 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 shutdown_connections 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89e80d7120 0x7f89e80d93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89e80e7410 0x7f89e80d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 --2- 172.21.15.22:0/2422296287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89e8052360 0x7f89e80dc890 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 >> 172.21.15.22:0/2422296287 conn(0x7f89e801d9a0 msgr2=0x7f89e8005de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 shutdown_connections 2024-09-21T05:38:58.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.185+0000 7f89effff640 1 -- 172.21.15.22:0/2422296287 wait complete. 2024-09-21T05:38:58.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.188+0000 7f478e7fc640 1 -- 172.21.15.22:0/31148096 <== mon.1 v2:172.21.15.134:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f479808bc10 con 0x7f4798072e00 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 msgr2=0x7f4798160020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 0x7f4798160020 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f479400c7a0 tx=0x7f479400cc60 comp rx=0 tx=0).stop 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 shutdown_connections 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f479816e750 0x7f47981698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f479808b2c0 0x7f4798201110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 --2- 172.21.15.22:0/31148096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4798072e00 0x7f4798160020 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 >> 172.21.15.22:0/31148096 conn(0x7f47980b1c20 msgr2=0x7f47980cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.194+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 shutdown_connections 2024-09-21T05:38:58.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.195+0000 7f4783fff640 1 -- 172.21.15.22:0/31148096 wait complete. 2024-09-21T05:38:58.207 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.228 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.228 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.276+0000 7fc1e5be6640 1 -- 172.21.15.22:0/2499982299 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fc1c0005580 con 0x7fc1d8054120 2024-09-21T05:38:58.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.264+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f13cc005850 con 0x7f13ec072e00 2024-09-21T05:38:58.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.277+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7ffad8005670 con 0x7ffaf81f8f80 2024-09-21T05:38:58.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.286+0000 7fc1cbfff640 1 -- 172.21.15.22:0/2499982299 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v13)=0 v13) ==== 125+0+0 (secure 0 0 0) 0x7fc1cc018040 con 0x7fc1d8054120 2024-09-21T05:38:58.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.289+0000 7f13e97fa640 1 -- 172.21.15.22:0/165850832 <== mon.0 v2:172.21.15.22:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7f13dc0145f0 con 0x7f13ec072e00 2024-09-21T05:38:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7fc1c9ffb640 1 -- 172.21.15.22:0/2499982299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 msgr2=0x7fc1d8101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7fc1c9ffb640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 0x7fc1d8101480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0143a0 tx=0x7fc1cc014860 comp rx=0 tx=0).stop 2024-09-21T05:38:58.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 msgr2=0x7f13ec200d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7f13d2ffd640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 0x7f13ec200d50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f13dc00b780 tx=0x7f13dc00bc40 comp rx=0 tx=0).stop 2024-09-21T05:38:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7fc1c9ffb640 1 -- 172.21.15.22:0/2499982299 shutdown_connections 2024-09-21T05:38:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7fc1c9ffb640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1d80550d0 0x7fc1d8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.292+0000 7fc1c9ffb640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1d8054120 0x7fc1d8101480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7fc1c9ffb640 1 --2- 172.21.15.22:0/2499982299 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1d80070a0 0x7fc1d804fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7fc1c9ffb640 1 -- 172.21.15.22:0/2499982299 >> 172.21.15.22:0/2499982299 conn(0x7fc1d801d990 msgr2=0x7fc1d80fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 shutdown_connections 2024-09-21T05:38:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13ec162490 0x7f13ec1652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ec16e750 0x7f13ec161f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 --2- 172.21.15.22:0/165850832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ec072e00 0x7f13ec200d50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 >> 172.21.15.22:0/165850832 conn(0x7f13ec0b1c20 msgr2=0x7f13ec1632b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 shutdown_connections 2024-09-21T05:38:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7fc1c9ffb640 1 -- 172.21.15.22:0/2499982299 shutdown_connections 2024-09-21T05:38:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7f13d2ffd640 1 -- 172.21.15.22:0/165850832 wait complete. 2024-09-21T05:38:58.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.293+0000 7fc1c9ffb640 1 -- 172.21.15.22:0/2499982299 wait complete. 2024-09-21T05:38:58.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.296+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f34b80b0640 con 0x7f34b808b800 2024-09-21T05:38:58.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.301+0000 7ffadffff640 1 -- 172.21.15.22:0/814313590 <== mon.1 v2:172.21.15.134:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v15)=0 v15) ==== 119+0+0 (secure 0 0 0) 0x7ffaf0006b30 con 0x7ffaf81f8f80 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf81f8f80 msgr2=0x7ffaf80b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf81f8f80 0x7ffaf80b47a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000c870 tx=0x7ffaf000cd30 comp rx=0 tx=0).stop 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 shutdown_connections 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaf81f9390 0x7ffaf80b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffaf81f8f80 0x7ffaf80b47a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 --2- 172.21.15.22:0/814313590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaf80cce80 0x7ffaf8212110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.302+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 >> 172.21.15.22:0/814313590 conn(0x7ffaf80b1c20 msgr2=0x7ffaf80b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.303+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 shutdown_connections 2024-09-21T05:38:58.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.303+0000 7ffaddffb640 1 -- 172.21.15.22:0/814313590 wait complete. 2024-09-21T05:38:58.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.313+0000 7f349ffff640 1 -- 172.21.15.22:0/3874329722 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) ==== 120+0+0 (secure 0 0 0) 0x7f34a8017c00 con 0x7f34b808b800 2024-09-21T05:38:58.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.318+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 msgr2=0x7f34b815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.318+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 0x7f34b815f830 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34a800fe70 tx=0x7f34a800d7b0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.319+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 shutdown_connections 2024-09-21T05:38:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.319+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34b808b350 0x7f34b8163c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.319+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34b816e750 0x7f34b8167eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.319+0000 7f34bdfa1640 1 --2- 172.21.15.22:0/3874329722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34b808b800 0x7f34b815f830 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.319+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 >> 172.21.15.22:0/3874329722 conn(0x7f34b80b1c20 msgr2=0x7f34b8073fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.319+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fad5c070550 con 0x7fad5c0afd70 2024-09-21T05:38:58.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.321+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 shutdown_connections 2024-09-21T05:38:58.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.321+0000 7f34bdfa1640 1 -- 172.21.15.22:0/3874329722 wait complete. 2024-09-21T05:38:58.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.329+0000 7fad43fff640 1 -- 172.21.15.22:0/3114505303 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v17)=0 v17) ==== 108+0+0 (secure 0 0 0) 0x7fad4c0193b0 con 0x7fad5c0afd70 2024-09-21T05:38:58.331 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.331 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.340 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad5c0afd70 msgr2=0x7fad5c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad5c0afd70 0x7fad5c154170 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00b3e0 tx=0x7fad4c00b8a0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 shutdown_connections 2024-09-21T05:38:58.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c14fbe0 0x7fad5c14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad5c0afd70 0x7fad5c154170 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 --2- 172.21.15.22:0/3114505303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c06e100 0x7fad5c149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 >> 172.21.15.22:0/3114505303 conn(0x7fad5c0cd4b0 msgr2=0x7fad5c0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 shutdown_connections 2024-09-21T05:38:58.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.339+0000 7fad41ffb640 1 -- 172.21.15.22:0/3114505303 wait complete. 2024-09-21T05:38:58.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.343+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f76d40e18f0 con 0x7f76d4006320 2024-09-21T05:38:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.346+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fdef41549f0 con 0x7fdef40aebe0 2024-09-21T05:38:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.346+0000 7f76c37fe640 1 -- 172.21.15.22:0/1034924012 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v18)=0 v18) ==== 111+0+0 (secure 0 0 0) 0x7f76cc012b50 con 0x7f76d4006320 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 msgr2=0x7f76d4180c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d4180c00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f76cc001290 tx=0x7f76cc00c890 comp rx=0 tx=0).stop 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 shutdown_connections 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76d40e71d0 0x7f76d40dfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76d4052360 0x7f76d40db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 --2- 172.21.15.22:0/1034924012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76d4006320 0x7f76d4180c00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 >> 172.21.15.22:0/1034924012 conn(0x7f76d401d9a0 msgr2=0x7f76d40515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 shutdown_connections 2024-09-21T05:38:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f76e1320640 1 -- 172.21.15.22:0/1034924012 wait complete. 2024-09-21T05:38:58.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.349+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f8ea415a110 con 0x7f8ea406dd30 2024-09-21T05:38:58.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.352+0000 7fdef0ff9640 1 -- 172.21.15.22:0/3752222945 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v19)=0 v19) ==== 119+0+0 (secure 0 0 0) 0x7fdeec012d70 con 0x7fdef40aebe0 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.354+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef40aebe0 msgr2=0x7fdef41ec320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.354+0000 7fdeda7fc640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef40aebe0 0x7fdef41ec320 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00caf0 tx=0x7fdeec002180 comp rx=0 tx=0).stop 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 shutdown_connections 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdef40aebe0 0x7fdef41ec320 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdef40af630 0x7fdef41ebde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 --2- 172.21.15.22:0/3752222945 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdef4070f20 0x7fdef4153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 >> 172.21.15.22:0/3752222945 conn(0x7fdef40b1cb0 msgr2=0x7fdef40b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 shutdown_connections 2024-09-21T05:38:58.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.355+0000 7fdeda7fc640 1 -- 172.21.15.22:0/3752222945 wait complete. 2024-09-21T05:38:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.358+0000 7f8e9affd640 1 -- 172.21.15.22:0/3700783422 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v20)=0 v20) ==== 106+0+0 (secure 0 0 0) 0x7f8ea001dc10 con 0x7f8ea406dd30 2024-09-21T05:38:58.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.358+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fc9740062b0 con 0x7fc974005ea0 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.359+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 msgr2=0x7f8ea4140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.359+0000 7f8eac320640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 0x7f8ea4140c50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000b7b0 tx=0x7f8ea000bc70 comp rx=0 tx=0).stop 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.359+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 shutdown_connections 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.359+0000 7f8eac320640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ea40d0350 0x7f8ea4142b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.359+0000 7f8eac320640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ea40afeb0 0x7f8ea4156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.360+0000 7f8eac320640 1 --2- 172.21.15.22:0/3700783422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ea406dd30 0x7f8ea4140c50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.360+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 >> 172.21.15.22:0/3700783422 conn(0x7f8ea40cd480 msgr2=0x7f8ea41517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.360+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 shutdown_connections 2024-09-21T05:38:58.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.360+0000 7f8eac320640 1 -- 172.21.15.22:0/3700783422 wait complete. 2024-09-21T05:38:58.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.367+0000 7fc971ffb640 1 -- 172.21.15.22:0/4180637478 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fc9780037d0 con 0x7fc974005ea0 2024-09-21T05:38:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.368+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 msgr2=0x7fc9740fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.368+0000 7fc95f7fe640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 0x7fc9740fa830 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc978009400 tx=0x7fc978034410 comp rx=0 tx=0).stop 2024-09-21T05:38:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 shutdown_connections 2024-09-21T05:38:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc974034c10 0x7fc974051730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9740340f0 0x7fc9740511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 --2- 172.21.15.22:0/4180637478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc974005ea0 0x7fc9740fa830 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 >> 172.21.15.22:0/4180637478 conn(0x7fc97401d9a0 msgr2=0x7fc974009c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 shutdown_connections 2024-09-21T05:38:58.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.369+0000 7fc95f7fe640 1 -- 172.21.15.22:0/4180637478 wait complete. 2024-09-21T05:38:58.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.384+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f3e0c005580 con 0x7f3e240e7410 2024-09-21T05:38:58.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.388+0000 7f3e297fa640 1 -- 172.21.15.22:0/2222201158 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f3e240508a0 con 0x7f3e240e7410 2024-09-21T05:38:58.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 msgr2=0x7f3e24032920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 0x7f3e24032920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c06af90 tx=0x7f3e2c06a000 comp rx=0 tx=0).stop 2024-09-21T05:38:58.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 shutdown_connections 2024-09-21T05:38:58.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e240e7410 0x7f3e24032920 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e240e7000 0x7f3e24050360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 --2- 172.21.15.22:0/2222201158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e24052360 0x7f3e2400b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 >> 172.21.15.22:0/2222201158 conn(0x7f3e2401d9a0 msgr2=0x7f3e24051810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 shutdown_connections 2024-09-21T05:38:58.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.389+0000 7f3e31e47640 1 -- 172.21.15.22:0/2222201158 wait complete. 2024-09-21T05:38:58.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.395+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f123809b1a0 con 0x7f123809aa30 2024-09-21T05:38:58.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.399+0000 7f121ffff640 1 -- 172.21.15.22:0/1238688467 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7f1238150e30 con 0x7f123809aa30 2024-09-21T05:38:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f123809aa30 msgr2=0x7f12381504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f123809aa30 0x7f12381504e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f123806b8e0 tx=0x7f122c00fd80 comp rx=0 tx=0).stop 2024-09-21T05:38:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 shutdown_connections 2024-09-21T05:38:58.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1238150a20 0x7f1238205b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f123809aa30 0x7f12381504e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 --2- 172.21.15.22:0/1238688467 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1220003aa0 0x7f12381ffb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 >> 172.21.15.22:0/1238688467 conn(0x7f123806d410 msgr2=0x7f12380d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 shutdown_connections 2024-09-21T05:38:58.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.400+0000 7f123dfa1640 1 -- 172.21.15.22:0/1238688467 wait complete. 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi022') 2024-09-21T05:38:58.413 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config dump 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 -- 172.21.15.22:0/3021362899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 msgr2=0x7f98800ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 --2- 172.21.15.22:0/3021362899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98800ccfa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9870009830 tx=0x7f987002a560 comp rx=0 tx=0).stop 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 -- 172.21.15.22:0/3021362899 shutdown_connections 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 --2- 172.21.15.22:0/3021362899 >> v2:172.21.15.134:3300/0 conn(0x7f9880073e70 0x7f9880074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 --2- 172.21.15.22:0/3021362899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98800ccfa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 -- 172.21.15.22:0/3021362899 >> 172.21.15.22:0/3021362899 conn(0x7f98800cdc60 msgr2=0x7f98800ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 -- 172.21.15.22:0/3021362899 shutdown_connections 2024-09-21T05:38:58.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.565+0000 7f9887e64640 1 -- 172.21.15.22:0/3021362899 wait complete. 2024-09-21T05:38:58.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.567+0000 7f9887e64640 1 Processor -- start 2024-09-21T05:38:58.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.567+0000 7f9887e64640 1 -- start start 2024-09-21T05:38:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9887e64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880073e70 0x7f98801f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9887e64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98801516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9887e64640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f988015ff30 0x7f9880151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9887e64640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98800d0690 con 0x7f988015c9b0 2024-09-21T05:38:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9887e64640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98800d0510 con 0x7f9880073e70 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9887e64640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98800d0810 con 0x7f988015ff30 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98801516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98801516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42210/0 (socket says 172.21.15.22:42210) 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 -- 172.21.15.22:0/3173502758 learned_addr learned my addr 172.21.15.22:0/3173502758 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9886401640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f988015ff30 0x7f9880151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f9885c00640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880073e70 0x7f98801f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 -- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f988015ff30 msgr2=0x7f9880151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f988015ff30 0x7f9880151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 -- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880073e70 msgr2=0x7f98801f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880073e70 0x7f98801f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 -- 172.21.15.22:0/3173502758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98801f9d70 con 0x7f988015c9b0 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98853ff640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98801516d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9870009950 tx=0x7f9870033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.569+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870031d60 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.570+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9870033e10 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.570+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9870033310 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.570+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9880150fd0 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.570+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98801f9a90 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.570+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98700334b0 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.570+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f987003b750 con 0x7f988015c9b0 2024-09-21T05:38:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.571+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988006a700 con 0x7f988015c9b0 2024-09-21T05:38:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.573+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f988006a700 con 0x7f988015c9b0 2024-09-21T05:38:58.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.635+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f98800d0d00 con 0x7f988015c9b0 2024-09-21T05:38:58.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.635+0000 7f98767fc640 1 -- 172.21.15.22:0/3173502758 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f9870052080 con 0x7f988015c9b0 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:38:58.638 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:38:58.639 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 msgr2=0x7f98801516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98801516d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9870009950 tx=0x7f9870033c20 comp rx=0 tx=0).stop 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 shutdown_connections 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f988015ff30 0x7f9880151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f988015c9b0 0x7f98801516d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 --2- 172.21.15.22:0/3173502758 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9880073e70 0x7f98801f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 >> 172.21.15.22:0/3173502758 conn(0x7f98800cdc60 msgr2=0x7f9880068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.636+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 shutdown_connections 2024-09-21T05:38:58.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.637+0000 7f9887e64640 1 -- 172.21.15.22:0/3173502758 wait complete. 2024-09-21T05:38:58.649 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T05:38:58.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T05:38:58.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.762+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1047956584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 msgr2=0x7f3e681673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.762+0000 7f3e6cabe640 1 --2- 172.21.15.22:0/1047956584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e681673c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3e60005fd0 tx=0x7f3e6002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.763+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1047956584 shutdown_connections 2024-09-21T05:38:58.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.763+0000 7f3e6cabe640 1 --2- 172.21.15.22:0/1047956584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e681673c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.763+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1047956584 >> 172.21.15.22:0/1047956584 conn(0x7f3e6806def0 msgr2=0x7f3e6806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.763+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1047956584 shutdown_connections 2024-09-21T05:38:58.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.763+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1047956584 wait complete. 2024-09-21T05:38:58.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.764+0000 7f3e6cabe640 1 Processor -- start 2024-09-21T05:38:58.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.765+0000 7f3e6cabe640 1 -- start start 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6cabe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e6806cde0 0x7f3e680b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6cabe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e68073e90 0x7f3e681f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6cabe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e68158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6cabe640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68167c90 con 0x7f3e680d1450 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6cabe640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68167b10 con 0x7f3e68073e90 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6cabe640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3e68167e10 con 0x7f3e6806cde0 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e66d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e68158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e66d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e68158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42224/0 (socket says 172.21.15.22:42224) 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e66d9d640 1 -- 172.21.15.22:0/1120341778 learned_addr learned my addr 172.21.15.22:0/1120341778 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6659c640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e6806cde0 0x7f3e680b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e65d9b640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e68073e90 0x7f3e681f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:58.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e65d9b640 1 -- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e6806cde0 msgr2=0x7f3e680b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e65d9b640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e6806cde0 0x7f3e680b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e65d9b640 1 -- 172.21.15.22:0/1120341778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 msgr2=0x7f3e68158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e65d9b640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e68158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e65d9b640 1 -- 172.21.15.22:0/1120341778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e68159ec0 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e6659c640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e6806cde0 0x7f3e680b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.766+0000 7f3e66d9d640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e68158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e65d9b640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e68073e90 0x7f3e681f9370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00c390 tx=0x7f3e5c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c017070 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e5c0035f0 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e680b5250 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003a40 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6813d770 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3e5c01a070 con 0x7f3e68073e90 2024-09-21T05:38:58.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.767+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3e5c012400 con 0x7f3e68073e90 2024-09-21T05:38:58.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.768+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e680ce6d0 con 0x7f3e68073e90 2024-09-21T05:38:58.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.770+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3e5c021070 con 0x7f3e68073e90 2024-09-21T05:38:58.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.836+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f3e680717d0 con 0x7f3e68073e90 2024-09-21T05:38:58.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.840+0000 7f3e56ffd640 1 -- 172.21.15.22:0/1120341778 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f3e680cede0 con 0x7f3e68073e90 2024-09-21T05:38:58.842 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-09-21T05:38:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e68073e90 msgr2=0x7f3e681f9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e68073e90 0x7f3e681f9370 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c00c390 tx=0x7f3e5c00c850 comp rx=0 tx=0).stop 2024-09-21T05:38:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 shutdown_connections 2024-09-21T05:38:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3e680d1450 0x7f3e68158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3e68073e90 0x7f3e681f9370 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 --2- 172.21.15.22:0/1120341778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3e6806cde0 0x7f3e680b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 >> 172.21.15.22:0/1120341778 conn(0x7f3e6806def0 msgr2=0x7f3e6806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 shutdown_connections 2024-09-21T05:38:58.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:38:58.841+0000 7f3e6cabe640 1 -- 172.21.15.22:0/1120341778 wait complete. 2024-09-21T05:38:58.854 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T05:38:58.854 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T05:38:58.854 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:58.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T05:38:58.879 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:58.880 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T05:38:58.945 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:58.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T05:38:59.011 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:38:59.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T05:38:59.077 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:59.078 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T05:38:59.103 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:59.103 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T05:38:59.168 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:59.168 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T05:38:59.232 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:38:59.232 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T05:38:59.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 92c772d4-9ad3-4bec-9dfc-c5fa87f93276 0 2024-09-21T05:38:59.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 -- 172.21.15.134:0/2251243197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 msgr2=0x7fc3ec0b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/2251243197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 0x7fc3ec0b1e10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009830 tx=0x7fc3e802a480 comp rx=0 tx=0).stop 2024-09-21T05:38:59.451 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 -- 172.21.15.134:0/2251243197 shutdown_connections 2024-09-21T05:38:59.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/2251243197 >> v2:172.21.15.22:3300/0 conn(0x7fc3ec0b2350 0x7fc3ec15f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/2251243197 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 0x7fc3ec0b1e10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 -- 172.21.15.134:0/2251243197 >> 172.21.15.134:0/2251243197 conn(0x7fc3ec06ea80 msgr2=0x7fc3ec0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:59.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 -- 172.21.15.134:0/2251243197 shutdown_connections 2024-09-21T05:38:59.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.448+0000 7fc3f3b13640 1 -- 172.21.15.134:0/2251243197 wait complete. 2024-09-21T05:38:59.452 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.450+0000 7fc3f3b13640 1 Processor -- start 2024-09-21T05:38:59.453 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.450+0000 7fc3f3b13640 1 -- start start 2024-09-21T05:38:59.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.451+0000 7fc3f3b13640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 0x7fc3ec1f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.455 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.451+0000 7fc3f3b13640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 0x7fc3ec143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.451+0000 7fc3f3b13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ec15b8c0 0x7fc3ec143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.451+0000 7fc3f3b13640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec15fac0 con 0x7fc3ec15b8c0 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.451+0000 7fc3f3b13640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec15f940 con 0x7fc3ec0b2350 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.451+0000 7fc3f3b13640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec15fc40 con 0x7fc3ec09aa30 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 0x7fc3ec143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 0x7fc3ec143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47952/0 (socket says 172.21.15.134:47952) 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 -- 172.21.15.134:0/3500585498 learned_addr learned my addr 172.21.15.134:0/3500585498 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f18af640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 0x7fc3ec1f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f20b0640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ec15b8c0 0x7fc3ec143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 -- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 msgr2=0x7fc3ec1f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 0x7fc3ec1f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 -- 172.21.15.134:0/3500585498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ec15b8c0 msgr2=0x7fc3ec143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ec15b8c0 0x7fc3ec143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 -- 172.21.15.134:0/3500585498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec15c710 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f10ae640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 0x7fc3ec143670 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc3e802aa70 tx=0x7fc3e8033c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005c90 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3e8031e50 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3ec15c9f0 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005260 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.452+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3ec15cfb0 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.453+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3e80047a0 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.453+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fc3e80332b0 con 0x7fc3ec0b2350 2024-09-21T05:38:59.457 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.453+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3ec0b1e10 con 0x7fc3ec0b2350 2024-09-21T05:38:59.459 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.457+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc3e803f030 con 0x7fc3ec0b2350 2024-09-21T05:38:59.529 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.528+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "92c772d4-9ad3-4bec-9dfc-c5fa87f93276", "id": 0} v 0) -- 0x7fc3ec0b1b10 con 0x7fc3ec0b2350 2024-09-21T05:38:59.533 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.531+0000 7fc3e27fc640 1 -- 172.21.15.134:0/3500585498 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92c772d4-9ad3-4bec-9dfc-c5fa87f93276", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fc3e8004c20 con 0x7fc3ec0b2350 2024-09-21T05:38:59.533 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-09-21T05:38:59.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 msgr2=0x7fc3ec143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 0x7fc3ec143670 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc3e802aa70 tx=0x7fc3e8033c20 comp rx=0 tx=0).stop 2024-09-21T05:38:59.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 shutdown_connections 2024-09-21T05:38:59.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3ec15b8c0 0x7fc3ec143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.534 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3ec0b2350 0x7fc3ec143670 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 --2- 172.21.15.134:0/3500585498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3ec09aa30 0x7fc3ec1f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 >> 172.21.15.134:0/3500585498 conn(0x7fc3ec06ea80 msgr2=0x7fc3ec0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:59.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 shutdown_connections 2024-09-21T05:38:59.535 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.532+0000 7fc3f3b13640 1 -- 172.21.15.134:0/3500585498 wait complete. 2024-09-21T05:38:59.544 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 6c399baa-b203-49d6-bb41-19e6c7235192 1 2024-09-21T05:38:59.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 -- 172.21.15.134:0/36835773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160550 msgr2=0x7fc1000cece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 --2- 172.21.15.134:0/36835773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160550 0x7fc1000cece0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009830 tx=0x7fc0f402a5a0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.657 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 -- 172.21.15.134:0/36835773 shutdown_connections 2024-09-21T05:38:59.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 --2- 172.21.15.134:0/36835773 >> v2:172.21.15.22:3300/0 conn(0x7fc100160960 0x7fc1000cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 --2- 172.21.15.134:0/36835773 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160550 0x7fc1000cece0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 -- 172.21.15.134:0/36835773 >> 172.21.15.134:0/36835773 conn(0x7fc10006d5d0 msgr2=0x7fc100071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:59.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 -- 172.21.15.134:0/36835773 shutdown_connections 2024-09-21T05:38:59.658 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.655+0000 7fc105e7e640 1 -- 172.21.15.134:0/36835773 wait complete. 2024-09-21T05:38:59.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.657+0000 7fc105e7e640 1 Processor -- start 2024-09-21T05:38:59.659 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.657+0000 7fc105e7e640 1 -- start start 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc105e7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1000d16f0 0x7fc1001f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc105e7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 0x7fc10015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc105e7e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 0x7fc1001587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc105e7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000d0390 con 0x7fc1000d16f0 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc105e7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1000d0210 con 0x7fc100160550 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc105e7e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1000d0510 con 0x7fc100160960 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 0x7fc10015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 0x7fc1001587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.662 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0fffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 0x7fc1001587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54312/0 (socket says 172.21.15.134:54312) 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 0x7fc10015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47962/0 (socket says 172.21.15.134:47962) 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0fffff640 1 -- 172.21.15.134:0/2846532973 learned_addr learned my addr 172.21.15.134:0/2846532973 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0ff7fe640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1000d16f0 0x7fc1001f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 -- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 msgr2=0x7fc1001587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 0x7fc1001587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 -- 172.21.15.134:0/2846532973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1000d16f0 msgr2=0x7fc1001f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1000d16f0 0x7fc1001f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 -- 172.21.15.134:0/2846532973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10015fb60 con 0x7fc100160550 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0fffff640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 0x7fc1001587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0feffd640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 0x7fc10015f3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4031000 tx=0x7fc0f4033ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.659+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4031950 con 0x7fc100160550 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.660+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0f4031b10 con 0x7fc100160550 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.660+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40330d0 con 0x7fc100160550 2024-09-21T05:38:59.663 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.660+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1001f39c0 con 0x7fc100160550 2024-09-21T05:38:59.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.660+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc100153de0 con 0x7fc100160550 2024-09-21T05:38:59.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.660+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc0f4033600 con 0x7fc100160550 2024-09-21T05:38:59.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.660+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fc0f40039a0 con 0x7fc100160550 2024-09-21T05:38:59.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.661+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1001f37b0 con 0x7fc100160550 2024-09-21T05:38:59.664 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.663+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc0f404b060 con 0x7fc100160550 2024-09-21T05:38:59.735 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.733+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6c399baa-b203-49d6-bb41-19e6c7235192", "id": 1} v 0) -- 0x7fc1000d0960 con 0x7fc100160550 2024-09-21T05:38:59.741 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.740+0000 7fc0e7fff640 1 -- 172.21.15.134:0/2846532973 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c399baa-b203-49d6-bb41-19e6c7235192", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fc0f4003bf0 con 0x7fc100160550 2024-09-21T05:38:59.741 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-09-21T05:38:59.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 msgr2=0x7fc10015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.742 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 0x7fc10015f3c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4031000 tx=0x7fc0f4033ba0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 shutdown_connections 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc100160960 0x7fc1001587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc100160550 0x7fc10015f3c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 --2- 172.21.15.134:0/2846532973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1000d16f0 0x7fc1001f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 >> 172.21.15.134:0/2846532973 conn(0x7fc10006d5d0 msgr2=0x7fc10015e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 shutdown_connections 2024-09-21T05:38:59.743 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.741+0000 7fc105e7e640 1 -- 172.21.15.134:0/2846532973 wait complete. 2024-09-21T05:38:59.753 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new acd6191b-3a15-4ec8-a7ce-bf0a5da639d9 2 2024-09-21T05:38:59.866 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.864+0000 7f3fd4e70640 1 -- 172.21.15.134:0/2566157391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 msgr2=0x7f3fd0074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.864+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/2566157391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd0074280 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8005fd0 tx=0x7f3fc802aae0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.865+0000 7f3fd4e70640 1 -- 172.21.15.134:0/2566157391 shutdown_connections 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.865+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/2566157391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd0074280 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.865+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/2566157391 >> v2:172.21.15.22:3300/0 conn(0x7f3fd015c9b0 0x7f3fd00ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.865+0000 7f3fd4e70640 1 -- 172.21.15.134:0/2566157391 >> 172.21.15.134:0/2566157391 conn(0x7f3fd00cdc60 msgr2=0x7f3fd00ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.865+0000 7f3fd4e70640 1 -- 172.21.15.134:0/2566157391 shutdown_connections 2024-09-21T05:38:59.867 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.865+0000 7f3fd4e70640 1 -- 172.21.15.134:0/2566157391 wait complete. 2024-09-21T05:38:59.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.867+0000 7f3fd4e70640 1 Processor -- start 2024-09-21T05:38:59.868 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.867+0000 7f3fd4e70640 1 -- start start 2024-09-21T05:38:59.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fd4e70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd01f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fd4e70640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 0x7f3fd01f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fd4e70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fd015ff30 0x7f3fd0159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:38:59.871 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fd4e70640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00d06a0 con 0x7f3fd015ff30 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fd4e70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd00d0520 con 0x7f3fd0073e70 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fd4e70640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3fd00d0820 con 0x7f3fd015c9b0 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd01f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fcdd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 0x7f3fd01f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fce59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd01f3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47978/0 (socket says 172.21.15.134:47978) 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fce59c640 1 -- 172.21.15.134:0/1494839589 learned_addr learned my addr 172.21.15.134:0/1494839589 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fcdd9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 0x7f3fd01f65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54328/0 (socket says 172.21.15.134:54328) 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.868+0000 7f3fced9d640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fd015ff30 0x7f3fd0159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fce59c640 1 -- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 msgr2=0x7f3fd01f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fce59c640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 0x7f3fd01f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fce59c640 1 -- 172.21.15.134:0/1494839589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fd015ff30 msgr2=0x7f3fd0159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fce59c640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fd015ff30 0x7f3fd0159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fce59c640 1 -- 172.21.15.134:0/1494839589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd01f7050 con 0x7f3fd0073e70 2024-09-21T05:38:59.872 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fcdd9b640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 0x7f3fd01f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fced9d640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fd015ff30 0x7f3fd0159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fce59c640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd01f3ad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000c370 tx=0x7f3fc000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0017070 con 0x7f3fd0073e70 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fc00053c0 con 0x7f3fd0073e70 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0003850 con 0x7f3fd0073e70 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd01f7330 con 0x7f3fd0073e70 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd01f2fc0 con 0x7f3fd0073e70 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3fc001a070 con 0x7f3fd0073e70 2024-09-21T05:38:59.873 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.869+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3fc0011c20 con 0x7f3fd0073e70 2024-09-21T05:38:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.870+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fd01f3710 con 0x7f3fd0073e70 2024-09-21T05:38:59.874 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.872+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3fc0021070 con 0x7f3fd0073e70 2024-09-21T05:38:59.945 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.943+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "acd6191b-3a15-4ec8-a7ce-bf0a5da639d9", "id": 2} v 0) -- 0x7f3fd00d0cd0 con 0x7f3fd0073e70 2024-09-21T05:38:59.948 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.947+0000 7f3fc6ffd640 1 -- 172.21.15.134:0/1494839589 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "acd6191b-3a15-4ec8-a7ce-bf0a5da639d9", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3fd00747c0 con 0x7f3fd0073e70 2024-09-21T05:38:59.949 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 msgr2=0x7f3fd01f3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd01f3ad0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000c370 tx=0x7f3fc000c830 comp rx=0 tx=0).stop 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 shutdown_connections 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3fd015ff30 0x7f3fd0159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3fd015c9b0 0x7f3fd01f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 --2- 172.21.15.134:0/1494839589 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3fd0073e70 0x7f3fd01f3ad0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 >> 172.21.15.134:0/1494839589 conn(0x7f3fd00cdc60 msgr2=0x7f3fd0154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 shutdown_connections 2024-09-21T05:38:59.950 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:38:59.948+0000 7f3fd4e70640 1 -- 172.21.15.134:0/1494839589 wait complete. 2024-09-21T05:38:59.961 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new f97b07de-93b7-4c01-8ed7-c996a3c64984 3 2024-09-21T05:39:00.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 -- 172.21.15.134:0/3001040946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef015c9b0 msgr2=0x7f8ef00ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.074 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 --2- 172.21.15.134:0/3001040946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef015c9b0 0x7f8ef00ccfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009900 tx=0x7f8ee402a6e0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 -- 172.21.15.134:0/3001040946 shutdown_connections 2024-09-21T05:39:00.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 --2- 172.21.15.134:0/3001040946 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef015c9b0 0x7f8ef00ccfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 -- 172.21.15.134:0/3001040946 >> 172.21.15.134:0/3001040946 conn(0x7f8ef00cdc60 msgr2=0x7f8ef00ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 -- 172.21.15.134:0/3001040946 shutdown_connections 2024-09-21T05:39:00.075 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.072+0000 7f8ef56a2640 1 -- 172.21.15.134:0/3001040946 wait complete. 2024-09-21T05:39:00.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.074+0000 7f8ef56a2640 1 Processor -- start 2024-09-21T05:39:00.076 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.074+0000 7f8ef56a2640 1 -- start start 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ef56a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 0x7f8ef0151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ef56a2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 0x7f8ef015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ef56a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef015ff30 0x7f8ef015b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ef56a2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0166b10 con 0x7f8ef015ff30 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ef56a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0166990 con 0x7f8ef0073e70 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ef56a2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef0166c90 con 0x7f8ef015c9b0 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 0x7f8ef015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 0x7f8ef0151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 0x7f8ef015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:54336/0 (socket says 172.21.15.134:54336) 2024-09-21T05:39:00.079 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eeeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 0x7f8ef0151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:47992/0 (socket says 172.21.15.134:47992) 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 -- 172.21.15.134:0/1312453464 learned_addr learned my addr 172.21.15.134:0/1312453464 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eef7fe640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef015ff30 0x7f8ef015b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 -- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 msgr2=0x7f8ef0151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 0x7f8ef0151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 -- 172.21.15.134:0/1312453464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef015ff30 msgr2=0x7f8ef015b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef015ff30 0x7f8ef015b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 -- 172.21.15.134:0/1312453464 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef01f6940 con 0x7f8ef015c9b0 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eeeffd640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 0x7f8ef0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8eee7fc640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 0x7f8ef015b000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4000c00 tx=0x7f8ee40039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.076+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee403e030 con 0x7f8ef015c9b0 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.077+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ee40047a0 con 0x7f8ef015c9b0 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.077+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4035040 con 0x7f8ef015c9b0 2024-09-21T05:39:00.080 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.077+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef006cf80 con 0x7f8ef015c9b0 2024-09-21T05:39:00.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.077+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef006d540 con 0x7f8ef015c9b0 2024-09-21T05:39:00.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.077+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ee4041030 con 0x7f8ef015c9b0 2024-09-21T05:39:00.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.077+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f8ee4043da0 con 0x7f8ef015c9b0 2024-09-21T05:39:00.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.078+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef00ccfa0 con 0x7f8ef015c9b0 2024-09-21T05:39:00.081 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.080+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f8ee4043410 con 0x7f8ef015c9b0 2024-09-21T05:39:00.152 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.150+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f97b07de-93b7-4c01-8ed7-c996a3c64984", "id": 3} v 0) -- 0x7f8ef0072500 con 0x7f8ef015c9b0 2024-09-21T05:39:00.155 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.154+0000 7f8ed77fe640 1 -- 172.21.15.134:0/1312453464 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f97b07de-93b7-4c01-8ed7-c996a3c64984", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f8ee4043910 con 0x7f8ef015c9b0 2024-09-21T05:39:00.155 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-09-21T05:39:00.156 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 msgr2=0x7f8ef015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 0x7f8ef015b000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4000c00 tx=0x7f8ee40039e0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 shutdown_connections 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8ef015ff30 0x7f8ef015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ef015c9b0 0x7f8ef015b000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 --2- 172.21.15.134:0/1312453464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ef0073e70 0x7f8ef0151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 >> 172.21.15.134:0/1312453464 conn(0x7f8ef00cdc60 msgr2=0x7f8ef015ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 shutdown_connections 2024-09-21T05:39:00.157 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.155+0000 7f8ef56a2640 1 -- 172.21.15.134:0/1312453464 wait complete. 2024-09-21T05:39:00.167 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2 4 2024-09-21T05:39:00.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 -- 172.21.15.134:0/1370167329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3040cf520 msgr2=0x7fd3040ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 --2- 172.21.15.134:0/1370167329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3040cf520 0x7fd3040ccad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005fd0 tx=0x7fd2fc02aa90 comp rx=0 tx=0).stop 2024-09-21T05:39:00.280 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 -- 172.21.15.134:0/1370167329 shutdown_connections 2024-09-21T05:39:00.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 --2- 172.21.15.134:0/1370167329 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd3040cf520 0x7fd3040ccad0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 --2- 172.21.15.134:0/1370167329 >> v2:172.21.15.22:3300/0 conn(0x7fd30415b900 0x7fd3040ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 -- 172.21.15.134:0/1370167329 >> 172.21.15.134:0/1370167329 conn(0x7fd304074170 msgr2=0x7fd3040b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 -- 172.21.15.134:0/1370167329 shutdown_connections 2024-09-21T05:39:00.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.278+0000 7fd30b415640 1 -- 172.21.15.134:0/1370167329 wait complete. 2024-09-21T05:39:00.281 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.280+0000 7fd30b415640 1 Processor -- start 2024-09-21T05:39:00.282 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.280+0000 7fd30b415640 1 -- start start 2024-09-21T05:39:00.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.281+0000 7fd30b415640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 0x7fd30415e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.281+0000 7fd30b415640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd30415b900 0x7fd304152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.281+0000 7fd30b415640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd304151bc0 0x7fd304152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.284 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.281+0000 7fd30b415640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd3040cd190 con 0x7fd304151bc0 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.281+0000 7fd30b415640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd3040cd010 con 0x7fd30409aa30 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.281+0000 7fd30b415640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd3040cd310 con 0x7fd30415b900 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 0x7fd30415e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3089b0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd30415b900 0x7fd304152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 0x7fd30415e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:48006/0 (socket says 172.21.15.134:48006) 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 -- 172.21.15.134:0/2817770168 learned_addr learned my addr 172.21.15.134:0/2817770168 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3099b2640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd304151bc0 0x7fd304152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 -- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd30415b900 msgr2=0x7fd304152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd30415b900 0x7fd304152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 -- 172.21.15.134:0/2817770168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd304151bc0 msgr2=0x7fd304152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd304151bc0 0x7fd304152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 -- 172.21.15.134:0/2817770168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd304159310 con 0x7fd30409aa30 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3089b0640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd30415b900 0x7fd304152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd3091b1640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 0x7fd30415e5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400e7c0 tx=0x7fd2f400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:00.285 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400ccc0 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2f4010430 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd304156f00 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4010860 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.282+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd304159890 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.283+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd2f4010a00 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.283+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd2f4018c30 con 0x7fd30409aa30 2024-09-21T05:39:00.286 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.283+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3040ccad0 con 0x7fd30409aa30 2024-09-21T05:39:00.287 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.285+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd2f4018420 con 0x7fd30409aa30 2024-09-21T05:39:00.357 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.356+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd new", "uuid": "afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2", "id": 4} v 0) -- 0x7fd30415ae90 con 0x7fd30409aa30 2024-09-21T05:39:00.360 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.359+0000 7fd2f9ffb640 1 -- 172.21.15.134:0/2817770168 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fd2f4018600 con 0x7fd30409aa30 2024-09-21T05:39:00.361 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 msgr2=0x7fd30415e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 0x7fd30415e5c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400e7c0 tx=0x7fd2f400ec80 comp rx=0 tx=0).stop 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 shutdown_connections 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd304151bc0 0x7fd304152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd30415b900 0x7fd304152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 --2- 172.21.15.134:0/2817770168 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd30409aa30 0x7fd30415e5c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.362 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 >> 172.21.15.134:0/2817770168 conn(0x7fd304074170 msgr2=0x7fd3040b5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 shutdown_connections 2024-09-21T05:39:00.363 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.360+0000 7fd30b415640 1 -- 172.21.15.134:0/2817770168 wait complete. 2024-09-21T05:39:00.372 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 2ab1fef7-ea46-4098-bd29-4d998a2ef698 5 2024-09-21T05:39:00.486 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 -- 172.21.15.134:0/2971964192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c15b8c0 msgr2=0x7f0b6c09aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 --2- 172.21.15.134:0/2971964192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c15b8c0 0x7f0b6c09aa30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009900 tx=0x7f0b6002a5c0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 -- 172.21.15.134:0/2971964192 shutdown_connections 2024-09-21T05:39:00.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 --2- 172.21.15.134:0/2971964192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c15b8c0 0x7f0b6c09aa30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 -- 172.21.15.134:0/2971964192 >> 172.21.15.134:0/2971964192 conn(0x7f0b6c06ea80 msgr2=0x7f0b6c0b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 -- 172.21.15.134:0/2971964192 shutdown_connections 2024-09-21T05:39:00.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.484+0000 7f0b7316e640 1 -- 172.21.15.134:0/2971964192 wait complete. 2024-09-21T05:39:00.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.486+0000 7f0b7316e640 1 Processor -- start 2024-09-21T05:39:00.488 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.486+0000 7f0b7316e640 1 -- start start 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7316e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b6c0aeb00 0x7f0b6c1e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7316e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 0x7f0b6c1e9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7316e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 0x7f0b6c146760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7316e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c15fc90 con 0x7f0b6c0aeb00 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7316e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6c15fb10 con 0x7f0b6c15b8c0 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7316e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6c15fe10 con 0x7f0b6c0aef10 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7170b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 0x7f0b6c146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 0x7f0b6c1e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.491 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7170b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 0x7f0b6c146760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44210/0 (socket says 172.21.15.134:44210) 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 0x7f0b6c1e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:49982/0 (socket says 172.21.15.134:49982) 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7170b640 1 -- 172.21.15.134:0/135562590 learned_addr learned my addr 172.21.15.134:0/135562590 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b70f0a640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b6c0aeb00 0x7f0b6c1e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 -- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 msgr2=0x7f0b6c146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 0x7f0b6c146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 -- 172.21.15.134:0/135562590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b6c0aeb00 msgr2=0x7f0b6c1e93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b6c0aeb00 0x7f0b6c1e93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 -- 172.21.15.134:0/135562590 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6c1f9150 con 0x7f0b6c0aef10 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b7170b640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 0x7f0b6c146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.488+0000 7f0b6bfff640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 0x7f0b6c1e9910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0ae8f0 tx=0x7f0b5800c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58017070 con 0x7f0b6c0aef10 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b580053c0 con 0x7f0b6c0aef10 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b7316e640 1 -- 172.21.15.134:0/135562590 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6c152510 con 0x7f0b6c0aef10 2024-09-21T05:39:00.492 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b58003850 con 0x7f0b6c0aef10 2024-09-21T05:39:00.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b7316e640 1 -- 172.21.15.134:0/135562590 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b6c143ba0 con 0x7f0b6c0aef10 2024-09-21T05:39:00.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0b5801a070 con 0x7f0b6c0aef10 2024-09-21T05:39:00.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.489+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f0b58011cc0 con 0x7f0b6c0aef10 2024-09-21T05:39:00.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.490+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b44005290 con 0x7f0b6c0aef10 2024-09-21T05:39:00.493 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.492+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0b58021020 con 0x7f0b6c0aef10 2024-09-21T05:39:00.564 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.562+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2ab1fef7-ea46-4098-bd29-4d998a2ef698", "id": 5} v 0) -- 0x7f0b44005580 con 0x7f0b6c0aef10 2024-09-21T05:39:00.567 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.566+0000 7f0b697fa640 1 -- 172.21.15.134:0/135562590 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2ab1fef7-ea46-4098-bd29-4d998a2ef698", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f0b58011590 con 0x7f0b6c0aef10 2024-09-21T05:39:00.567 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 msgr2=0x7f0b6c1e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 0x7f0b6c1e9910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0ae8f0 tx=0x7f0b5800c7a0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 shutdown_connections 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b6c15b8c0 0x7f0b6c146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b6c0aef10 0x7f0b6c1e9910 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 --2- 172.21.15.134:0/135562590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b6c0aeb00 0x7f0b6c1e93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 >> 172.21.15.134:0/135562590 conn(0x7f0b6c06ea80 msgr2=0x7f0b6c156930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 shutdown_connections 2024-09-21T05:39:00.569 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.567+0000 7f0b56ffd640 1 -- 172.21.15.134:0/135562590 wait complete. 2024-09-21T05:39:00.579 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 7d8b7b3f-e396-4cdf-ada4-ecae5686900c 6 2024-09-21T05:39:00.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.689+0000 7f67d328b640 1 -- 172.21.15.134:0/3743871792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc074560 msgr2=0x7f67cc15d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.689+0000 7f67d328b640 1 --2- 172.21.15.134:0/3743871792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc074560 0x7f67cc15d7a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f67cc073510 tx=0x7f67bc02a4d0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.691 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.689+0000 7f67d328b640 1 -- 172.21.15.134:0/3743871792 shutdown_connections 2024-09-21T05:39:00.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.689+0000 7f67d328b640 1 --2- 172.21.15.134:0/3743871792 >> v2:172.21.15.22:3300/0 conn(0x7f67cc06df10 0x7f67cc0b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.689+0000 7f67d328b640 1 --2- 172.21.15.134:0/3743871792 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc074560 0x7f67cc15d7a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.689+0000 7f67d328b640 1 -- 172.21.15.134:0/3743871792 >> 172.21.15.134:0/3743871792 conn(0x7f67cc0b2fe0 msgr2=0x7f67cc073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.690+0000 7f67d328b640 1 -- 172.21.15.134:0/3743871792 shutdown_connections 2024-09-21T05:39:00.692 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.690+0000 7f67d328b640 1 -- 172.21.15.134:0/3743871792 wait complete. 2024-09-21T05:39:00.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.691+0000 7f67d328b640 1 Processor -- start 2024-09-21T05:39:00.693 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.691+0000 7f67d328b640 1 -- start start 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 0x7f67cc1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 0x7f67cc155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67cc160350 0x7f67cc156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc153c80 con 0x7f67cc160350 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc153b00 con 0x7f67cc06df10 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc0b3fa0 con 0x7f67cc0b01d0 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d1027640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 0x7f67cc1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 0x7f67cc155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 0x7f67cc155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:49986/0 (socket says 172.21.15.134:49986) 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d1027640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 0x7f67cc1f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.134:44222/0 (socket says 172.21.15.134:44222) 2024-09-21T05:39:00.696 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 -- 172.21.15.134:0/3548897115 learned_addr learned my addr 172.21.15.134:0/3548897115 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d1828640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67cc160350 0x7f67cc156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 -- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 msgr2=0x7f67cc1f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 0x7f67cc1f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 -- 172.21.15.134:0/3548897115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67cc160350 msgr2=0x7f67cc156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67cc160350 0x7f67cc156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 -- 172.21.15.134:0/3548897115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc1f3280 con 0x7f67cc0b01d0 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d1027640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 0x7f67cc1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d0826640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 0x7f67cc155fd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67bc000c00 tx=0x7f67bc003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc03e030 con 0x7f67cc0b01d0 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67bc003d10 con 0x7f67cc0b01d0 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc1f34f0 con 0x7f67cc0b01d0 2024-09-21T05:39:00.697 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.693+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc035040 con 0x7f67cc0b01d0 2024-09-21T05:39:00.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.694+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc1f3ab0 con 0x7f67cc0b01d0 2024-09-21T05:39:00.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.694+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f67bc041030 con 0x7f67cc0b01d0 2024-09-21T05:39:00.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.694+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f67bc031400 con 0x7f67cc0b01d0 2024-09-21T05:39:00.698 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.695+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67cc0b4530 con 0x7f67cc0b01d0 2024-09-21T05:39:00.700 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.699+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f67bc043410 con 0x7f67cc0b01d0 2024-09-21T05:39:00.769 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.767+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7d8b7b3f-e396-4cdf-ada4-ecae5686900c", "id": 6} v 0) -- 0x7f67cc1f7270 con 0x7f67cc0b01d0 2024-09-21T05:39:00.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.772+0000 7f67c1ffb640 1 -- 172.21.15.134:0/3548897115 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7d8b7b3f-e396-4cdf-ada4-ecae5686900c", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f67bc004ee0 con 0x7f67cc0b01d0 2024-09-21T05:39:00.773 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-09-21T05:39:00.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 msgr2=0x7f67cc155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.774 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 0x7f67cc155fd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67bc000c00 tx=0x7f67bc003820 comp rx=0 tx=0).stop 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 shutdown_connections 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67cc160350 0x7f67cc156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67cc0b01d0 0x7f67cc155fd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 --2- 172.21.15.134:0/3548897115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67cc06df10 0x7f67cc1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 >> 172.21.15.134:0/3548897115 conn(0x7f67cc0b2fe0 msgr2=0x7f67cc1608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 shutdown_connections 2024-09-21T05:39:00.775 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.773+0000 7f67d328b640 1 -- 172.21.15.134:0/3548897115 wait complete. 2024-09-21T05:39:00.785 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 5e3b598f-75b2-46e0-9c5b-a3268b93badb 7 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 -- 172.21.15.134:0/200373475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 msgr2=0x7f815006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 --2- 172.21.15.134:0/200373475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815006d1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8144009860 tx=0x7f814402a510 comp rx=0 tx=0).stop 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 -- 172.21.15.134:0/200373475 shutdown_connections 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 --2- 172.21.15.134:0/200373475 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815006d1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 -- 172.21.15.134:0/200373475 >> 172.21.15.134:0/200373475 conn(0x7f8150074170 msgr2=0x7f81500b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 -- 172.21.15.134:0/200373475 shutdown_connections 2024-09-21T05:39:00.898 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.896+0000 7f81585c6640 1 -- 172.21.15.134:0/200373475 wait complete. 2024-09-21T05:39:00.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.898+0000 7f81585c6640 1 Processor -- start 2024-09-21T05:39:00.899 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.898+0000 7f81585c6640 1 -- start start 2024-09-21T05:39:00.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f81585c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f81585c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815009aa30 0x7f81500b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.902 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f81585c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81500cc4c0 0x7f81500b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f81585c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81500cd3b0 con 0x7f815009aa30 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f81585c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f81500cd230 con 0x7f81500cc4c0 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f81585c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f81500cd530 con 0x7f815006cde0 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f8156362640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f8156362640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815014da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.134:50004/0 (socket says 172.21.15.134:50004) 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f8156362640 1 -- 172.21.15.134:0/1343257017 learned_addr learned my addr 172.21.15.134:0/1343257017 (peer_addr_for_me v2:172.21.15.134:0/0) 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f8156b63640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81500cc4c0 0x7f81500b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.899+0000 7f8155b61640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815009aa30 0x7f81500b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156362640 1 -- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81500cc4c0 msgr2=0x7f81500b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156362640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81500cc4c0 0x7f81500b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156362640 1 -- 172.21.15.134:0/1343257017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815009aa30 msgr2=0x7f81500b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156362640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815009aa30 0x7f81500b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.903 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156362640 1 -- 172.21.15.134:0/1343257017 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8150156800 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156b63640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81500cc4c0 0x7f81500b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8156362640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815014da50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f814c00c400 tx=0x7f814c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c011790 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f814c017070 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815015da50 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c011cc0 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f815015e010 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f814c01c460 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.900+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f814c012df0 con 0x7f815006cde0 2024-09-21T05:39:00.904 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.901+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f815006d1f0 con 0x7f815006cde0 2024-09-21T05:39:00.905 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.903+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f814c012790 con 0x7f815006cde0 2024-09-21T05:39:00.975 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.974+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5e3b598f-75b2-46e0-9c5b-a3268b93badb", "id": 7} v 0) -- 0x7f815006d510 con 0x7f815006cde0 2024-09-21T05:39:00.979 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.977+0000 7f8142ffd640 1 -- 172.21.15.134:0/1343257017 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e3b598f-75b2-46e0-9c5b-a3268b93badb", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f814c012970 con 0x7f815006cde0 2024-09-21T05:39:00.979 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-09-21T05:39:00.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 msgr2=0x7f815014da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:00.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815014da50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f814c00c400 tx=0x7f814c00c8c0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 shutdown_connections 2024-09-21T05:39:00.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81500cc4c0 0x7f81500b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f815009aa30 0x7f81500b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.980 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 --2- 172.21.15.134:0/1343257017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f815006cde0 0x7f815014da50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:00.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 >> 172.21.15.134:0/1343257017 conn(0x7f8150074170 msgr2=0x7f815015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:00.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 shutdown_connections 2024-09-21T05:39:00.981 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:00.978+0000 7f81585c6640 1 -- 172.21.15.134:0/1343257017 wait complete. 2024-09-21T05:39:00.990 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T05:39:00.991 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T05:39:00.993 INFO:tasks.ceph.osd.0:Started 2024-09-21T05:39:00.993 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T05:39:00.993 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T05:39:00.995 INFO:tasks.ceph.osd.1:Started 2024-09-21T05:39:00.995 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T05:39:00.995 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T05:39:00.997 INFO:tasks.ceph.osd.2:Started 2024-09-21T05:39:00.997 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T05:39:00.997 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T05:39:00.999 INFO:tasks.ceph.osd.3:Started 2024-09-21T05:39:00.999 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T05:39:00.999 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T05:39:01.002 INFO:tasks.ceph.osd.4:Started 2024-09-21T05:39:01.002 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T05:39:01.002 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T05:39:01.004 INFO:tasks.ceph.osd.5:Started 2024-09-21T05:39:01.004 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T05:39:01.004 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T05:39:01.006 INFO:tasks.ceph.osd.6:Started 2024-09-21T05:39:01.006 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T05:39:01.006 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T05:39:01.008 INFO:tasks.ceph.osd.7:Started 2024-09-21T05:39:01.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.188+0000 7f3005ef9640 1 -- 172.21.15.22:0/2715437388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000ceb40 msgr2=0x7f30000cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.188+0000 7f3005ef9640 1 --2- 172.21.15.22:0/2715437388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000ceb40 0x7f30000cef50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8005fd0 tx=0x7f2ff802a740 comp rx=0 tx=0).stop 2024-09-21T05:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.189+0000 7f3005ef9640 1 -- 172.21.15.22:0/2715437388 shutdown_connections 2024-09-21T05:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.189+0000 7f3005ef9640 1 --2- 172.21.15.22:0/2715437388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000ceb40 0x7f30000cef50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.189+0000 7f3005ef9640 1 -- 172.21.15.22:0/2715437388 >> 172.21.15.22:0/2715437388 conn(0x7f300008bfc0 msgr2=0x7f300009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.189+0000 7f3005ef9640 1 -- 172.21.15.22:0/2715437388 shutdown_connections 2024-09-21T05:39:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.189+0000 7f3005ef9640 1 -- 172.21.15.22:0/2715437388 wait complete. 2024-09-21T05:39:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.192+0000 7f3005ef9640 1 Processor -- start 2024-09-21T05:39:01.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.192+0000 7f3005ef9640 1 -- start start 2024-09-21T05:39:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f3005ef9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30000b3540 0x7f30000cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f3005ef9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30000ceb40 0x7f30000ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f3005ef9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 0x7f30000b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f3005ef9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f30000d1070 con 0x7f30000cb5c0 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f3005ef9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f30000d0ef0 con 0x7f30000ceb40 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f3005ef9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f30000d11f0 con 0x7f30000b3540 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 0x7f30000b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f2fff7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30000b3540 0x7f30000cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 0x7f30000b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42240/0 (socket says 172.21.15.22:42240) 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f2ffffff640 1 -- 172.21.15.22:0/802886686 learned_addr learned my addr 172.21.15.22:0/802886686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.194+0000 7f2ffeffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30000ceb40 0x7f30000ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2ffffff640 1 -- 172.21.15.22:0/802886686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30000b3540 msgr2=0x7f30000cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2ffffff640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30000b3540 0x7f30000cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2ffffff640 1 -- 172.21.15.22:0/802886686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30000ceb40 msgr2=0x7f30000ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2ffffff640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30000ceb40 0x7f30000ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2ffffff640 1 -- 172.21.15.22:0/802886686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30001f5780 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2ffffff640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 0x7f30000b4fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009c50 tx=0x7f2ff8032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8041020 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ff8004130 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30000cc770 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8036040 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30001f5d20 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.195+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ff8044020 con 0x7f30000cb5c0 2024-09-21T05:39:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.196+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2ff803dbe0 con 0x7f30000cb5c0 2024-09-21T05:39:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.197+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30000cc4e0 con 0x7f30000cb5c0 2024-09-21T05:39:01.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.199+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2ff804b030 con 0x7f30000cb5c0 2024-09-21T05:39:01.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.281+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f300006e170 con 0x7f30000cb5c0 2024-09-21T05:39:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.282+0000 7f2fe7fff640 1 -- 172.21.15.22:0/802886686 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2ff803d510 con 0x7f30000cb5c0 2024-09-21T05:39:01.284 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:01.284 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":10,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:00.975574+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 msgr2=0x7f30000b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 0x7f30000b4fa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009c50 tx=0x7f2ff8032e90 comp rx=0 tx=0).stop 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 shutdown_connections 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30000cb5c0 0x7f30000b4fa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30000ceb40 0x7f30000ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 --2- 172.21.15.22:0/802886686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30000b3540 0x7f30000cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 >> 172.21.15.22:0/802886686 conn(0x7f300008bfc0 msgr2=0x7f300006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 shutdown_connections 2024-09-21T05:39:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:01.283+0000 7f3005ef9640 1 -- 172.21.15.22:0/802886686 wait complete. 2024-09-21T05:39:01.296 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T05:39:01.297 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T05:39:02.102 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:02.210 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f172de77640 1 -- 172.21.15.22:0/4186570561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 msgr2=0x7f1710003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:02.210 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f172de77640 1 --2- 172.21.15.22:0/4186570561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f1710003ea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1718009b50 tx=0x7f1710005830 comp rx=0 tx=0).stop 2024-09-21T05:39:02.211 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f1727fff640 1 -- 172.21.15.22:0/4186570561 reap_dead start 2024-09-21T05:39:02.211 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f172de77640 1 -- 172.21.15.22:0/4186570561 shutdown_connections 2024-09-21T05:39:02.211 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f172de77640 1 -- 172.21.15.22:0/4186570561 >> 172.21.15.22:0/4186570561 conn(0x7f17280d0a30 msgr2=0x7f17280b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:02.211 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f172de77640 1 -- 172.21.15.22:0/4186570561 shutdown_connections 2024-09-21T05:39:02.211 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.208+0000 7f172de77640 1 -- 172.21.15.22:0/4186570561 wait complete. 2024-09-21T05:39:02.212 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.210+0000 7f172de77640 1 Processor -- start 2024-09-21T05:39:02.212 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.210+0000 7f172de77640 1 -- start start 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f172de77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f17280b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f172de77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17281f3c90 0x7f1728099270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f172de77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1728158c80 0x7f17280ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f172de77640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f172815c060 con 0x7f1710003a30 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f172de77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17280cc7c0 con 0x7f1728158c80 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f172de77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f172815c1e0 con 0x7f17281f3c90 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f17277fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f17280b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f17277fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f17280b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42246/0 (socket says 172.21.15.22:42246) 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.211+0000 7f17277fe640 1 -- 172.21.15.22:0/1270888198 learned_addr learned my addr 172.21.15.22:0/1270888198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f1726ffd640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17281f3c90 0x7f1728099270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f1727fff640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1728158c80 0x7f17280ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:02.215 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f17277fe640 1 -- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17281f3c90 msgr2=0x7f1728099270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f17277fe640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17281f3c90 0x7f1728099270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f17277fe640 1 -- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1728158c80 msgr2=0x7f17280ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f17277fe640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1728158c80 0x7f17280ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f17277fe640 1 -- 172.21.15.22:0/1270888198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17280b2b80 con 0x7f1710003a30 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f17277fe640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f17280b2640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1718009c70 tx=0x7f1718033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718038040 con 0x7f1710003a30 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f171802b7e0 con 0x7f1710003a30 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17280c6730 con 0x7f1710003a30 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718004280 con 0x7f1710003a30 2024-09-21T05:39:02.216 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.212+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17281f3770 con 0x7f1710003a30 2024-09-21T05:39:02.217 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.213+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f1718045020 con 0x7f1710003a30 2024-09-21T05:39:02.217 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.213+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1718072bc0 con 0x7f1710003a30 2024-09-21T05:39:02.217 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.213+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17280cfb40 con 0x7f1710003a30 2024-09-21T05:39:02.217 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-21T05:39:02.214+0000 7efc0be14640 -1 Falling back to public interface 2024-09-21T05:39:02.218 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-09-21T05:39:02.216+0000 7f745dfa2640 -1 Falling back to public interface 2024-09-21T05:39:02.218 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.216+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1718004420 con 0x7f1710003a30 2024-09-21T05:39:02.241 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-21T05:39:02.240+0000 7f00ade5e640 -1 Falling back to public interface 2024-09-21T05:39:02.283 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.281+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1728074c20 con 0x7f1710003a30 2024-09-21T05:39:02.284 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.282+0000 7f170ffff640 1 -- 172.21.15.22:0/1270888198 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f171803de30 con 0x7f1710003a30 2024-09-21T05:39:02.284 INFO:teuthology.misc.health.smithi022.stdout: 2024-09-21T05:39:02.284 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":10,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:00.975574+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:02.285 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 msgr2=0x7f17280b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:02.285 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f17280b2640 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1718009c70 tx=0x7f1718033670 comp rx=0 tx=0).stop 2024-09-21T05:39:02.285 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 shutdown_connections 2024-09-21T05:39:02.285 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1728158c80 0x7f17280ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:02.286 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17281f3c90 0x7f1728099270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:02.286 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 --2- 172.21.15.22:0/1270888198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1710003a30 0x7f17280b2640 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:02.286 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 >> 172.21.15.22:0/1270888198 conn(0x7f17280d0a30 msgr2=0x7f172815e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:02.286 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 shutdown_connections 2024-09-21T05:39:02.286 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:02.283+0000 7f172de77640 1 -- 172.21.15.22:0/1270888198 wait complete. 2024-09-21T05:39:02.295 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T05:39:02.458 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-21T05:39:02.457+0000 7f5ae6c18640 -1 Falling back to public interface 2024-09-21T05:39:02.479 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-09-21T05:39:02.477+0000 7fdb5ae26640 -1 Falling back to public interface 2024-09-21T05:39:02.483 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-09-21T05:39:02.481+0000 7f574631e640 -1 Falling back to public interface 2024-09-21T05:39:02.501 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-21T05:39:02.499+0000 7f959dc17640 -1 Falling back to public interface 2024-09-21T05:39:02.741 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-21T05:39:02.740+0000 7fe6c27f8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T05:39:02.744 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-21T05:39:02.743+0000 7fe6c27f8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T05:39:02.755 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-21T05:39:02.753+0000 7f4188c0c640 -1 Falling back to public interface 2024-09-21T05:39:05.970 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-21T05:39:05.969+0000 7f5ae6c18640 -1 osd.6 0 log_to_monitors true 2024-09-21T05:39:06.274 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-09-21T05:39:06.272+0000 7f745dfa2640 -1 osd.3 0 log_to_monitors true 2024-09-21T05:39:06.293 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-21T05:39:06.291+0000 7f00ade5e640 -1 osd.5 0 log_to_monitors true 2024-09-21T05:39:06.328 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-21T05:39:06.326+0000 7efc0be14640 -1 osd.0 0 log_to_monitors true 2024-09-21T05:39:06.353 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-09-21T05:39:06.351+0000 7fdb5ae26640 -1 osd.2 0 log_to_monitors true 2024-09-21T05:39:06.533 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-21T05:39:06.531+0000 7f4188c0c640 -1 osd.4 0 log_to_monitors true 2024-09-21T05:39:06.621 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-09-21T05:39:06.619+0000 7f574631e640 -1 osd.1 0 log_to_monitors true 2024-09-21T05:39:06.838 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-21T05:39:06.836+0000 7f959dc17640 -1 osd.7 0 log_to_monitors true 2024-09-21T05:39:08.729 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-09-21T05:39:08.726+0000 7fdb56db4640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T05:39:08.742 INFO:tasks.ceph.osd.2.smithi022.stderr:2024-09-21T05:39:08.739+0000 7fdb4ebaa640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:08.744 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-09-21T05:39:08.741+0000 7f7459f30640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T05:39:08.760 INFO:tasks.ceph.osd.3.smithi022.stderr:2024-09-21T05:39:08.757+0000 7f7451d26640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:08.786 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-21T05:39:08.783+0000 7efc0769e640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T05:39:08.788 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-21T05:39:08.787+0000 7f00a9dec640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T05:39:08.802 INFO:tasks.ceph.osd.0.smithi022.stderr:2024-09-21T05:39:08.799+0000 7efbff494640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:08.806 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-21T05:39:08.805+0000 7f00a1be2640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:08.887 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-21T05:39:08.885+0000 7f5ae2d8e640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T05:39:08.942 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-21T05:39:08.941+0000 7f5ada36f640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:09.101 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:09.260 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 -- 172.21.15.22:0/346723302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cea70 msgr2=0x7f87c40cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:09.260 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 --2- 172.21.15.22:0/346723302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cea70 0x7f87c40cee80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87c4069e30 tx=0x7f87bc02aa30 comp rx=0 tx=0).stop 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 -- 172.21.15.22:0/346723302 shutdown_connections 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 --2- 172.21.15.22:0/346723302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cea70 0x7f87c40cee80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 --2- 172.21.15.22:0/346723302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40cd670 0x7f87c40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 --2- 172.21.15.22:0/346723302 >> v2:172.21.15.134:3300/0 conn(0x7f87c40b77b0 0x7f87c40cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 -- 172.21.15.22:0/346723302 >> 172.21.15.22:0/346723302 conn(0x7f87c40d2480 msgr2=0x7f87c40d2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 -- 172.21.15.22:0/346723302 shutdown_connections 2024-09-21T05:39:09.261 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.257+0000 7f87c90bd640 1 -- 172.21.15.22:0/346723302 wait complete. 2024-09-21T05:39:09.263 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.260+0000 7f87c90bd640 1 Processor -- start 2024-09-21T05:39:09.263 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.260+0000 7f87c90bd640 1 -- start start 2024-09-21T05:39:09.267 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c90bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40b77b0 0x7f87c40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c90bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 0x7f87c41d5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c90bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c40cea70 0x7f87c40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c90bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f87c40cf840 con 0x7f87c40cd670 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c90bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f87c40cf6c0 con 0x7f87c40cea70 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c90bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f87c40cf9c0 con 0x7f87c40b77b0 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87ba59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 0x7f87c41d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c2d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40b77b0 0x7f87c40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87ba59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 0x7f87c41d5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42616/0 (socket says 172.21.15.22:42616) 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87ba59c640 1 -- 172.21.15.22:0/3311869043 learned_addr learned my addr 172.21.15.22:0/3311869043 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.262+0000 7f87c359e640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c40cea70 0x7f87c40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87ba59c640 1 -- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40b77b0 msgr2=0x7f87c40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87ba59c640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40b77b0 0x7f87c40c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87ba59c640 1 -- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c40cea70 msgr2=0x7f87c40cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:09.268 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87ba59c640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c40cea70 0x7f87c40cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87ba59c640 1 -- 172.21.15.22:0/3311869043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c41d5ac0 con 0x7f87c40cd670 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87c2d9d640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40b77b0 0x7f87c40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87c359e640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c40cea70 0x7f87c40cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87ba59c640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 0x7f87c41d5580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00d680 tx=0x7f87ac00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac017030 con 0x7f87c40cd670 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87ac004140 con 0x7f87c40cd670 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87c40c6710 con 0x7f87c40cd670 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ac005020 con 0x7f87c40cd670 2024-09-21T05:39:09.269 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.263+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87c40c6cd0 con 0x7f87c40cd670 2024-09-21T05:39:09.270 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.265+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87c40cda80 con 0x7f87c40cd670 2024-09-21T05:39:09.270 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.266+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f87ac01a070 con 0x7f87c40cd670 2024-09-21T05:39:09.270 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.266+0000 7f87bbfff640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87a4078b00 0x7f87a407b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:09.270 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.267+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f87ac099c80 con 0x7f87c40cd670 2024-09-21T05:39:09.270 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.267+0000 7f87c2d9d640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87a4078b00 0x7f87a407b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:09.271 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.267+0000 7f87c2d9d640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87a4078b00 0x7f87a407b010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009f70 tx=0x7f87b4005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:09.278 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.275+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f87ac062860 con 0x7f87c40cd670 2024-09-21T05:39:09.381 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-09-21T05:39:09.378+0000 7f5742ac1640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T05:39:09.391 INFO:tasks.ceph.osd.1.smithi022.stderr:2024-09-21T05:39:09.388+0000 7f573a0a2640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:09.521 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.518+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87c40c64a0 con 0x7f87c40cd670 2024-09-21T05:39:09.522 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.519+0000 7f87bbfff640 1 -- 172.21.15.22:0/3311869043 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f87ac067290 con 0x7f87c40cd670 2024-09-21T05:39:09.522 INFO:teuthology.misc.health.smithi022.stdout: 2024-09-21T05:39:09.522 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":13,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:08.403422+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:09.524 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87a4078b00 msgr2=0x7f87a407b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:09.524 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87a4078b00 0x7f87a407b010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009f70 tx=0x7f87b4005f30 comp rx=0 tx=0).stop 2024-09-21T05:39:09.524 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 msgr2=0x7f87c41d5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:09.524 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 0x7f87c41d5580 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00d680 tx=0x7f87ac00db40 comp rx=0 tx=0).stop 2024-09-21T05:39:09.524 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 shutdown_connections 2024-09-21T05:39:09.524 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f87c40cea70 0x7f87c40cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.525 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f87c40cd670 0x7f87c41d5580 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.525 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87a4078b00 0x7f87a407b010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.525 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 --2- 172.21.15.22:0/3311869043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f87c40b77b0 0x7f87c40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:09.525 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 >> 172.21.15.22:0/3311869043 conn(0x7f87c40d2480 msgr2=0x7f87c40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:09.525 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 shutdown_connections 2024-09-21T05:39:09.525 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:09.521+0000 7f87c90bd640 1 -- 172.21.15.22:0/3311869043 wait complete. 2024-09-21T05:39:09.533 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T05:39:09.744 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-21T05:39:09.743+0000 7f9599600640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T05:39:09.759 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-21T05:39:09.758+0000 7f95913f6640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:09.760 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-21T05:39:09.758+0000 7f4184d32640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T05:39:09.775 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-21T05:39:09.774+0000 7f417c313640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T05:39:16.340 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.459+0000 7f7198b72640 1 -- 172.21.15.22:0/2543201861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940b3780 msgr2=0x7f71940d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.459+0000 7f7198b72640 1 --2- 172.21.15.22:0/2543201861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940b3780 0x7f71940d24c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7184009820 tx=0x7f718402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.460+0000 7f7198b72640 1 -- 172.21.15.22:0/2543201861 shutdown_connections 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.460+0000 7f7198b72640 1 --2- 172.21.15.22:0/2543201861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940b3780 0x7f71940d24c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.460+0000 7f7198b72640 1 -- 172.21.15.22:0/2543201861 >> 172.21.15.22:0/2543201861 conn(0x7f71940d46e0 msgr2=0x7f71940d4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.460+0000 7f7198b72640 1 -- 172.21.15.22:0/2543201861 shutdown_connections 2024-09-21T05:39:16.462 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.460+0000 7f7198b72640 1 -- 172.21.15.22:0/2543201861 wait complete. 2024-09-21T05:39:16.464 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.462+0000 7f7198b72640 1 Processor -- start 2024-09-21T05:39:16.464 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.462+0000 7f7198b72640 1 -- start start 2024-09-21T05:39:16.466 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7198b72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 0x7f7194152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.466 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7198b72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940d2a00 0x7f71940a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.466 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7198b72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f719415eb00 0x7f71940a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7198b72640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71940b5030 con 0x7f71940b3780 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7198b72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71940b4eb0 con 0x7f71940d2a00 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7198b72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71940b51b0 con 0x7f719415eb00 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f719259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 0x7f7194152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f719259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 0x7f7194152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47468/0 (socket says 172.21.15.22:47468) 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f719259c640 1 -- 172.21.15.22:0/1118085740 learned_addr learned my addr 172.21.15.22:0/1118085740 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7189d9b640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940d2a00 0x7f71940a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.463+0000 7f7192d9d640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f719415eb00 0x7f71940a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f719259c640 1 -- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f719415eb00 msgr2=0x7f71940a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.467 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f719259c640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f719415eb00 0x7f71940a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.468 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f719259c640 1 -- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940d2a00 msgr2=0x7f71940a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.468 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f719259c640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940d2a00 0x7f71940a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.468 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f719259c640 1 -- 172.21.15.22:0/1118085740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71940ce610 con 0x7f71940b3780 2024-09-21T05:39:16.468 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f719259c640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 0x7f7194152a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f718402aa90 tx=0x7f7184003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:16.468 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718403f070 con 0x7f71940b3780 2024-09-21T05:39:16.468 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71840049b0 con 0x7f71940b3780 2024-09-21T05:39:16.469 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719409ee10 con 0x7f71940b3780 2024-09-21T05:39:16.469 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184004e00 con 0x7f71940b3780 2024-09-21T05:39:16.469 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.464+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719409f3d0 con 0x7f71940b3780 2024-09-21T05:39:16.469 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.465+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71940b5ef0 con 0x7f71940b3780 2024-09-21T05:39:16.470 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.466+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7184042070 con 0x7f71940b3780 2024-09-21T05:39:16.470 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.467+0000 7f718b7fe640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7174078b90 0x7f717407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.470 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.467+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f71840bb810 con 0x7f71940b3780 2024-09-21T05:39:16.470 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.467+0000 7f7189d9b640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7174078b90 0x7f717407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.470 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.467+0000 7f7189d9b640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7174078b90 0x7f717407b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f719409ed40 tx=0x7f717c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:16.473 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.471+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7184083960 con 0x7f71940b3780 2024-09-21T05:39:16.664 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.662+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71940b9330 con 0x7f71940b3780 2024-09-21T05:39:16.665 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.663+0000 7f718b7fe640 1 -- 172.21.15.22:0/1118085740 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7f7184088390 con 0x7f71940b3780 2024-09-21T05:39:16.665 INFO:teuthology.misc.health.smithi022.stdout: 2024-09-21T05:39:16.665 INFO:teuthology.misc.health.smithi022.stdout:{"epoch":17,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:12.752878+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7174078b90 msgr2=0x7f717407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7174078b90 0x7f717407b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f719409ed40 tx=0x7f717c009040 comp rx=0 tx=0).stop 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 msgr2=0x7f7194152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 0x7f7194152a40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f718402aa90 tx=0x7f7184003c10 comp rx=0 tx=0).stop 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 shutdown_connections 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7174078b90 0x7f717407b0a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f719415eb00 0x7f71940a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71940d2a00 0x7f71940a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 --2- 172.21.15.22:0/1118085740 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71940b3780 0x7f7194152a40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 >> 172.21.15.22:0/1118085740 conn(0x7f71940d46e0 msgr2=0x7f719406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 shutdown_connections 2024-09-21T05:39:16.668 INFO:teuthology.misc.health.smithi022.stderr:2024-09-21T05:39:16.665+0000 7f7198b72640 1 -- 172.21.15.22:0/1118085740 wait complete. 2024-09-21T05:39:16.677 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T05:39:16.678 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T05:39:16.678 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:39:16.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:39:16.680 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:39:16.680 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:39:16.680 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:39:16.682 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:39:16.682 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:39:16.682 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:39:16.686 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:39:16.686 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:39:16.686 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:39:16.688 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:39:16.689 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T05:39:16.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:16.771 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:39:16.771 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:39:16.787 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:39:16.787 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:39:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 -- 172.21.15.22:0/3384693556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 msgr2=0x7fee700ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 --2- 172.21.15.22:0/3384693556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee700ccdd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fee70074ad0 tx=0x7fee6802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 -- 172.21.15.22:0/3384693556 shutdown_connections 2024-09-21T05:39:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 --2- 172.21.15.22:0/3384693556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee700ccdd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 --2- 172.21.15.22:0/3384693556 >> v2:172.21.15.134:3300/0 conn(0x7fee7015e850 0x7fee700d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 -- 172.21.15.22:0/3384693556 >> 172.21.15.22:0/3384693556 conn(0x7fee70072ae0 msgr2=0x7fee70072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 -- 172.21.15.22:0/3384693556 shutdown_connections 2024-09-21T05:39:16.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.809+0000 7fee7864d640 1 -- 172.21.15.22:0/3384693556 wait complete. 2024-09-21T05:39:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.811+0000 7fee7864d640 1 Processor -- start 2024-09-21T05:39:16.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.811+0000 7fee7864d640 1 -- start start 2024-09-21T05:39:16.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee7864d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee701f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee7864d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700d02e0 0x7fee70157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee7864d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee7015e850 0x7fee70153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee7864d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fee700cd310 con 0x7fee700cc9c0 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee7864d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fee700d25e0 con 0x7fee700d02e0 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee7864d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fee700cd490 con 0x7fee7015e850 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee763e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee701f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee763e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee701f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47514/0 (socket says 172.21.15.22:47514) 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.812+0000 7fee763e9640 1 -- 172.21.15.22:0/718028834 learned_addr learned my addr 172.21.15.22:0/718028834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee75be8640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700d02e0 0x7fee70157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee76bea640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee7015e850 0x7fee70153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee763e9640 1 -- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee7015e850 msgr2=0x7fee70153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee763e9640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee7015e850 0x7fee70153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee763e9640 1 -- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700d02e0 msgr2=0x7fee70157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:16.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee763e9640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700d02e0 0x7fee70157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee763e9640 1 -- 172.21.15.22:0/718028834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee700b61c0 con 0x7fee700cc9c0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee75be8640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700d02e0 0x7fee70157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee763e9640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee701f5680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee6400e790 tx=0x7fee6400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6400eea0 con 0x7fee700cc9c0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee6400cdb0 con 0x7fee700cc9c0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64010690 con 0x7fee700cc9c0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee701580c0 con 0x7fee700cc9c0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.813+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee701f5cc0 con 0x7fee700cc9c0 2024-09-21T05:39:16.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.814+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee700cd9b0 con 0x7fee700cc9c0 2024-09-21T05:39:16.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.815+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee64010830 con 0x7fee700cc9c0 2024-09-21T05:39:16.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.816+0000 7fee62ffd640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee54078b90 0x7fee5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:16.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.816+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fee6409b370 con 0x7fee700cc9c0 2024-09-21T05:39:16.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.816+0000 7fee75be8640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee54078b90 0x7fee5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:16.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.816+0000 7fee75be8640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee54078b90 0x7fee5407b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee70160b70 tx=0x7fee6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:16.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:16.820+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee64063c20 con 0x7fee700cc9c0 2024-09-21T05:39:17.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.043+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee700b2dc0 con 0x7fee700cc9c0 2024-09-21T05:39:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.044+0000 7fee62ffd640 1 -- 172.21.15.22:0/718028834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7fee64063520 con 0x7fee700cc9c0 2024-09-21T05:39:17.046 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:17.046 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":2,"btime":"2024-09-21T05:39:16:777109+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T05:39:17.046 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 2 2024-09-21T05:39:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee54078b90 msgr2=0x7fee5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee54078b90 0x7fee5407b0a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fee70160b70 tx=0x7fee6c009040 comp rx=0 tx=0).stop 2024-09-21T05:39:17.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 msgr2=0x7fee701f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee701f5680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee6400e790 tx=0x7fee6400ec50 comp rx=0 tx=0).stop 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 shutdown_connections 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee54078b90 0x7fee5407b0a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee7015e850 0x7fee70153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fee700d02e0 0x7fee70157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 --2- 172.21.15.22:0/718028834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee700cc9c0 0x7fee701f5680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 >> 172.21.15.22:0/718028834 conn(0x7fee70072ae0 msgr2=0x7fee70068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 shutdown_connections 2024-09-21T05:39:17.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:17.046+0000 7fee7864d640 1 -- 172.21.15.22:0/718028834 wait complete. 2024-09-21T05:39:19.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:19.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 -- 172.21.15.22:0/542812218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac0d0e50 msgr2=0x7fefac0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 --2- 172.21.15.22:0/542812218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac0d0e50 0x7fefac0d1260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005fd0 tx=0x7fefa402a980 comp rx=0 tx=0).stop 2024-09-21T05:39:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 -- 172.21.15.22:0/542812218 shutdown_connections 2024-09-21T05:39:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 --2- 172.21.15.22:0/542812218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac0d0e50 0x7fefac0d1260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 --2- 172.21.15.22:0/542812218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefac06e0a0 0x7fefac0b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 -- 172.21.15.22:0/542812218 >> 172.21.15.22:0/542812218 conn(0x7fefac09d840 msgr2=0x7fefac08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 -- 172.21.15.22:0/542812218 shutdown_connections 2024-09-21T05:39:19.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.189+0000 7fefb3870640 1 -- 172.21.15.22:0/542812218 wait complete. 2024-09-21T05:39:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.191+0000 7fefb3870640 1 Processor -- start 2024-09-21T05:39:19.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.191+0000 7fefb3870640 1 -- start start 2024-09-21T05:39:19.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb3870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefac06e0a0 0x7fefac1f97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb3870640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 0x7fefac0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb3870640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefac0d0e50 0x7fefac0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb3870640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fefac0cfea0 con 0x7fefac06f6e0 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb3870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fefac0cfd20 con 0x7fefac06e0a0 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb3870640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fefac0d0020 con 0x7fefac0d0e50 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb0e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 0x7fefac0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb0e0b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 0x7fefac0b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47538/0 (socket says 172.21.15.22:47538) 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.192+0000 7fefb0e0b640 1 -- 172.21.15.22:0/2255762841 learned_addr learned my addr 172.21.15.22:0/2255762841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb160c640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefac06e0a0 0x7fefac1f97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb1e0d640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefac0d0e50 0x7fefac0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb0e0b640 1 -- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefac0d0e50 msgr2=0x7fefac0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb0e0b640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefac0d0e50 0x7fefac0b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb0e0b640 1 -- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefac06e0a0 msgr2=0x7fefac1f97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb0e0b640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefac06e0a0 0x7fefac1f97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb0e0b640 1 -- 172.21.15.22:0/2255762841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac1f9cf0 con 0x7fefac06f6e0 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb0e0b640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 0x7fefac0b26f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c3a0 tx=0x7fef9c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c017070 con 0x7fefac06f6e0 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef9c0053c0 con 0x7fefac06f6e0 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac1f5950 con 0x7fefac06f6e0 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c003850 con 0x7fefac06f6e0 2024-09-21T05:39:19.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.193+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1f5ed0 con 0x7fefac06f6e0 2024-09-21T05:39:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.194+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac0d0490 con 0x7fefac06f6e0 2024-09-21T05:39:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.195+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef9c01a070 con 0x7fefac06f6e0 2024-09-21T05:39:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.196+0000 7fefa1ffb640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fef90078bc0 0x7fef9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.196+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fef9c099cd0 con 0x7fefac06f6e0 2024-09-21T05:39:19.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.196+0000 7fefb160c640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fef90078bc0 0x7fef9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.197+0000 7fefb160c640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fef90078bc0 0x7fef9007b0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009b90 tx=0x7fefa8005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:19.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.200+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fefac1f9cf0 con 0x7fefac06f6e0 2024-09-21T05:39:19.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.424+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefac06e860 con 0x7fefac06f6e0 2024-09-21T05:39:19.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.424+0000 7fefa1ffb640 1 -- 172.21.15.22:0/2255762841 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7fef9c062580 con 0x7fefac06f6e0 2024-09-21T05:39:19.427 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:19.427 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":3,"btime":"2024-09-21T05:39:17:778317+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4395,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T05:39:19.427 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 3 2024-09-21T05:39:19.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fef90078bc0 msgr2=0x7fef9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fef90078bc0 0x7fef9007b0d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009b90 tx=0x7fefa8005f30 comp rx=0 tx=0).stop 2024-09-21T05:39:19.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 msgr2=0x7fefac0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 0x7fefac0b26f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c3a0 tx=0x7fef9c00c860 comp rx=0 tx=0).stop 2024-09-21T05:39:19.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 shutdown_connections 2024-09-21T05:39:19.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fef90078bc0 0x7fef9007b0d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fefac0d0e50 0x7fefac0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fefac06f6e0 0x7fefac0b26f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 --2- 172.21.15.22:0/2255762841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fefac06e0a0 0x7fefac1f97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 >> 172.21.15.22:0/2255762841 conn(0x7fefac09d840 msgr2=0x7fefac0ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 shutdown_connections 2024-09-21T05:39:19.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.427+0000 7fefb3870640 1 -- 172.21.15.22:0/2255762841 wait complete. 2024-09-21T05:39:19.440 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:39:19.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 -- 172.21.15.22:0/2572567709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80b3540 msgr2=0x7fc5a80d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2572567709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80b3540 0x7fc5a80d3330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009860 tx=0x7fc59c02a4b0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 -- 172.21.15.22:0/2572567709 shutdown_connections 2024-09-21T05:39:19.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2572567709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5a80d3870 0x7fc5a80ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2572567709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80b3540 0x7fc5a80d3330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 -- 172.21.15.22:0/2572567709 >> 172.21.15.22:0/2572567709 conn(0x7fc5a808bfc0 msgr2=0x7fc5a809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:19.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 -- 172.21.15.22:0/2572567709 shutdown_connections 2024-09-21T05:39:19.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.554+0000 7fc5aec93640 1 -- 172.21.15.22:0/2572567709 wait complete. 2024-09-21T05:39:19.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.556+0000 7fc5aec93640 1 Processor -- start 2024-09-21T05:39:19.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.556+0000 7fc5aec93640 1 -- start start 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.557+0000 7fc5aec93640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a80b3540 0x7fc5a814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.557+0000 7fc5aec93640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 0x7fc5a80cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.557+0000 7fc5aec93640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5a815fca0 0x7fc5a80ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.557+0000 7fc5aec93640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d0e60 con 0x7fc5a80d3870 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.557+0000 7fc5aec93640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d0ce0 con 0x7fc5a80b3540 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.557+0000 7fc5aec93640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80d0fe0 con 0x7fc5a815fca0 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 0x7fc5a80cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 0x7fc5a80cdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47560/0 (socket says 172.21.15.22:47560) 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 -- 172.21.15.22:0/2922011247 learned_addr learned my addr 172.21.15.22:0/2922011247 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5aca2f640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a80b3540 0x7fc5a814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5ad230640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5a815fca0 0x7fc5a80ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 -- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5a815fca0 msgr2=0x7fc5a80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5a815fca0 0x7fc5a80ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 -- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a80b3540 msgr2=0x7fc5a814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a80b3540 0x7fc5a814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 -- 172.21.15.22:0/2922011247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80c8510 con 0x7fc5a80d3870 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a7fff640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 0x7fc5a80cdc20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc59c000c00 tx=0x7fc59c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c031d60 con 0x7fc5a80d3870 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a80cb210 con 0x7fc5a80d3870 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc59c033e10 con 0x7fc5a80d3870 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c033310 con 0x7fc5a80d3870 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.558+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a80c8a90 con 0x7fc5a80d3870 2024-09-21T05:39:19.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.559+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a80d1500 con 0x7fc5a80d3870 2024-09-21T05:39:19.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.560+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc59c0334b0 con 0x7fc5a80d3870 2024-09-21T05:39:19.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.561+0000 7fc5a57fa640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc594078b90 0x7fc59407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.561+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc59c0bbda0 con 0x7fc5a80d3870 2024-09-21T05:39:19.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.561+0000 7fc5aca2f640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc594078b90 0x7fc59407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.561+0000 7fc5aca2f640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc594078b90 0x7fc59407b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc598007fd0 tx=0x7fc598001390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:19.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.565+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc59c03f030 con 0x7fc5a80d3870 2024-09-21T05:39:19.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.755+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc5a80caf80 con 0x7fc5a80d3870 2024-09-21T05:39:19.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.755+0000 7fc5a57fa640 1 -- 172.21.15.22:0/2922011247 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7fc59c083e50 con 0x7fc5a80d3870 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:19.758 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:39:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc594078b90 msgr2=0x7fc59407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc594078b90 0x7fc59407b0a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc598007fd0 tx=0x7fc598001390 comp rx=0 tx=0).stop 2024-09-21T05:39:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 msgr2=0x7fc5a80cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 0x7fc5a80cdc20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc59c000c00 tx=0x7fc59c033c20 comp rx=0 tx=0).stop 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 shutdown_connections 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc594078b90 0x7fc59407b0a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc5a815fca0 0x7fc5a80ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc5a80d3870 0x7fc5a80cdc20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 --2- 172.21.15.22:0/2922011247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5a80b3540 0x7fc5a814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 >> 172.21.15.22:0/2922011247 conn(0x7fc5a808bfc0 msgr2=0x7fc5a806ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 shutdown_connections 2024-09-21T05:39:19.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.758+0000 7fc5aec93640 1 -- 172.21.15.22:0/2922011247 wait complete. 2024-09-21T05:39:19.771 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:39:19.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/3248210455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740cf540 msgr2=0x7fbf7415e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/3248210455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740cf540 0x7fbf7415e870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbf7408ca30 tx=0x7fbf6402a4d0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/3248210455 shutdown_connections 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/3248210455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740cf540 0x7fbf7415e870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/3248210455 >> v2:172.21.15.134:3300/0 conn(0x7fbf7406fa70 0x7fbf7406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/3248210455 >> 172.21.15.22:0/3248210455 conn(0x7fbf7408dc50 msgr2=0x7fbf7409d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/3248210455 shutdown_connections 2024-09-21T05:39:19.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.890+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/3248210455 wait complete. 2024-09-21T05:39:19.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.892+0000 7fbf7b6a4640 1 Processor -- start 2024-09-21T05:39:19.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.892+0000 7fbf7b6a4640 1 -- start start 2024-09-21T05:39:19.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf7b6a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf7406fa70 0x7fbf740a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf7b6a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf740cf540 0x7fbf740b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf7b6a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 0x7fbf741f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf7b6a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74162b20 con 0x7fbf740d0ef0 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf7b6a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741629a0 con 0x7fbf740cf540 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf7b6a4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbf74162ca0 con 0x7fbf7406fa70 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf79c41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 0x7fbf741f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf79c41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 0x7fbf741f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47584/0 (socket says 172.21.15.22:47584) 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.893+0000 7fbf79c41640 1 -- 172.21.15.22:0/2353410013 learned_addr learned my addr 172.21.15.22:0/2353410013 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf78c3f640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf740cf540 0x7fbf740b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79440640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf7406fa70 0x7fbf740a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79c41640 1 -- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf7406fa70 msgr2=0x7fbf740a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79c41640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf7406fa70 0x7fbf740a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79c41640 1 -- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf740cf540 msgr2=0x7fbf740b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79c41640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf740cf540 0x7fbf740b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79c41640 1 -- 172.21.15.22:0/2353410013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7409ea40 con 0x7fbf740d0ef0 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf79c41640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 0x7fbf741f9290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b9c0 tx=0x7fbf6c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c019070 con 0x7fbf740d0ef0 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf6c00cc90 con 0x7fbf740d0ef0 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0146a0 con 0x7fbf740d0ef0 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7409ed20 con 0x7fbf740d0ef0 2024-09-21T05:39:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.894+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf7409f2e0 con 0x7fbf740d0ef0 2024-09-21T05:39:19.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.895+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf740d03a0 con 0x7fbf740d0ef0 2024-09-21T05:39:19.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.896+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf6c01c070 con 0x7fbf740d0ef0 2024-09-21T05:39:19.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.897+0000 7fbf69ffb640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbf58078b90 0x7fbf5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:19.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.897+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbf6c09a740 con 0x7fbf740d0ef0 2024-09-21T05:39:19.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.897+0000 7fbf79440640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbf58078b90 0x7fbf5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:19.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.898+0000 7fbf79440640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbf58078b90 0x7fbf5807b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf700058c0 tx=0x7fbf70005f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:19.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:19.901+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf6c062880 con 0x7fbf740d0ef0 2024-09-21T05:39:20.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.090+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbf74163110 con 0x7fbf740d0ef0 2024-09-21T05:39:20.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.796+0000 7fbf69ffb640 1 -- 172.21.15.22:0/2353410013 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fbf6c0672b0 con 0x7fbf740d0ef0 2024-09-21T05:39:20.799 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:39:20.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbf58078b90 msgr2=0x7fbf5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:20.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbf58078b90 0x7fbf5807b0a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf700058c0 tx=0x7fbf70005f60 comp rx=0 tx=0).stop 2024-09-21T05:39:20.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 msgr2=0x7fbf741f9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:20.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 0x7fbf741f9290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00b9c0 tx=0x7fbf6c00be80 comp rx=0 tx=0).stop 2024-09-21T05:39:20.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 shutdown_connections 2024-09-21T05:39:20.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf740d0ef0 0x7fbf741f9290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:20.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf740cf540 0x7fbf740b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:20.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbf58078b90 0x7fbf5807b0a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:20.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 --2- 172.21.15.22:0/2353410013 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf7406fa70 0x7fbf740a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:20.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.801+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 >> 172.21.15.22:0/2353410013 conn(0x7fbf7408dc50 msgr2=0x7fbf7415ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:20.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.802+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 shutdown_connections 2024-09-21T05:39:20.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.802+0000 7fbf7b6a4640 1 -- 172.21.15.22:0/2353410013 wait complete. 2024-09-21T05:39:20.821 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:39:20.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:39:20.868 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:39:20.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 -- 172.21.15.22:0/3133444915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 msgr2=0x7f7c600b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 --2- 172.21.15.22:0/3133444915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c600b45a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c6006eeb0 tx=0x7f7c5002a5f0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 -- 172.21.15.22:0/3133444915 shutdown_connections 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 --2- 172.21.15.22:0/3133444915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c600b45a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 --2- 172.21.15.22:0/3133444915 >> v2:172.21.15.134:3300/0 conn(0x7f7c60074420 0x7f7c6006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 -- 172.21.15.22:0/3133444915 >> 172.21.15.22:0/3133444915 conn(0x7f7c6006def0 msgr2=0x7f7c6006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 -- 172.21.15.22:0/3133444915 shutdown_connections 2024-09-21T05:39:21.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.951+0000 7f7c65333640 1 -- 172.21.15.22:0/3133444915 wait complete. 2024-09-21T05:39:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.953+0000 7f7c65333640 1 Processor -- start 2024-09-21T05:39:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.953+0000 7f7c65333640 1 -- start start 2024-09-21T05:39:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.954+0000 7f7c65333640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c60074420 0x7f7c60156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.954+0000 7f7c65333640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c6015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.954+0000 7f7c65333640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c6014aa80 0x7f7c6006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.954+0000 7f7c65333640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600b5030 con 0x7f7c60155ce0 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.954+0000 7f7c65333640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600b4eb0 con 0x7f7c60074420 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.954+0000 7f7c65333640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c600b51b0 con 0x7f7c6014aa80 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c6015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c6015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47604/0 (socket says 172.21.15.22:47604) 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 -- 172.21.15.22:0/468986253 learned_addr learned my addr 172.21.15.22:0/468986253 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5effd640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c60074420 0x7f7c60156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5f7fe640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c6014aa80 0x7f7c6006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 -- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c6014aa80 msgr2=0x7f7c6006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c6014aa80 0x7f7c6006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 -- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c60074420 msgr2=0x7f7c60156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c60074420 0x7f7c60156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 -- 172.21.15.22:0/468986253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6013d940 con 0x7f7c60155ce0 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c5e7fc640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c6015b320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c48001eb0 tx=0x7f7c4800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:21.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48017070 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c480053c0 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6013d660 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48003830 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.955+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6013ad30 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.956+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c600b56d0 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.958+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c4801a070 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.958+0000 7f7c577fe640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c40078b90 0x7f7c4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.959+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f7c4809a2f0 con 0x7f7c60155ce0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.959+0000 7f7c5effd640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c40078b90 0x7f7c4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.959+0000 7f7c5effd640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c40078b90 0x7f7c4007b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005fd0 tx=0x7f7c58009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:21.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:20.963+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c48021020 con 0x7f7c60155ce0 2024-09-21T05:39:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.151+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7c6015dcc0 con 0x7f7c60155ce0 2024-09-21T05:39:21.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.803+0000 7f7c577fe640 1 -- 172.21.15.22:0/468986253 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f7c480622a0 con 0x7f7c60155ce0 2024-09-21T05:39:21.805 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c40078b90 msgr2=0x7f7c4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c40078b90 0x7f7c4007b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7c58005fd0 tx=0x7f7c58009500 comp rx=0 tx=0).stop 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 msgr2=0x7f7c6015b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c6015b320 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c48001eb0 tx=0x7f7c4800c890 comp rx=0 tx=0).stop 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 shutdown_connections 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c40078b90 0x7f7c4007b0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c6014aa80 0x7f7c6006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c60155ce0 0x7f7c6015b320 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 --2- 172.21.15.22:0/468986253 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c60074420 0x7f7c60156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 >> 172.21.15.22:0/468986253 conn(0x7f7c6006def0 msgr2=0x7f7c600cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 shutdown_connections 2024-09-21T05:39:21.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.806+0000 7f7c65333640 1 -- 172.21.15.22:0/468986253 wait complete. 2024-09-21T05:39:21.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 -- 172.21.15.22:0/1339651876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c15fc00 msgr2=0x7f581c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 --2- 172.21.15.22:0/1339651876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c15fc00 0x7f581c06fdb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f581c073510 tx=0x7f580c02aa30 comp rx=0 tx=0).stop 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 -- 172.21.15.22:0/1339651876 shutdown_connections 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 --2- 172.21.15.22:0/1339651876 >> v2:172.21.15.134:3300/0 conn(0x7f581c06e500 0x7f581c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 --2- 172.21.15.22:0/1339651876 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c15fc00 0x7f581c06fdb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 -- 172.21.15.22:0/1339651876 >> 172.21.15.22:0/1339651876 conn(0x7f581c0d1210 msgr2=0x7f581c073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:21.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 -- 172.21.15.22:0/1339651876 shutdown_connections 2024-09-21T05:39:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.957+0000 7f5821f94640 1 -- 172.21.15.22:0/1339651876 wait complete. 2024-09-21T05:39:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.959+0000 7f5821f94640 1 Processor -- start 2024-09-21T05:39:21.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.959+0000 7f5821f94640 1 -- start start 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 0x7f581c1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f581c15fc00 0x7f581c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f581c0cf0f0 0x7f581c13ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f581c15c400 con 0x7f581c0702f0 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f581c15c280 con 0x7f581c0cf0f0 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f581c15c580 con 0x7f581c15fc00 2024-09-21T05:39:21.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 0x7f581c1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 0x7f581c1f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47610/0 (socket says 172.21.15.22:47610) 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 -- 172.21.15.22:0/10770506 learned_addr learned my addr 172.21.15.22:0/10770506 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581bfff640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f581c0cf0f0 0x7f581c13ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581affd640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f581c15fc00 0x7f581c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 -- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f581c15fc00 msgr2=0x7f581c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f581c15fc00 0x7f581c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 -- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f581c0cf0f0 msgr2=0x7f581c13ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f581c0cf0f0 0x7f581c13ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 -- 172.21.15.22:0/10770506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c1f3d80 con 0x7f581c0702f0 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581bfff640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f581c0cf0f0 0x7f581c13ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581affd640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f581c15fc00 0x7f581c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f581b7fe640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 0x7f581c1f3130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d60 tx=0x7f580c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c033050 con 0x7f581c0702f0 2024-09-21T05:39:21.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f580c0044f0 con 0x7f581c0702f0 2024-09-21T05:39:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c038450 con 0x7f581c0702f0 2024-09-21T05:39:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.961+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c1f4060 con 0x7f581c0702f0 2024-09-21T05:39:21.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.962+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c1f4670 con 0x7f581c0702f0 2024-09-21T05:39:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.962+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f581c0b1950 con 0x7f581c0702f0 2024-09-21T05:39:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.963+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f580c036070 con 0x7f581c0702f0 2024-09-21T05:39:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.964+0000 7f5813fff640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57fc078b90 0x7f57fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.964+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f580c0bc350 con 0x7f581c0702f0 2024-09-21T05:39:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.964+0000 7f581affd640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57fc078b90 0x7f57fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:21.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.965+0000 7f581affd640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57fc078b90 0x7f57fc07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f581c0b1240 tx=0x7f5804009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:21.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:21.969+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f581c0b1950 con 0x7f581c0702f0 2024-09-21T05:39:22.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.194+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f581c15bb30 con 0x7f581c0702f0 2024-09-21T05:39:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.806+0000 7f5813fff640 1 -- 172.21.15.22:0/10770506 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f580c084750 con 0x7f581c0702f0 2024-09-21T05:39:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T05:39:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57fc078b90 msgr2=0x7f57fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57fc078b90 0x7f57fc07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f581c0b1240 tx=0x7f5804009040 comp rx=0 tx=0).stop 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 msgr2=0x7f581c1f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 0x7f581c1f3130 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d60 tx=0x7f580c003f10 comp rx=0 tx=0).stop 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 shutdown_connections 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f581c0cf0f0 0x7f581c13ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57fc078b90 0x7f57fc07b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f581c15fc00 0x7f581c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 --2- 172.21.15.22:0/10770506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f581c0702f0 0x7f581c1f3130 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f580c009d60 tx=0x7f580c003f10 comp rx=0 tx=0).stop 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.811+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 >> 172.21.15.22:0/10770506 conn(0x7f581c0d1210 msgr2=0x7f581c0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.812+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 shutdown_connections 2024-09-21T05:39:22.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.813+0000 7f5821f94640 1 -- 172.21.15.22:0/10770506 wait complete. 2024-09-21T05:39:22.854 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:22.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 -- 172.21.15.22:0/2115228382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 msgr2=0x7f071c0cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:22.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 --2- 172.21.15.22:0/2115228382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 0x7f071c0cf9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0714009830 tx=0x7f071402a510 comp rx=0 tx=0).stop 2024-09-21T05:39:22.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 -- 172.21.15.22:0/2115228382 shutdown_connections 2024-09-21T05:39:22.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 --2- 172.21.15.22:0/2115228382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 0x7f071c0cf9e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 --2- 172.21.15.22:0/2115228382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f071c0d1a80 0x7f071c0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 -- 172.21.15.22:0/2115228382 >> 172.21.15.22:0/2115228382 conn(0x7f071c08bfc0 msgr2=0x7f071c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 -- 172.21.15.22:0/2115228382 shutdown_connections 2024-09-21T05:39:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.973+0000 7f07235a8640 1 -- 172.21.15.22:0/2115228382 wait complete. 2024-09-21T05:39:22.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.975+0000 7f07235a8640 1 Processor -- start 2024-09-21T05:39:22.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.975+0000 7f07235a8640 1 -- start start 2024-09-21T05:39:22.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.976+0000 7f07235a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 0x7f071c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.976+0000 7f07235a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f071c0d1a80 0x7f071c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.976+0000 7f07235a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 0x7f071c0b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.976+0000 7f07235a8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f071c162cb0 con 0x7f071c0d2ed0 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.976+0000 7f07235a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f071c162b30 con 0x7f071c0cf5d0 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.976+0000 7f07235a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f071c162e30 con 0x7f071c0d1a80 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 0x7f071c0b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 0x7f071c0b8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47626/0 (socket says 172.21.15.22:47626) 2024-09-21T05:39:22.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 -- 172.21.15.22:0/1735062777 learned_addr learned my addr 172.21.15.22:0/1735062777 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0720b43640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f071c0d1a80 0x7f071c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721344640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 0x7f071c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 -- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f071c0d1a80 msgr2=0x7f071c0b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f071c0d1a80 0x7f071c0b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 -- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 msgr2=0x7f071c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 0x7f071c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 -- 172.21.15.22:0/1735062777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071c1f93c0 con 0x7f071c0d2ed0 2024-09-21T05:39:22.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0721b45640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 0x7f071c0b8a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f071402ab00 tx=0x7f0714033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:22.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714005c90 con 0x7f071c0d2ed0 2024-09-21T05:39:22.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0714031e50 con 0x7f071c0d2ed0 2024-09-21T05:39:22.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071c1f96a0 con 0x7f071c0d2ed0 2024-09-21T05:39:22.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.977+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714005220 con 0x7f071c0d2ed0 2024-09-21T05:39:22.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.978+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071c1f9c60 con 0x7f071c0d2ed0 2024-09-21T05:39:22.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.978+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f071c1633c0 con 0x7f071c0d2ed0 2024-09-21T05:39:22.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.979+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07140053c0 con 0x7f071c0d2ed0 2024-09-21T05:39:22.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.980+0000 7f0711ffb640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0700078b90 0x7f070007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:22.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.980+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f07140bbcc0 con 0x7f071c0d2ed0 2024-09-21T05:39:22.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.980+0000 7f0721344640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0700078b90 0x7f070007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:22.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.980+0000 7f0721344640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0700078b90 0x7f070007b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0718007fd0 tx=0x7f0718009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:22.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:22.984+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f071403f030 con 0x7f071c0d2ed0 2024-09-21T05:39:23.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.173+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f071c0b47e0 con 0x7f071c0d2ed0 2024-09-21T05:39:23.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.174+0000 7f0711ffb640 1 -- 172.21.15.22:0/1735062777 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16666 (secure 0 0 0) 0x7f0714031770 con 0x7f071c0d2ed0 2024-09-21T05:39:23.176 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:23.176 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:22.195894+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0700078b90 msgr2=0x7f070007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0700078b90 0x7f070007b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0718007fd0 tx=0x7f0718009040 comp rx=0 tx=0).stop 2024-09-21T05:39:23.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 msgr2=0x7f071c0b8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 0x7f071c0b8a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f071402ab00 tx=0x7f0714033c20 comp rx=0 tx=0).stop 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 shutdown_connections 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f071c0d2ed0 0x7f071c0b8a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0700078b90 0x7f070007b0a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f071c0d1a80 0x7f071c0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 --2- 172.21.15.22:0/1735062777 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f071c0cf5d0 0x7f071c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 >> 172.21.15.22:0/1735062777 conn(0x7f071c08bfc0 msgr2=0x7f071c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 shutdown_connections 2024-09-21T05:39:23.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.176+0000 7f07235a8640 1 -- 172.21.15.22:0/1735062777 wait complete. 2024-09-21T05:39:23.190 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:23.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 -- 172.21.15.22:0/1103063361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65d0003a50 msgr2=0x7f65d0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 --2- 172.21.15.22:0/1103063361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65d0003a50 0x7f65d0003ec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c3f0 tx=0x7f65d0005830 comp rx=0 tx=0).stop 2024-09-21T05:39:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 -- 172.21.15.22:0/1103063361 shutdown_connections 2024-09-21T05:39:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 --2- 172.21.15.22:0/1103063361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65d0003a50 0x7f65d0003ec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 -- 172.21.15.22:0/1103063361 >> 172.21.15.22:0/1103063361 conn(0x7f65e408bfc0 msgr2=0x7f65e409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 -- 172.21.15.22:0/1103063361 shutdown_connections 2024-09-21T05:39:23.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.304+0000 7f65eac4a640 1 -- 172.21.15.22:0/1103063361 wait complete. 2024-09-21T05:39:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.306+0000 7f65eac4a640 1 Processor -- start 2024-09-21T05:39:23.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.306+0000 7f65eac4a640 1 -- start start 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65eac4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65d0003a50 0x7f65e41f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65eac4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 0x7f65e4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65eac4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65e4160900 0x7f65e4160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65eac4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40cf230 con 0x7f65e4158e80 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65eac4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40cf0b0 con 0x7f65e4160900 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65eac4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f65e40cf3b0 con 0x7f65d0003a50 2024-09-21T05:39:23.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 0x7f65e4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 0x7f65e4150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47636/0 (socket says 172.21.15.22:47636) 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65e3fff640 1 -- 172.21.15.22:0/4248329761 learned_addr learned my addr 172.21.15.22:0/4248329761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65e89e6640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65d0003a50 0x7f65e41f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.307+0000 7f65e91e7640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65e4160900 0x7f65e4160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e3fff640 1 -- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65d0003a50 msgr2=0x7f65e41f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e3fff640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65d0003a50 0x7f65e41f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e3fff640 1 -- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65e4160900 msgr2=0x7f65e4160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e3fff640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65e4160900 0x7f65e4160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e3fff640 1 -- 172.21.15.22:0/4248329761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e4161250 con 0x7f65e4158e80 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e3fff640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 0x7f65e4150a30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65d4011d40 tx=0x7f65d400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4003d80 con 0x7f65e4158e80 2024-09-21T05:39:23.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65d4004b20 con 0x7f65e4158e80 2024-09-21T05:39:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e41596d0 con 0x7f65e4158e80 2024-09-21T05:39:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4004050 con 0x7f65e4158e80 2024-09-21T05:39:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.308+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e41f98c0 con 0x7f65e4158e80 2024-09-21T05:39:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.309+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e40d24a0 con 0x7f65e4158e80 2024-09-21T05:39:23.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.310+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f65d40041f0 con 0x7f65e4158e80 2024-09-21T05:39:23.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.310+0000 7f65e17fa640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f65c4078bc0 0x7f65c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.310+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f65d409bdf0 con 0x7f65e4158e80 2024-09-21T05:39:23.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.310+0000 7f65e89e6640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f65c4078bc0 0x7f65c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.311+0000 7f65e89e6640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f65c4078bc0 0x7f65c407b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc000fe0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.315+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65e4161250 con 0x7f65e4158e80 2024-09-21T05:39:23.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.504+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65e40b59d0 con 0x7f65e4158e80 2024-09-21T05:39:23.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.506+0000 7f65e17fa640 1 -- 172.21.15.22:0/4248329761 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16666 (secure 0 0 0) 0x7f65d4063c70 con 0x7f65e4158e80 2024-09-21T05:39:23.508 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:23.508 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":20,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:22.195894+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f65c4078bc0 msgr2=0x7f65c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f65c4078bc0 0x7f65c407b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f65cc033000 tx=0x7f65cc000fe0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 msgr2=0x7f65e4150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 0x7f65e4150a30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f65d4011d40 tx=0x7f65d400f790 comp rx=0 tx=0).stop 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 shutdown_connections 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f65e4160900 0x7f65e4160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f65e4158e80 0x7f65e4150a30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f65c4078bc0 0x7f65c407b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 --2- 172.21.15.22:0/4248329761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f65d0003a50 0x7f65e41f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 >> 172.21.15.22:0/4248329761 conn(0x7f65e408bfc0 msgr2=0x7f65e40b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 shutdown_connections 2024-09-21T05:39:23.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.508+0000 7f65eac4a640 1 -- 172.21.15.22:0/4248329761 wait complete. 2024-09-21T05:39:23.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:39:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.640+0000 7f3eea971640 1 -- 172.21.15.22:0/4203620151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ecc003a30 msgr2=0x7f3ecc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.640+0000 7f3eea971640 1 --2- 172.21.15.22:0/4203620151 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ecc003a30 0x7f3ecc003ea0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3ed40016c0 tx=0x7f3ecc005830 comp rx=0 tx=0).stop 2024-09-21T05:39:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.640+0000 7f3ee8f0e640 1 -- 172.21.15.22:0/4203620151 reap_dead start 2024-09-21T05:39:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.640+0000 7f3eea971640 1 -- 172.21.15.22:0/4203620151 shutdown_connections 2024-09-21T05:39:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.641+0000 7f3eea971640 1 -- 172.21.15.22:0/4203620151 >> 172.21.15.22:0/4203620151 conn(0x7f3ee408bfc0 msgr2=0x7f3ee409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:23.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.641+0000 7f3eea971640 1 -- 172.21.15.22:0/4203620151 shutdown_connections 2024-09-21T05:39:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.641+0000 7f3eea971640 1 -- 172.21.15.22:0/4203620151 wait complete. 2024-09-21T05:39:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.642+0000 7f3eea971640 1 Processor -- start 2024-09-21T05:39:23.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.643+0000 7f3eea971640 1 -- start start 2024-09-21T05:39:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3eea971640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ecc003a30 0x7f3ee41f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3eea971640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee41519d0 0x7f3ee40c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3eea971640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 0x7f3ee41f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3eea971640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40cf2e0 con 0x7f3ee40c8560 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3eea971640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee40cf160 con 0x7f3ee41519d0 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3eea971640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee40d0ce0 con 0x7f3ecc003a30 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 0x7f3ee41f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 0x7f3ee41f87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:47650/0 (socket says 172.21.15.22:47650) 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 -- 172.21.15.22:0/534777053 learned_addr learned my addr 172.21.15.22:0/534777053 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee3fff640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ecc003a30 0x7f3ee41f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee37fe640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee41519d0 0x7f3ee40c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 -- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ecc003a30 msgr2=0x7f3ee41f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ecc003a30 0x7f3ee41f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 -- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee41519d0 msgr2=0x7f3ee40c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee41519d0 0x7f3ee40c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.644+0000 7f3ee8f0e640 1 -- 172.21.15.22:0/534777053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee40c8970 con 0x7f3ee40c8560 2024-09-21T05:39:23.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.645+0000 7f3ee8f0e640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 0x7f3ee41f87f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.645+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc004b20 con 0x7f3ee40c8560 2024-09-21T05:39:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.645+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3edc004cc0 con 0x7f3ee40c8560 2024-09-21T05:39:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.645+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee40ca650 con 0x7f3ee40c8560 2024-09-21T05:39:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.645+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3edc003bd0 con 0x7f3ee40c8560 2024-09-21T05:39:23.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.645+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee40cac10 con 0x7f3ee40c8560 2024-09-21T05:39:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.646+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee406e170 con 0x7f3ee40c8560 2024-09-21T05:39:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.647+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3edc004030 con 0x7f3ee40c8560 2024-09-21T05:39:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.647+0000 7f3ee0ff9640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.647+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3edc09b0a0 con 0x7f3ee40c8560 2024-09-21T05:39:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.647+0000 7f3ee3fff640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:23.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.648+0000 7f3ee3fff640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ed0078b90 0x7f3ed007b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4032000 tx=0x7f3ed402c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:23.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.652+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ee40c8970 con 0x7f3ee40c8560 2024-09-21T05:39:23.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:23.882+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3ee40d3f10 con 0x7f3ee40c8560 2024-09-21T05:39:24.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.831+0000 7f3ee0ff9640 1 -- 172.21.15.22:0/534777053 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f3edc062f20 con 0x7f3ee40c8560 2024-09-21T05:39:24.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ed0078b90 msgr2=0x7f3ed007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:24.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ed0078b90 0x7f3ed007b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4032000 tx=0x7f3ed402c040 comp rx=0 tx=0).stop 2024-09-21T05:39:24.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 msgr2=0x7f3ee41f87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:24.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 0x7f3ee41f87f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3edc005fd0 tx=0x7f3edc00f3c0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 shutdown_connections 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee40c8560 0x7f3ee41f87f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee41519d0 0x7f3ee40c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ed0078b90 0x7f3ed007b0a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 --2- 172.21.15.22:0/534777053 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ecc003a30 0x7f3ee41f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 >> 172.21.15.22:0/534777053 conn(0x7f3ee408bfc0 msgr2=0x7f3ee40b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 shutdown_connections 2024-09-21T05:39:24.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.835+0000 7f3eea971640 1 -- 172.21.15.22:0/534777053 wait complete. 2024-09-21T05:39:24.851 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:39:24.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 -- 172.21.15.22:0/431508230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0d2ed0 msgr2=0x7f3a3c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 --2- 172.21.15.22:0/431508230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0d2ed0 0x7f3a3c0d1a80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009820 tx=0x7f3a3402a510 comp rx=0 tx=0).stop 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 -- 172.21.15.22:0/431508230 shutdown_connections 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 --2- 172.21.15.22:0/431508230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0d2ed0 0x7f3a3c0d1a80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 -- 172.21.15.22:0/431508230 >> 172.21.15.22:0/431508230 conn(0x7f3a3c08bfc0 msgr2=0x7f3a3c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 -- 172.21.15.22:0/431508230 shutdown_connections 2024-09-21T05:39:24.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.975+0000 7f3a43f58640 1 -- 172.21.15.22:0/431508230 wait complete. 2024-09-21T05:39:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.977+0000 7f3a43f58640 1 Processor -- start 2024-09-21T05:39:24.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.977+0000 7f3a43f58640 1 -- start start 2024-09-21T05:39:24.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a3c0b9b70 0x7f3a3c1f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:24.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 0x7f3a3c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a3c0d2ed0 0x7f3a3c0b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c162f70 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3c162df0 con 0x7f3a3c0b9b70 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3a3c1630f0 con 0x7f3a3c0d2ed0 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 0x7f3a3c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 0x7f3a3c155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46458/0 (socket says 172.21.15.22:46458) 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 -- 172.21.15.22:0/3051813701 learned_addr learned my addr 172.21.15.22:0/3051813701 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a41cf4640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a3c0b9b70 0x7f3a3c1f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:24.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a424f5640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a3c0d2ed0 0x7f3a3c0b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 -- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a3c0d2ed0 msgr2=0x7f3a3c0b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a3c0d2ed0 0x7f3a3c0b8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 -- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a3c0b9b70 msgr2=0x7f3a3c1f64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a3c0b9b70 0x7f3a3c1f64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 -- 172.21.15.22:0/3051813701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3c0c8190 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a414f3640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 0x7f3a3c155c60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c3c0 tx=0x7f3a3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30017070 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a300035f0 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3c1561a0 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30003a40 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.979+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a3c0c8710 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.980+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3c0d1a80 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.981+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a3001a070 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.982+0000 7f3a3a7fc640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a28078bc0 0x7f3a2807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.982+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3a30099fe0 con 0x7f3a3c0cf7a0 2024-09-21T05:39:24.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.982+0000 7f3a41cf4640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a28078bc0 0x7f3a2807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:24.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.982+0000 7f3a41cf4640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a28078bc0 0x7f3a2807b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a34034000 tx=0x7f3a34004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:24.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:24.986+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a300624b0 con 0x7f3a3c0cf7a0 2024-09-21T05:39:25.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.212+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3a3c0b4020 con 0x7f3a3c0cf7a0 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.837+0000 7f3a3a7fc640 1 -- 172.21.15.22:0/3051813701 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f3a30061db0 con 0x7f3a3c0cf7a0 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a28078bc0 msgr2=0x7f3a2807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a28078bc0 0x7f3a2807b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a34034000 tx=0x7f3a34004730 comp rx=0 tx=0).stop 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 msgr2=0x7f3a3c155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 0x7f3a3c155c60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000c3c0 tx=0x7f3a3000c880 comp rx=0 tx=0).stop 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 shutdown_connections 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a28078bc0 0x7f3a2807b0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a3c0d2ed0 0x7f3a3c0b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a3c0cf7a0 0x7f3a3c155c60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 --2- 172.21.15.22:0/3051813701 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a3c0b9b70 0x7f3a3c1f64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 >> 172.21.15.22:0/3051813701 conn(0x7f3a3c08bfc0 msgr2=0x7f3a3c0cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:25.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 shutdown_connections 2024-09-21T05:39:25.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.840+0000 7f3a43f58640 1 -- 172.21.15.22:0/3051813701 wait complete. 2024-09-21T05:39:25.856 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:39:25.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.978+0000 7f453bdb6640 1 -- 172.21.15.22:0/559061193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520004d40 msgr2=0x7f4520005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.978+0000 7f453bdb6640 1 --2- 172.21.15.22:0/559061193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4520004d40 0x7f4520005150 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45340ba0d0 tx=0x7f4520005830 comp rx=0 tx=0).stop 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.978+0000 7f453a353640 1 -- 172.21.15.22:0/559061193 reap_dead start 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.978+0000 7f453bdb6640 1 -- 172.21.15.22:0/559061193 shutdown_connections 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.978+0000 7f453bdb6640 1 -- 172.21.15.22:0/559061193 >> 172.21.15.22:0/559061193 conn(0x7f453408bfc0 msgr2=0x7f453409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.979+0000 7f453bdb6640 1 -- 172.21.15.22:0/559061193 shutdown_connections 2024-09-21T05:39:25.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.979+0000 7f453bdb6640 1 -- 172.21.15.22:0/559061193 wait complete. 2024-09-21T05:39:25.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.980+0000 7f453bdb6640 1 Processor -- start 2024-09-21T05:39:25.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.980+0000 7f453bdb6640 1 -- start start 2024-09-21T05:39:25.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4520004d40 0x7f45341f9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:25.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 0x7f453415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:25.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453409f580 0x7f453409f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45340cf0d0 con 0x7f45340a17f0 2024-09-21T05:39:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45340cef50 con 0x7f453409f580 2024-09-21T05:39:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45340cf250 con 0x7f4520004d40 2024-09-21T05:39:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 0x7f453415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 0x7f453415eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46480/0 (socket says 172.21.15.22:46480) 2024-09-21T05:39:25.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 -- 172.21.15.22:0/746149641 learned_addr learned my addr 172.21.15.22:0/746149641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453a353640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453409f580 0x7f453409f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539b52640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4520004d40 0x7f45341f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 -- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4520004d40 msgr2=0x7f45341f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4520004d40 0x7f45341f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 -- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453409f580 msgr2=0x7f453409f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453409f580 0x7f453409f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 -- 172.21.15.22:0/746149641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453409fed0 con 0x7f45340a17f0 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539b52640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4520004d40 0x7f45341f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453a353640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453409f580 0x7f453409f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f4539351640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 0x7f453415eb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4524006660 tx=0x7f4524003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524015070 con 0x7f45340a17f0 2024-09-21T05:39:25.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f452400f980 con 0x7f45340a17f0 2024-09-21T05:39:25.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.982+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453415f070 con 0x7f45340a17f0 2024-09-21T05:39:25.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.983+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452400fdd0 con 0x7f45340a17f0 2024-09-21T05:39:25.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.983+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45340a1e90 con 0x7f45340a17f0 2024-09-21T05:39:25.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.983+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45340d07c0 con 0x7f45340a17f0 2024-09-21T05:39:25.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.984+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4524011430 con 0x7f45340a17f0 2024-09-21T05:39:25.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.985+0000 7f452a7fc640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4518078bc0 0x7f451807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:25.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.985+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f452409bbd0 con 0x7f45340a17f0 2024-09-21T05:39:25.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.985+0000 7f4539b52640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4518078bc0 0x7f451807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:25.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.986+0000 7f4539b52640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4518078bc0 0x7f451807b0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4530009b40 tx=0x7f4530005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:25.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:25.989+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45340d07c0 con 0x7f45340a17f0 2024-09-21T05:39:26.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.212+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45340a1c20 con 0x7f45340a17f0 2024-09-21T05:39:26.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.213+0000 7f452a7fc640 1 -- 172.21.15.22:0/746149641 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f453409fed0 con 0x7f45340a17f0 2024-09-21T05:39:26.215 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:26.216 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:26.216 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:26.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4518078bc0 msgr2=0x7f451807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4518078bc0 0x7f451807b0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4530009b40 tx=0x7f4530005f90 comp rx=0 tx=0).stop 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 msgr2=0x7f453415eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 0x7f453415eb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4524006660 tx=0x7f4524003aa0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 shutdown_connections 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f453409f580 0x7f453409f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45340a17f0 0x7f453415eb30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4518078bc0 0x7f451807b0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 --2- 172.21.15.22:0/746149641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4520004d40 0x7f45341f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 >> 172.21.15.22:0/746149641 conn(0x7f453408bfc0 msgr2=0x7f4534161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 shutdown_connections 2024-09-21T05:39:26.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.215+0000 7f453bdb6640 1 -- 172.21.15.22:0/746149641 wait complete. 2024-09-21T05:39:26.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:26.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 -- 172.21.15.22:0/1635056246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 msgr2=0x7fdfbc070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 --2- 172.21.15.22:0/1635056246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc070180 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0005fd0 tx=0x7fdfb002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 -- 172.21.15.22:0/1635056246 shutdown_connections 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 --2- 172.21.15.22:0/1635056246 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc0706c0 0x7fdfbc072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 --2- 172.21.15.22:0/1635056246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc070180 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 -- 172.21.15.22:0/1635056246 >> 172.21.15.22:0/1635056246 conn(0x7fdfbc06d7d0 msgr2=0x7fdfbc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 -- 172.21.15.22:0/1635056246 shutdown_connections 2024-09-21T05:39:26.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.342+0000 7fdfc3550640 1 -- 172.21.15.22:0/1635056246 wait complete. 2024-09-21T05:39:26.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.344+0000 7fdfc3550640 1 Processor -- start 2024-09-21T05:39:26.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.344+0000 7fdfc3550640 1 -- start start 2024-09-21T05:39:26.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc3550640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc0706c0 0x7fdfbc09be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc3550640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc0af780 0x7fdfbc152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc3550640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc0ae110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc3550640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc073b70 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc3550640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0739f0 con 0x7fdfbc0af780 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc3550640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc073cf0 con 0x7fdfbc0706c0 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc1aed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc0ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc1aed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc0ae110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46498/0 (socket says 172.21.15.22:46498) 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc1aed640 1 -- 172.21.15.22:0/2645166590 learned_addr learned my addr 172.21.15.22:0/2645166590 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc0aeb640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc0af780 0x7fdfbc152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.345+0000 7fdfc12ec640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc0706c0 0x7fdfbc09be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc1aed640 1 -- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc0706c0 msgr2=0x7fdfbc09be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc1aed640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc0706c0 0x7fdfbc09be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc1aed640 1 -- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc0af780 msgr2=0x7fdfbc152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc1aed640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc0af780 0x7fdfbc152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc1aed640 1 -- 172.21.15.22:0/2645166590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc153330 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc1aed640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc0ae110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005d00 tx=0x7fdfb400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4017070 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfb4005330 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc09c810 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40047a0 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.346+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc09cdd0 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.347+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc072ff0 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.348+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfb4004940 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.348+0000 7fdfb9ffb640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078b90 0x7fdfa807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.348+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdfb409acd0 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.349+0000 7fdfc12ec640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078b90 0x7fdfa807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.349+0000 7fdfc12ec640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078b90 0x7fdfa807b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b450 tx=0x7fdfac00d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:26.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.353+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfbc153330 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.542+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfbc09de80 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.543+0000 7fdfb9ffb640 1 -- 172.21.15.22:0/2645166590 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7fdfb4062b50 con 0x7fdfbc0cd3e0 2024-09-21T05:39:26.545 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:26.545 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:26.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078b90 msgr2=0x7fdfa807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078b90 0x7fdfa807b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b450 tx=0x7fdfac00d040 comp rx=0 tx=0).stop 2024-09-21T05:39:26.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 msgr2=0x7fdfbc0ae110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc0ae110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4005d00 tx=0x7fdfb400dae0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 shutdown_connections 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfbc0cd3e0 0x7fdfbc0ae110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc0af780 0x7fdfbc152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078b90 0x7fdfa807b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 --2- 172.21.15.22:0/2645166590 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc0706c0 0x7fdfbc09be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 >> 172.21.15.22:0/2645166590 conn(0x7fdfbc06d7d0 msgr2=0x7fdfbc0ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 shutdown_connections 2024-09-21T05:39:26.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.545+0000 7fdfc3550640 1 -- 172.21.15.22:0/2645166590 wait complete. 2024-09-21T05:39:26.559 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:39:26.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:26.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 -- 172.21.15.22:0/2896220180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca8004800 msgr2=0x7fdca8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/2896220180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca8004800 0x7fdca8004c10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0d4500 tx=0x7fdca8005830 comp rx=0 tx=0).stop 2024-09-21T05:39:26.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 -- 172.21.15.22:0/2896220180 shutdown_connections 2024-09-21T05:39:26.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/2896220180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdca8004800 0x7fdca8004c10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 -- 172.21.15.22:0/2896220180 >> 172.21.15.22:0/2896220180 conn(0x7fdcbc09d840 msgr2=0x7fdcbc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:26.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 -- 172.21.15.22:0/2896220180 shutdown_connections 2024-09-21T05:39:26.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.673+0000 7fdcc3bba640 1 -- 172.21.15.22:0/2896220180 wait complete. 2024-09-21T05:39:26.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.675+0000 7fdcc3bba640 1 Processor -- start 2024-09-21T05:39:26.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.675+0000 7fdcc3bba640 1 -- start start 2024-09-21T05:39:26.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc3bba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdca8004800 0x7fdcbc0cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc3bba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcbc1545a0 0x7fdcbc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc3bba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 0x7fdcbc0b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc3bba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06fb20 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc3bba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc06f9a0 con 0x7fdcbc1545a0 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc3bba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc06e610 con 0x7fdca8004800 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc2157640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 0x7fdcbc0b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc2157640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 0x7fdcbc0b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46512/0 (socket says 172.21.15.22:46512) 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.676+0000 7fdcc2157640 1 -- 172.21.15.22:0/3700168442 learned_addr learned my addr 172.21.15.22:0/3700168442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc1956640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdca8004800 0x7fdcbc0cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc1155640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcbc1545a0 0x7fdcbc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc2157640 1 -- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdca8004800 msgr2=0x7fdcbc0cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc2157640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdca8004800 0x7fdcbc0cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc2157640 1 -- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcbc1545a0 msgr2=0x7fdcbc153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc2157640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcbc1545a0 0x7fdcbc153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc2157640 1 -- 172.21.15.22:0/3700168442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc0cc5d0 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc2157640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 0x7fdcbc0b42a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0ceb80 tx=0x7fdcb40019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4011870 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb4017070 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4011dc0 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc0b47e0 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.677+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc0ccda0 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.678+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc06ea50 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.679+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdcb401c460 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.679+0000 7fdcb27fc640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdca0078b90 0x7fdca007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:26.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.679+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdcb409b550 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.680+0000 7fdcc1956640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdca0078b90 0x7fdca007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:26.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.680+0000 7fdcc1956640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdca0078b90 0x7fdca007b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80047a0 tx=0x7fdcb8000ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:26.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.684+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdcb4063a20 con 0x7fdcbc0b3e90 2024-09-21T05:39:26.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.867+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdcbc0d3a00 con 0x7fdca0078b90 2024-09-21T05:39:26.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.872+0000 7fdcb27fc640 1 -- 172.21.15.22:0/3700168442 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249056 (secure 0 0 0) 0x7fdcbc0d3a00 con 0x7fdca0078b90 2024-09-21T05:39:26.877 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:26.878 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:26.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdca0078b90 msgr2=0x7fdca007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdca0078b90 0x7fdca007b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80047a0 tx=0x7fdcb8000ef0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 msgr2=0x7fdcbc0b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 0x7fdcbc0b42a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0ceb80 tx=0x7fdcb40019b0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 shutdown_connections 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcbc0b3e90 0x7fdcbc0b42a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdcbc1545a0 0x7fdcbc153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdca0078b90 0x7fdca007b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 --2- 172.21.15.22:0/3700168442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdca8004800 0x7fdcbc0cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 >> 172.21.15.22:0/3700168442 conn(0x7fdcbc09d840 msgr2=0x7fdcbc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 shutdown_connections 2024-09-21T05:39:26.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:26.875+0000 7fdcc3bba640 1 -- 172.21.15.22:0/3700168442 wait complete. 2024-09-21T05:39:26.888 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T05:39:25.737609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239784,"kb_used_data":2756,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734936,"statfs":{"total":773094113280,"available":772848574464,"internally_reserved":0,"allocated":2822144,"data_stored":1728563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.939454"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.803579+0000","last_change":"2024-09-21T05:39:20.803579+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.803579+0000","last_undegraded":"2024-09-21T05:39:20.803579+0000","last_fullsized":"2024-09-21T05:39:20.803579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-21T05:39:20.803231+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:20.803231+0000","last_peered":"2024-09-21T05:39:20.803231+0000","last_clean":"2024-09-21T05:39:20.803231+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:20.803231+0000","last_undegraded":"2024-09-21T05:39:20.803231+0000","last_fullsized":"2024-09-21T05:39:20.803231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801309+0000","last_change":"2024-09-21T05:39:20.801309+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801309+0000","last_undegraded":"2024-09-21T05:39:20.801309+0000","last_fullsized":"2024-09-21T05:39:20.801309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.800760+0000","last_change":"2024-09-21T05:39:20.800760+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.800760+0000","last_undegraded":"2024-09-21T05:39:20.800760+0000","last_fullsized":"2024-09-21T05:39:20.800760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.802369+0000","last_change":"2024-09-21T05:39:20.802369+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.802369+0000","last_undegraded":"2024-09-21T05:39:20.802369+0000","last_fullsized":"2024-09-21T05:39:20.802369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.800638+0000","last_change":"2024-09-21T05:39:20.800638+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.800638+0000","last_undegraded":"2024-09-21T05:39:20.800638+0000","last_fullsized":"2024-09-21T05:39:20.800638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801642+0000","last_change":"2024-09-21T05:39:20.801642+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801642+0000","last_undegraded":"2024-09-21T05:39:20.801642+0000","last_fullsized":"2024-09-21T05:39:20.801642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801345+0000","last_change":"2024-09-21T05:39:20.801345+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801345+0000","last_undegraded":"2024-09-21T05:39:20.801345+0000","last_fullsized":"2024-09-21T05:39:20.801345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":644547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":48362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:26.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 -- 172.21.15.22:0/3605947265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 msgr2=0x7f01580cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 --2- 172.21.15.22:0/3605947265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f01580cf850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0150005fd0 tx=0x7f015002a820 comp rx=0 tx=0).stop 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 -- 172.21.15.22:0/3605947265 shutdown_connections 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 --2- 172.21.15.22:0/3605947265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f01580cf850 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 --2- 172.21.15.22:0/3605947265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01580d16c0 0x7f01580cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 -- 172.21.15.22:0/3605947265 >> 172.21.15.22:0/3605947265 conn(0x7f015809d8f0 msgr2=0x7f01580d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 -- 172.21.15.22:0/3605947265 shutdown_connections 2024-09-21T05:39:27.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.043+0000 7f015fefa640 1 -- 172.21.15.22:0/3605947265 wait complete. 2024-09-21T05:39:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.045+0000 7f015fefa640 1 Processor -- start 2024-09-21T05:39:27.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.045+0000 7f015fefa640 1 -- start start 2024-09-21T05:39:27.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015fefa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f0158141ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015fefa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01580b9b60 0x7f01580b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015fefa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158157990 0x7f0158157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015fefa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01580d0120 con 0x7f01580b3a10 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015fefa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01580cffa0 con 0x7f0158157990 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015fefa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01580d02a0 con 0x7f01580b9b60 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015dc96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f0158141ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015dc96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f0158141ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46534/0 (socket says 172.21.15.22:46534) 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.046+0000 7f015dc96640 1 -- 172.21.15.22:0/2933203641 learned_addr learned my addr 172.21.15.22:0/2933203641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015e497640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158157990 0x7f0158157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015d495640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01580b9b60 0x7f01580b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015dc96640 1 -- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01580b9b60 msgr2=0x7f01580b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015dc96640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01580b9b60 0x7f01580b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015dc96640 1 -- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158157990 msgr2=0x7f0158157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015dc96640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158157990 0x7f0158157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015dc96640 1 -- 172.21.15.22:0/2933203641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0158142530 con 0x7f01580b3a10 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015e497640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158157990 0x7f0158157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015dc96640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f0158141ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f015400c3a0 tx=0x7f015400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154011790 con 0x7f01580b3a10 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0154017070 con 0x7f01580b3a10 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01581582e0 con 0x7f01580b3a10 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154011cc0 con 0x7f01580b3a10 2024-09-21T05:39:27.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.047+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01581f9390 con 0x7f01580b3a10 2024-09-21T05:39:27.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.048+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01580cf310 con 0x7f01580b3a10 2024-09-21T05:39:27.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.049+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f015401c460 con 0x7f01580b3a10 2024-09-21T05:39:27.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.050+0000 7f014e7fc640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f013c078b90 0x7f013c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.050+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f015409ac60 con 0x7f01580b3a10 2024-09-21T05:39:27.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.050+0000 7f015d495640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f013c078b90 0x7f013c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.050+0000 7f015d495640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f013c078b90 0x7f013c07b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0148005fd0 tx=0x7f0148005da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.054+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01581582e0 con 0x7f01580b3a10 2024-09-21T05:39:27.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.238+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01580ba0d0 con 0x7f013c078b90 2024-09-21T05:39:27.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.243+0000 7f014e7fc640 1 -- 172.21.15.22:0/2933203641 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249056 (secure 0 0 0) 0x7f01580ba0d0 con 0x7f013c078b90 2024-09-21T05:39:27.247 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:27.249 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:27.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f013c078b90 msgr2=0x7f013c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f013c078b90 0x7f013c07b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0148005fd0 tx=0x7f0148005da0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 msgr2=0x7f0158141ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f0158141ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f015400c3a0 tx=0x7f015400c860 comp rx=0 tx=0).stop 2024-09-21T05:39:27.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 shutdown_connections 2024-09-21T05:39:27.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0158157990 0x7f0158157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f013c078b90 0x7f013c07b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01580b9b60 0x7f01580b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 --2- 172.21.15.22:0/2933203641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01580b3a10 0x7f0158141ff0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 >> 172.21.15.22:0/2933203641 conn(0x7f015809d8f0 msgr2=0x7f01580b8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:27.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 shutdown_connections 2024-09-21T05:39:27.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.246+0000 7f015fefa640 1 -- 172.21.15.22:0/2933203641 wait complete. 2024-09-21T05:39:27.259 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T05:39:25.737609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239784,"kb_used_data":2756,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734936,"statfs":{"total":773094113280,"available":772848574464,"internally_reserved":0,"allocated":2822144,"data_stored":1728563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.939454"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.803579+0000","last_change":"2024-09-21T05:39:20.803579+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.803579+0000","last_undegraded":"2024-09-21T05:39:20.803579+0000","last_fullsized":"2024-09-21T05:39:20.803579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-21T05:39:20.803231+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:20.803231+0000","last_peered":"2024-09-21T05:39:20.803231+0000","last_clean":"2024-09-21T05:39:20.803231+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:20.803231+0000","last_undegraded":"2024-09-21T05:39:20.803231+0000","last_fullsized":"2024-09-21T05:39:20.803231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801309+0000","last_change":"2024-09-21T05:39:20.801309+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801309+0000","last_undegraded":"2024-09-21T05:39:20.801309+0000","last_fullsized":"2024-09-21T05:39:20.801309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.800760+0000","last_change":"2024-09-21T05:39:20.800760+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.800760+0000","last_undegraded":"2024-09-21T05:39:20.800760+0000","last_fullsized":"2024-09-21T05:39:20.800760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.802369+0000","last_change":"2024-09-21T05:39:20.802369+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.802369+0000","last_undegraded":"2024-09-21T05:39:20.802369+0000","last_fullsized":"2024-09-21T05:39:20.802369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.800638+0000","last_change":"2024-09-21T05:39:20.800638+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.800638+0000","last_undegraded":"2024-09-21T05:39:20.800638+0000","last_fullsized":"2024-09-21T05:39:20.800638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801642+0000","last_change":"2024-09-21T05:39:20.801642+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801642+0000","last_undegraded":"2024-09-21T05:39:20.801642+0000","last_fullsized":"2024-09-21T05:39:20.801642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801345+0000","last_change":"2024-09-21T05:39:20.801345+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801345+0000","last_undegraded":"2024-09-21T05:39:20.801345+0000","last_fullsized":"2024-09-21T05:39:20.801345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":644547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":48362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:27.263 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f55846f9640 1 -- 172.21.15.22:0/1295823748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5564003a00 msgr2=0x7f5564003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f55846f9640 1 --2- 172.21.15.22:0/1295823748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5564003a00 0x7f5564003e70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f557400c640 tx=0x7f5564005830 comp rx=0 tx=0).stop 2024-09-21T05:39:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f5582c96640 1 -- 172.21.15.22:0/1295823748 reap_dead start 2024-09-21T05:39:27.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f55846f9640 1 -- 172.21.15.22:0/1295823748 shutdown_connections 2024-09-21T05:39:27.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f55846f9640 1 -- 172.21.15.22:0/1295823748 >> 172.21.15.22:0/1295823748 conn(0x7f557c0cee20 msgr2=0x7f557c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:27.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f55846f9640 1 -- 172.21.15.22:0/1295823748 shutdown_connections 2024-09-21T05:39:27.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.415+0000 7f55846f9640 1 -- 172.21.15.22:0/1295823748 wait complete. 2024-09-21T05:39:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.417+0000 7f55846f9640 1 Processor -- start 2024-09-21T05:39:27.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.417+0000 7f55846f9640 1 -- start start 2024-09-21T05:39:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f55846f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5564003a00 0x7f557c0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f55846f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 0x7f557c158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f55846f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f557c1e8080 0x7f557c1e8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f55846f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f557c15ca40 con 0x7f557c0b51d0 2024-09-21T05:39:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f55846f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f557c0b47d0 con 0x7f557c1e8080 2024-09-21T05:39:27.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f55846f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f557c15cbc0 con 0x7f5564003a00 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 0x7f557c158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 0x7f557c158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46554/0 (socket says 172.21.15.22:46554) 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 -- 172.21.15.22:0/748826654 learned_addr learned my addr 172.21.15.22:0/748826654 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5582c96640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f557c1e8080 0x7f557c1e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5582495640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5564003a00 0x7f557c0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 -- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5564003a00 msgr2=0x7f557c0b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5564003a00 0x7f557c0b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 -- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f557c1e8080 msgr2=0x7f557c1e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f557c1e8080 0x7f557c1e8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.418+0000 7f5581c94640 1 -- 172.21.15.22:0/748826654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1e89d0 con 0x7f557c0b51d0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.419+0000 7f5581c94640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 0x7f557c158260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5570005f60 tx=0x7f5570011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.419+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557001f070 con 0x7f557c0b51d0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.419+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f557000fb50 con 0x7f557c0b51d0 2024-09-21T05:39:27.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.419+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570012450 con 0x7f557c0b51d0 2024-09-21T05:39:27.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.419+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1f6ea0 con 0x7f557c0b51d0 2024-09-21T05:39:27.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.419+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c1f7460 con 0x7f557c0b51d0 2024-09-21T05:39:27.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.420+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c1f6c10 con 0x7f557c0b51d0 2024-09-21T05:39:27.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.420+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5570022070 con 0x7f557c0b51d0 2024-09-21T05:39:27.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.421+0000 7f557affd640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5568078b90 0x7f556807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.421+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f557009b5b0 con 0x7f557c0b51d0 2024-09-21T05:39:27.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.421+0000 7f5582495640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5568078b90 0x7f556807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.422+0000 7f5582495640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5568078b90 0x7f556807b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5574033000 tx=0x7f557c0c93e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:27.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.426+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f557c1e89d0 con 0x7f557c0b51d0 2024-09-21T05:39:27.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.610+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f557c0d0bf0 con 0x7f5568078b90 2024-09-21T05:39:27.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.614+0000 7f557affd640 1 -- 172.21.15.22:0/748826654 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249056 (secure 0 0 0) 0x7f557c0d0bf0 con 0x7f5568078b90 2024-09-21T05:39:27.619 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:27.620 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:27.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5568078b90 msgr2=0x7f556807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5568078b90 0x7f556807b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5574033000 tx=0x7f557c0c93e0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 msgr2=0x7f557c158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 0x7f557c158260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5570005f60 tx=0x7f5570011bf0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 shutdown_connections 2024-09-21T05:39:27.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f557c1e8080 0x7f557c1e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f557c0b51d0 0x7f557c158260 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5568078b90 0x7f556807b0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 --2- 172.21.15.22:0/748826654 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5564003a00 0x7f557c0b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.617+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 >> 172.21.15.22:0/748826654 conn(0x7f557c0cee20 msgr2=0x7f557c073080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:27.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.618+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 shutdown_connections 2024-09-21T05:39:27.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.618+0000 7f55846f9640 1 -- 172.21.15.22:0/748826654 wait complete. 2024-09-21T05:39:27.630 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-21T05:39:25.737609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592756,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":138,"num_write_kb":1641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":162,"acting":162,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239784,"kb_used_data":2756,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734936,"statfs":{"total":773094113280,"available":772848574464,"internally_reserved":0,"allocated":2822144,"data_stored":1728563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.939454"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.803579+0000","last_change":"2024-09-21T05:39:20.803579+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.803579+0000","last_undegraded":"2024-09-21T05:39:20.803579+0000","last_fullsized":"2024-09-21T05:39:20.803579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":105,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-21T05:39:20.803231+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:20.803231+0000","last_peered":"2024-09-21T05:39:20.803231+0000","last_clean":"2024-09-21T05:39:20.803231+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:20.803231+0000","last_undegraded":"2024-09-21T05:39:20.803231+0000","last_fullsized":"2024-09-21T05:39:20.803231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801309+0000","last_change":"2024-09-21T05:39:20.801309+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801309+0000","last_undegraded":"2024-09-21T05:39:20.801309+0000","last_fullsized":"2024-09-21T05:39:20.801309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.800760+0000","last_change":"2024-09-21T05:39:20.800760+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.800760+0000","last_undegraded":"2024-09-21T05:39:20.800760+0000","last_fullsized":"2024-09-21T05:39:20.800760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.802369+0000","last_change":"2024-09-21T05:39:20.802369+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.802369+0000","last_undegraded":"2024-09-21T05:39:20.802369+0000","last_fullsized":"2024-09-21T05:39:20.802369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:39:21.797734+0000","last_change":"2024-09-21T05:39:21.797734+0000","last_active":"2024-09-21T05:39:21.797734+0000","last_peered":"2024-09-21T05:39:21.797734+0000","last_clean":"2024-09-21T05:39:21.797734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:21.797734+0000","last_undegraded":"2024-09-21T05:39:21.797734+0000","last_fullsized":"2024-09-21T05:39:21.797734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.800638+0000","last_change":"2024-09-21T05:39:20.800638+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.800638+0000","last_undegraded":"2024-09-21T05:39:20.800638+0000","last_fullsized":"2024-09-21T05:39:20.800638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801642+0000","last_change":"2024-09-21T05:39:20.801642+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801642+0000","last_undegraded":"2024-09-21T05:39:20.801642+0000","last_fullsized":"2024-09-21T05:39:20.801642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-09-21T05:39:20.801345+0000","last_change":"2024-09-21T05:39:20.801345+0000","last_active":"2024-09-21T05:39:20.792192+0000","last_peered":"2024-09-21T05:39:20.792192+0000","last_clean":"2024-09-21T05:39:20.792192+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:39:20.801345+0000","last_undegraded":"2024-09-21T05:39:20.801345+0000","last_fullsized":"2024-09-21T05:39:20.801345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":6},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2388,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27776,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344064,"statfs":{"total":96636764160,"available":96608321536,"internally_reserved":0,"allocated":765952,"data_stored":644547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":159744,"data_stored":48362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:27.634 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-21T05:39:27.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.634 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-21T05:39:27.634 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.635 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.803579+0000', 'last_change': '2024-09-21T05:39:20.803579+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.803579+0000', 'last_undegraded': '2024-09-21T05:39:20.803579+0000', 'last_fullsized': '2024-09-21T05:39:20.803579+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.636 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.801309+0000', 'last_change': '2024-09-21T05:39:20.801309+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.801309+0000', 'last_undegraded': '2024-09-21T05:39:20.801309+0000', 'last_fullsized': '2024-09-21T05:39:20.801309+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.800760+0000', 'last_change': '2024-09-21T05:39:20.800760+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.800760+0000', 'last_undegraded': '2024-09-21T05:39:20.800760+0000', 'last_fullsized': '2024-09-21T05:39:20.800760+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-21T05:39:27.637 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.802369+0000', 'last_change': '2024-09-21T05:39:20.802369+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.802369+0000', 'last_undegraded': '2024-09-21T05:39:20.802369+0000', 'last_fullsized': '2024-09-21T05:39:20.802369+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-21T05:39:21.797734+0000', 'last_change': '2024-09-21T05:39:21.797734+0000', 'last_active': '2024-09-21T05:39:21.797734+0000', 'last_peered': '2024-09-21T05:39:21.797734+0000', 'last_clean': '2024-09-21T05:39:21.797734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:21.797734+0000', 'last_undegraded': '2024-09-21T05:39:21.797734+0000', 'last_fullsized': '2024-09-21T05:39:21.797734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:21.797734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.800638+0000', 'last_change': '2024-09-21T05:39:20.800638+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.800638+0000', 'last_undegraded': '2024-09-21T05:39:20.800638+0000', 'last_fullsized': '2024-09-21T05:39:20.800638+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.801642+0000', 'last_change': '2024-09-21T05:39:20.801642+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.801642+0000', 'last_undegraded': '2024-09-21T05:39:20.801642+0000', 'last_fullsized': '2024-09-21T05:39:20.801642+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-21T05:39:27.638 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 18, 'state': 'creating+peering', 'last_fresh': '2024-09-21T05:39:20.801345+0000', 'last_change': '2024-09-21T05:39:20.801345+0000', 'last_active': '2024-09-21T05:39:20.792192+0000', 'last_peered': '2024-09-21T05:39:20.792192+0000', 'last_clean': '2024-09-21T05:39:20.792192+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T05:39:20.801345+0000', 'last_undegraded': '2024-09-21T05:39:20.801345+0000', 'last_fullsized': '2024-09-21T05:39:20.801345+0000', 'mapping_epoch': 18, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 18, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'last_clean_scrub_stamp': '2024-09-21T05:39:20.792192+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T05:39:27.639 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.784+0000 7efcbd869640 1 -- 172.21.15.22:0/3981148509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80cf440 msgr2=0x7efcb80d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.784+0000 7efcbd869640 1 --2- 172.21.15.22:0/3981148509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80cf440 0x7efcb80d0190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efcb0005fd0 tx=0x7efcb002aa10 comp rx=0 tx=0).stop 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.784+0000 7efcbd869640 1 -- 172.21.15.22:0/3981148509 shutdown_connections 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.784+0000 7efcbd869640 1 --2- 172.21.15.22:0/3981148509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80cf440 0x7efcb80d0190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.784+0000 7efcbd869640 1 --2- 172.21.15.22:0/3981148509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcb80cd360 0x7efcb815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.784+0000 7efcbd869640 1 -- 172.21.15.22:0/3981148509 >> 172.21.15.22:0/3981148509 conn(0x7efcb80710b0 msgr2=0x7efcb80714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.785+0000 7efcbd869640 1 -- 172.21.15.22:0/3981148509 shutdown_connections 2024-09-21T05:39:27.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.785+0000 7efcbd869640 1 -- 172.21.15.22:0/3981148509 wait complete. 2024-09-21T05:39:27.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.786+0000 7efcbd869640 1 Processor -- start 2024-09-21T05:39:27.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.786+0000 7efcbd869640 1 -- start start 2024-09-21T05:39:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 0x7efcb806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcb80cd360 0x7efcb8150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb815f820 0x7efcb80c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80d0b70 con 0x7efcb80b7fe0 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efcb80d09f0 con 0x7efcb815f820 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efcb80d0cf0 con 0x7efcb80cd360 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 0x7efcb806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 0x7efcb806e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46578/0 (socket says 172.21.15.22:46578) 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 -- 172.21.15.22:0/1679240468 learned_addr learned my addr 172.21.15.22:0/1679240468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 -- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcb80cd360 msgr2=0x7efcb8150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcb80cd360 0x7efcb8150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 -- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb815f820 msgr2=0x7efcb80c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb815f820 0x7efcb80c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 -- 172.21.15.22:0/1679240468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb815cfe0 con 0x7efcb80b7fe0 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb6ffd640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 0x7efcb806e2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efca800e7c0 tx=0x7efca800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:27.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca800ccc0 con 0x7efcb80b7fe0 2024-09-21T05:39:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efca8010430 con 0x7efcb80b7fe0 2024-09-21T05:39:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca8010860 con 0x7efcb80b7fe0 2024-09-21T05:39:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb80c6f20 con 0x7efcb80b7fe0 2024-09-21T05:39:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.788+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb80c74e0 con 0x7efcb80b7fe0 2024-09-21T05:39:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.789+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb80c5bd0 con 0x7efcb80b7fe0 2024-09-21T05:39:27.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.790+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efca8010a00 con 0x7efcb80b7fe0 2024-09-21T05:39:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.791+0000 7efcb4ff9640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc98078bc0 0x7efc9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.791+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efca809b790 con 0x7efcb80b7fe0 2024-09-21T05:39:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.791+0000 7efcaffff640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc98078bc0 0x7efc9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:27.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.791+0000 7efcaffff640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc98078bc0 0x7efc9807b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efca0000f60 tx=0x7efca00079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:27.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:27.795+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efcb80c5bd0 con 0x7efcb80b7fe0 2024-09-21T05:39:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.055+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7efcb80d1210 con 0x7efcb80b7fe0 2024-09-21T05:39:28.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.055+0000 7efcb4ff9640 1 -- 172.21.15.22:0/1679240468 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1053 (secure 0 0 0) 0x7efca8063b40 con 0x7efcb80b7fe0 2024-09-21T05:39:28.058 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:28.058 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1726897150,"num_in_osds":8,"osd_in_since":1726897140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":245653504,"bytes_avail":772848459776,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4395}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-21T05:38:57.362856+0000","services":{}},"progress_events":{}} 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc98078bc0 msgr2=0x7efc9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc98078bc0 0x7efc9807b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efca0000f60 tx=0x7efca00079b0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 msgr2=0x7efcb806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 0x7efcb806e2c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efca800e7c0 tx=0x7efca800ec80 comp rx=0 tx=0).stop 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 shutdown_connections 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efcb815f820 0x7efcb80c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc98078bc0 0x7efc9807b0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efcb80cd360 0x7efcb8150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 --2- 172.21.15.22:0/1679240468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efcb80b7fe0 0x7efcb806e2c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 >> 172.21.15.22:0/1679240468 conn(0x7efcb80710b0 msgr2=0x7efcb815ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 shutdown_connections 2024-09-21T05:39:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.057+0000 7efcbd869640 1 -- 172.21.15.22:0/1679240468 wait complete. 2024-09-21T05:39:28.070 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T05:39:28.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:28.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 -- 172.21.15.22:0/4150278407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240b51c0 msgr2=0x7ff42415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 --2- 172.21.15.22:0/4150278407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240b51c0 0x7ff42415c6e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff420005fd0 tx=0x7ff42002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 -- 172.21.15.22:0/4150278407 shutdown_connections 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 --2- 172.21.15.22:0/4150278407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240ccbd0 0x7ff4240d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 --2- 172.21.15.22:0/4150278407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240b51c0 0x7ff42415c6e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 -- 172.21.15.22:0/4150278407 >> 172.21.15.22:0/4150278407 conn(0x7ff424072ae0 msgr2=0x7ff424072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 -- 172.21.15.22:0/4150278407 shutdown_connections 2024-09-21T05:39:28.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.184+0000 7ff42be8b640 1 -- 172.21.15.22:0/4150278407 wait complete. 2024-09-21T05:39:28.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.186+0000 7ff42be8b640 1 Processor -- start 2024-09-21T05:39:28.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.186+0000 7ff42be8b640 1 -- start start 2024-09-21T05:39:28.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42be8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4240b51c0 0x7ff4240cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42be8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 0x7ff4240ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42be8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240d12c0 0x7ff4240aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42be8b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff424160a60 con 0x7ff4240ccbd0 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42be8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4240d1a20 con 0x7ff4240b51c0 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42be8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff424160be0 con 0x7ff4240d12c0 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff429426640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 0x7ff4240ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff429426640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 0x7ff4240ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46598/0 (socket says 172.21.15.22:46598) 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff429426640 1 -- 172.21.15.22:0/3502732645 learned_addr learned my addr 172.21.15.22:0/3502732645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff429c27640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4240b51c0 0x7ff4240cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.187+0000 7ff42a428640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240d12c0 0x7ff4240aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:28.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429426640 1 -- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240d12c0 msgr2=0x7ff4240aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429426640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240d12c0 0x7ff4240aea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429426640 1 -- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4240b51c0 msgr2=0x7ff4240cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429426640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4240b51c0 0x7ff4240cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429426640 1 -- 172.21.15.22:0/3502732645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4240b16d0 con 0x7ff4240ccbd0 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff42a428640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240d12c0 0x7ff4240aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429c27640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4240b51c0 0x7ff4240cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff429426640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 0x7ff4240ae530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d700 tx=0x7ff41400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414017070 con 0x7ff4240ccbd0 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff414009b10 con 0x7ff4240ccbd0 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414010460 con 0x7ff4240ccbd0 2024-09-21T05:39:28.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42414aa00 con 0x7ff4240ccbd0 2024-09-21T05:39:28.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.188+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42414afc0 con 0x7ff4240ccbd0 2024-09-21T05:39:28.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.189+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4240b1440 con 0x7ff4240ccbd0 2024-09-21T05:39:28.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.190+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff41401a070 con 0x7ff4240ccbd0 2024-09-21T05:39:28.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.190+0000 7ff41a7fc640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff408078bc0 0x7ff40807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:28.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.190+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff41409ae60 con 0x7ff4240ccbd0 2024-09-21T05:39:28.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.191+0000 7ff429c27640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff408078bc0 0x7ff40807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:28.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.191+0000 7ff429c27640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff408078bc0 0x7ff40807b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff420034000 tx=0x7ff420004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:28.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.195+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff414063290 con 0x7ff4240ccbd0 2024-09-21T05:39:28.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.378+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff4240b19b0 con 0x7ff408078bc0 2024-09-21T05:39:28.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.383+0000 7ff41a7fc640 1 -- 172.21.15.22:0/3502732645 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7ff4240b19b0 con 0x7ff408078bc0 2024-09-21T05:39:28.387 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:28.389 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:28.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff408078bc0 msgr2=0x7ff40807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff408078bc0 0x7ff40807b0d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff420034000 tx=0x7ff420004ab0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 msgr2=0x7ff4240ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:28.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 0x7ff4240ae530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d700 tx=0x7ff41400dbc0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 shutdown_connections 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff408078bc0 0x7ff40807b0d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4240d12c0 0x7ff4240aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4240ccbd0 0x7ff4240ae530 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 --2- 172.21.15.22:0/3502732645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4240b51c0 0x7ff4240cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.386+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 >> 172.21.15.22:0/3502732645 conn(0x7ff424072ae0 msgr2=0x7ff4240cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.387+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 shutdown_connections 2024-09-21T05:39:28.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:28.387+0000 7ff42be8b640 1 -- 172.21.15.22:0/3502732645 wait complete. 2024-09-21T05:39:28.400 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-21T05:39:27.738198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239896,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734824,"statfs":{"total":773094113280,"available":772848459776,"internally_reserved":0,"allocated":2924544,"data_stored":1773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001669"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226309+0000","last_change":"2024-09-21T05:39:22.834732+0000","last_active":"2024-09-21T05:39:24.226309+0000","last_peered":"2024-09-21T05:39:24.226309+0000","last_clean":"2024-09-21T05:39:24.226309+0000","last_became_active":"2024-09-21T05:39:22.833098+0000","last_became_peered":"2024-09-21T05:39:22.833098+0000","last_unstale":"2024-09-21T05:39:24.226309+0000","last_undegraded":"2024-09-21T05:39:24.226309+0000","last_fullsized":"2024-09-21T05:39:24.226309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.341720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225995+0000","last_change":"2024-09-21T05:39:22.838816+0000","last_active":"2024-09-21T05:39:24.225995+0000","last_peered":"2024-09-21T05:39:24.225995+0000","last_clean":"2024-09-21T05:39:24.225995+0000","last_became_active":"2024-09-21T05:39:22.838511+0000","last_became_peered":"2024-09-21T05:39:22.838511+0000","last_unstale":"2024-09-21T05:39:24.225995+0000","last_undegraded":"2024-09-21T05:39:24.225995+0000","last_fullsized":"2024-09-21T05:39:24.225995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:27.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225652+0000","last_change":"2024-09-21T05:39:22.843633+0000","last_active":"2024-09-21T05:39:24.225652+0000","last_peered":"2024-09-21T05:39:24.225652+0000","last_clean":"2024-09-21T05:39:24.225652+0000","last_became_active":"2024-09-21T05:39:22.843200+0000","last_became_peered":"2024-09-21T05:39:22.843200+0000","last_unstale":"2024-09-21T05:39:24.225652+0000","last_undegraded":"2024-09-21T05:39:24.225652+0000","last_fullsized":"2024-09-21T05:39:24.225652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:30.085035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226390+0000","last_change":"2024-09-21T05:39:22.836751+0000","last_active":"2024-09-21T05:39:24.226390+0000","last_peered":"2024-09-21T05:39:24.226390+0000","last_clean":"2024-09-21T05:39:24.226390+0000","last_became_active":"2024-09-21T05:39:22.833532+0000","last_became_peered":"2024-09-21T05:39:22.833532+0000","last_unstale":"2024-09-21T05:39:24.226390+0000","last_undegraded":"2024-09-21T05:39:24.226390+0000","last_fullsized":"2024-09-21T05:39:24.226390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:07.246445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831080+0000","last_change":"2024-09-21T05:39:22.821659+0000","last_active":"2024-09-21T05:39:23.831080+0000","last_peered":"2024-09-21T05:39:23.831080+0000","last_clean":"2024-09-21T05:39:23.831080+0000","last_became_active":"2024-09-21T05:39:22.818471+0000","last_became_peered":"2024-09-21T05:39:22.818471+0000","last_unstale":"2024-09-21T05:39:23.831080+0000","last_undegraded":"2024-09-21T05:39:23.831080+0000","last_fullsized":"2024-09-21T05:39:23.831080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:11.829615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226338+0000","last_change":"2024-09-21T05:39:22.837584+0000","last_active":"2024-09-21T05:39:24.226338+0000","last_peered":"2024-09-21T05:39:24.226338+0000","last_clean":"2024-09-21T05:39:24.226338+0000","last_became_active":"2024-09-21T05:39:22.837243+0000","last_became_peered":"2024-09-21T05:39:22.837243+0000","last_unstale":"2024-09-21T05:39:24.226338+0000","last_undegraded":"2024-09-21T05:39:24.226338+0000","last_fullsized":"2024-09-21T05:39:24.226338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:46.224414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830586+0000","last_change":"2024-09-21T05:39:22.822050+0000","last_active":"2024-09-21T05:39:23.830586+0000","last_peered":"2024-09-21T05:39:23.830586+0000","last_clean":"2024-09-21T05:39:23.830586+0000","last_became_active":"2024-09-21T05:39:22.821705+0000","last_became_peered":"2024-09-21T05:39:22.821705+0000","last_unstale":"2024-09-21T05:39:23.830586+0000","last_undegraded":"2024-09-21T05:39:23.830586+0000","last_fullsized":"2024-09-21T05:39:23.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:42.204486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831067+0000","last_change":"2024-09-21T05:39:22.820929+0000","last_active":"2024-09-21T05:39:23.831067+0000","last_peered":"2024-09-21T05:39:23.831067+0000","last_clean":"2024-09-21T05:39:23.831067+0000","last_became_active":"2024-09-21T05:39:22.820587+0000","last_became_peered":"2024-09-21T05:39:22.820587+0000","last_unstale":"2024-09-21T05:39:23.831067+0000","last_undegraded":"2024-09-21T05:39:23.831067+0000","last_fullsized":"2024-09-21T05:39:23.831067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:42.698282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226051+0000","last_change":"2024-09-21T05:39:22.838353+0000","last_active":"2024-09-21T05:39:24.226051+0000","last_peered":"2024-09-21T05:39:24.226051+0000","last_clean":"2024-09-21T05:39:24.226051+0000","last_became_active":"2024-09-21T05:39:22.838044+0000","last_became_peered":"2024-09-21T05:39:22.838044+0000","last_unstale":"2024-09-21T05:39:24.226051+0000","last_undegraded":"2024-09-21T05:39:24.226051+0000","last_fullsized":"2024-09-21T05:39:24.226051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:27.463085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225891+0000","last_change":"2024-09-21T05:39:22.833336+0000","last_active":"2024-09-21T05:39:24.225891+0000","last_peered":"2024-09-21T05:39:24.225891+0000","last_clean":"2024-09-21T05:39:24.225891+0000","last_became_active":"2024-09-21T05:39:22.832959+0000","last_became_peered":"2024-09-21T05:39:22.832959+0000","last_unstale":"2024-09-21T05:39:24.225891+0000","last_undegraded":"2024-09-21T05:39:24.225891+0000","last_fullsized":"2024-09-21T05:39:24.225891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.450059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225617+0000","last_change":"2024-09-21T05:39:22.834980+0000","last_active":"2024-09-21T05:39:24.225617+0000","last_peered":"2024-09-21T05:39:24.225617+0000","last_clean":"2024-09-21T05:39:24.225617+0000","last_became_active":"2024-09-21T05:39:22.834075+0000","last_became_peered":"2024-09-21T05:39:22.834075+0000","last_unstale":"2024-09-21T05:39:24.225617+0000","last_undegraded":"2024-09-21T05:39:24.225617+0000","last_fullsized":"2024-09-21T05:39:24.225617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:31.814988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830803+0000","last_change":"2024-09-21T05:39:21.824655+0000","last_active":"2024-09-21T05:39:23.830803+0000","last_peered":"2024-09-21T05:39:23.830803+0000","last_clean":"2024-09-21T05:39:23.830803+0000","last_became_active":"2024-09-21T05:39:21.824309+0000","last_became_peered":"2024-09-21T05:39:21.824309+0000","last_unstale":"2024-09-21T05:39:23.830803+0000","last_undegraded":"2024-09-21T05:39:23.830803+0000","last_fullsized":"2024-09-21T05:39:23.830803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:50.559527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226272+0000","last_change":"2024-09-21T05:39:22.830496+0000","last_active":"2024-09-21T05:39:24.226272+0000","last_peered":"2024-09-21T05:39:24.226272+0000","last_clean":"2024-09-21T05:39:24.226272+0000","last_became_active":"2024-09-21T05:39:22.829859+0000","last_became_peered":"2024-09-21T05:39:22.829859+0000","last_unstale":"2024-09-21T05:39:24.226272+0000","last_undegraded":"2024-09-21T05:39:24.226272+0000","last_fullsized":"2024-09-21T05:39:24.226272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:09.763411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226478+0000","last_change":"2024-09-21T05:39:22.837101+0000","last_active":"2024-09-21T05:39:24.226478+0000","last_peered":"2024-09-21T05:39:24.226478+0000","last_clean":"2024-09-21T05:39:24.226478+0000","last_became_active":"2024-09-21T05:39:22.834638+0000","last_became_peered":"2024-09-21T05:39:22.834638+0000","last_unstale":"2024-09-21T05:39:24.226478+0000","last_undegraded":"2024-09-21T05:39:24.226478+0000","last_fullsized":"2024-09-21T05:39:24.226478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.717004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225997+0000","last_change":"2024-09-21T05:39:22.828178+0000","last_active":"2024-09-21T05:39:24.225997+0000","last_peered":"2024-09-21T05:39:24.225997+0000","last_clean":"2024-09-21T05:39:24.225997+0000","last_became_active":"2024-09-21T05:39:22.827833+0000","last_became_peered":"2024-09-21T05:39:22.827833+0000","last_unstale":"2024-09-21T05:39:24.225997+0000","last_undegraded":"2024-09-21T05:39:24.225997+0000","last_fullsized":"2024-09-21T05:39:24.225997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:58.078709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225514+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:24.225514+0000","last_peered":"2024-09-21T05:39:24.225514+0000","last_clean":"2024-09-21T05:39:24.225514+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:24.225514+0000","last_undegraded":"2024-09-21T05:39:24.225514+0000","last_fullsized":"2024-09-21T05:39:24.225514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225383+0000","last_change":"2024-09-21T05:39:21.833248+0000","last_active":"2024-09-21T05:39:24.225383+0000","last_peered":"2024-09-21T05:39:24.225383+0000","last_clean":"2024-09-21T05:39:24.225383+0000","last_became_active":"2024-09-21T05:39:21.832873+0000","last_became_peered":"2024-09-21T05:39:21.832873+0000","last_unstale":"2024-09-21T05:39:24.225383+0000","last_undegraded":"2024-09-21T05:39:24.225383+0000","last_fullsized":"2024-09-21T05:39:24.225383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:29.420862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830594+0000","last_change":"2024-09-21T05:39:22.831920+0000","last_active":"2024-09-21T05:39:23.830594+0000","last_peered":"2024-09-21T05:39:23.830594+0000","last_clean":"2024-09-21T05:39:23.830594+0000","last_became_active":"2024-09-21T05:39:22.831421+0000","last_became_peered":"2024-09-21T05:39:22.831421+0000","last_unstale":"2024-09-21T05:39:23.830594+0000","last_undegraded":"2024-09-21T05:39:23.830594+0000","last_fullsized":"2024-09-21T05:39:23.830594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:56.775839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226188+0000","last_change":"2024-09-21T05:39:21.831044+0000","last_active":"2024-09-21T05:39:24.226188+0000","last_peered":"2024-09-21T05:39:24.226188+0000","last_clean":"2024-09-21T05:39:24.226188+0000","last_became_active":"2024-09-21T05:39:21.830412+0000","last_became_peered":"2024-09-21T05:39:21.830412+0000","last_unstale":"2024-09-21T05:39:24.226188+0000","last_undegraded":"2024-09-21T05:39:24.226188+0000","last_fullsized":"2024-09-21T05:39:24.226188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:36.640541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225380+0000","last_change":"2024-09-21T05:39:21.833615+0000","last_active":"2024-09-21T05:39:24.225380+0000","last_peered":"2024-09-21T05:39:24.225380+0000","last_clean":"2024-09-21T05:39:24.225380+0000","last_became_active":"2024-09-21T05:39:21.833028+0000","last_became_peered":"2024-09-21T05:39:21.833028+0000","last_unstale":"2024-09-21T05:39:24.225380+0000","last_undegraded":"2024-09-21T05:39:24.225380+0000","last_fullsized":"2024-09-21T05:39:24.225380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:28.358924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829777+0000","last_change":"2024-09-21T05:39:22.827047+0000","last_active":"2024-09-21T05:39:23.829777+0000","last_peered":"2024-09-21T05:39:23.829777+0000","last_clean":"2024-09-21T05:39:23.829777+0000","last_became_active":"2024-09-21T05:39:22.825920+0000","last_became_peered":"2024-09-21T05:39:22.825920+0000","last_unstale":"2024-09-21T05:39:23.829777+0000","last_undegraded":"2024-09-21T05:39:23.829777+0000","last_fullsized":"2024-09-21T05:39:23.829777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:56.401187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226082+0000","last_change":"2024-09-21T05:39:21.829961+0000","last_active":"2024-09-21T05:39:24.226082+0000","last_peered":"2024-09-21T05:39:24.226082+0000","last_clean":"2024-09-21T05:39:24.226082+0000","last_became_active":"2024-09-21T05:39:21.829484+0000","last_became_peered":"2024-09-21T05:39:21.829484+0000","last_unstale":"2024-09-21T05:39:24.226082+0000","last_undegraded":"2024-09-21T05:39:24.226082+0000","last_fullsized":"2024-09-21T05:39:24.226082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:16.446491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225816+0000","last_change":"2024-09-21T05:39:21.830335+0000","last_active":"2024-09-21T05:39:24.225816+0000","last_peered":"2024-09-21T05:39:24.225816+0000","last_clean":"2024-09-21T05:39:24.225816+0000","last_became_active":"2024-09-21T05:39:21.829550+0000","last_became_peered":"2024-09-21T05:39:21.829550+0000","last_unstale":"2024-09-21T05:39:24.225816+0000","last_undegraded":"2024-09-21T05:39:24.225816+0000","last_fullsized":"2024-09-21T05:39:24.225816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:51.148096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226090+0000","last_change":"2024-09-21T05:39:21.830622+0000","last_active":"2024-09-21T05:39:24.226090+0000","last_peered":"2024-09-21T05:39:24.226090+0000","last_clean":"2024-09-21T05:39:24.226090+0000","last_became_active":"2024-09-21T05:39:21.829994+0000","last_became_peered":"2024-09-21T05:39:21.829994+0000","last_unstale":"2024-09-21T05:39:24.226090+0000","last_undegraded":"2024-09-21T05:39:24.226090+0000","last_fullsized":"2024-09-21T05:39:24.226090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:41.692740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":664479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:28.789 INFO:tasks.ceph.mds.a.smithi022.stderr:2024-09-21T05:39:28.787+0000 7f46c25e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:39:31.404 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 -- 172.21.15.22:0/483809218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0b3910 msgr2=0x7f96dc0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/483809218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0b3910 0x7f96dc0b4790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f96dc08e3c0 tx=0x7f96cc02a820 comp rx=0 tx=0).stop 2024-09-21T05:39:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 -- 172.21.15.22:0/483809218 shutdown_connections 2024-09-21T05:39:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/483809218 >> v2:172.21.15.134:3300/0 conn(0x7f96dc0cfbc0 0x7f96dc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/483809218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0b3910 0x7f96dc0b4790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 -- 172.21.15.22:0/483809218 >> 172.21.15.22:0/483809218 conn(0x7f96dc08bfc0 msgr2=0x7f96dc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 -- 172.21.15.22:0/483809218 shutdown_connections 2024-09-21T05:39:31.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.534+0000 7f96e0aa2640 1 -- 172.21.15.22:0/483809218 wait complete. 2024-09-21T05:39:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.536+0000 7f96e0aa2640 1 Processor -- start 2024-09-21T05:39:31.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.536+0000 7f96e0aa2640 1 -- start start 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.537+0000 7f96e0aa2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96dc0b3910 0x7f96dc15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.537+0000 7f96e0aa2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc0cfbc0 0x7f96dc0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.537+0000 7f96e0aa2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 0x7f96dc1f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.537+0000 7f96e0aa2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0b4d80 con 0x7f96dc0d4230 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.537+0000 7f96e0aa2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc0b5a80 con 0x7f96dc0cfbc0 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.537+0000 7f96e0aa2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f96dc0b5c00 con 0x7f96dc0b3910 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 0x7f96dc1f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 0x7f96dc1f92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46614/0 (socket says 172.21.15.22:46614) 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 -- 172.21.15.22:0/1152353537 learned_addr learned my addr 172.21.15.22:0/1152353537 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96da59c640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96dc0b3910 0x7f96dc15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96d9d9b640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc0cfbc0 0x7f96dc0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 -- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96dc0b3910 msgr2=0x7f96dc15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96dc0b3910 0x7f96dc15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 -- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc0cfbc0 msgr2=0x7f96dc0cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc0cfbc0 0x7f96dc0cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 -- 172.21.15.22:0/1152353537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc0cb7a0 con 0x7f96dc0d4230 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96dad9d640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 0x7f96dc1f92b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f96d4001eb0 tx=0x7f96d400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4011790 con 0x7f96dc0d4230 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96d4017070 con 0x7f96dc0d4230 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc15fa80 con 0x7f96dc0d4230 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4011ce0 con 0x7f96dc0d4230 2024-09-21T05:39:31.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.538+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc15fd60 con 0x7f96dc0d4230 2024-09-21T05:39:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.539+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc0ce320 con 0x7f96dc0d4230 2024-09-21T05:39:31.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.540+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96d401c460 con 0x7f96dc0d4230 2024-09-21T05:39:31.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.541+0000 7f96caffd640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078bc0 0x7f96bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.541+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f96d409a960 con 0x7f96dc0d4230 2024-09-21T05:39:31.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.541+0000 7f96da59c640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078bc0 0x7f96bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.541+0000 7f96da59c640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078bc0 0x7f96bc07b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0b7180 tx=0x7f96cc004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:31.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.545+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96d4062750 con 0x7f96dc0d4230 2024-09-21T05:39:31.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.729+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96dc069550 con 0x7f96bc078bc0 2024-09-21T05:39:31.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.734+0000 7f96caffd640 1 -- 172.21.15.22:0/1152353537 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f96dc069550 con 0x7f96bc078bc0 2024-09-21T05:39:31.738 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:31.739 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078bc0 msgr2=0x7f96bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078bc0 0x7f96bc07b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0b7180 tx=0x7f96cc004ab0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 msgr2=0x7f96dc1f92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 0x7f96dc1f92b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f96d4001eb0 tx=0x7f96d400c820 comp rx=0 tx=0).stop 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 shutdown_connections 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96dc0d4230 0x7f96dc1f92b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96dc0cfbc0 0x7f96dc0cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078bc0 0x7f96bc07b0d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 --2- 172.21.15.22:0/1152353537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96dc0b3910 0x7f96dc15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 >> 172.21.15.22:0/1152353537 conn(0x7f96dc08bfc0 msgr2=0x7f96dc0b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 shutdown_connections 2024-09-21T05:39:31.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.737+0000 7f96e0aa2640 1 -- 172.21.15.22:0/1152353537 wait complete. 2024-09-21T05:39:31.750 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-21T05:39:29.738730+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239896,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734824,"statfs":{"total":773094113280,"available":772848459776,"internally_reserved":0,"allocated":2924544,"data_stored":1773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.929781"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226309+0000","last_change":"2024-09-21T05:39:22.834732+0000","last_active":"2024-09-21T05:39:24.226309+0000","last_peered":"2024-09-21T05:39:24.226309+0000","last_clean":"2024-09-21T05:39:24.226309+0000","last_became_active":"2024-09-21T05:39:22.833098+0000","last_became_peered":"2024-09-21T05:39:22.833098+0000","last_unstale":"2024-09-21T05:39:24.226309+0000","last_undegraded":"2024-09-21T05:39:24.226309+0000","last_fullsized":"2024-09-21T05:39:24.226309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.341720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225995+0000","last_change":"2024-09-21T05:39:22.838816+0000","last_active":"2024-09-21T05:39:24.225995+0000","last_peered":"2024-09-21T05:39:24.225995+0000","last_clean":"2024-09-21T05:39:24.225995+0000","last_became_active":"2024-09-21T05:39:22.838511+0000","last_became_peered":"2024-09-21T05:39:22.838511+0000","last_unstale":"2024-09-21T05:39:24.225995+0000","last_undegraded":"2024-09-21T05:39:24.225995+0000","last_fullsized":"2024-09-21T05:39:24.225995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:27.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225652+0000","last_change":"2024-09-21T05:39:22.843633+0000","last_active":"2024-09-21T05:39:24.225652+0000","last_peered":"2024-09-21T05:39:24.225652+0000","last_clean":"2024-09-21T05:39:24.225652+0000","last_became_active":"2024-09-21T05:39:22.843200+0000","last_became_peered":"2024-09-21T05:39:22.843200+0000","last_unstale":"2024-09-21T05:39:24.225652+0000","last_undegraded":"2024-09-21T05:39:24.225652+0000","last_fullsized":"2024-09-21T05:39:24.225652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:30.085035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226390+0000","last_change":"2024-09-21T05:39:22.836751+0000","last_active":"2024-09-21T05:39:24.226390+0000","last_peered":"2024-09-21T05:39:24.226390+0000","last_clean":"2024-09-21T05:39:24.226390+0000","last_became_active":"2024-09-21T05:39:22.833532+0000","last_became_peered":"2024-09-21T05:39:22.833532+0000","last_unstale":"2024-09-21T05:39:24.226390+0000","last_undegraded":"2024-09-21T05:39:24.226390+0000","last_fullsized":"2024-09-21T05:39:24.226390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:07.246445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831080+0000","last_change":"2024-09-21T05:39:22.821659+0000","last_active":"2024-09-21T05:39:23.831080+0000","last_peered":"2024-09-21T05:39:23.831080+0000","last_clean":"2024-09-21T05:39:23.831080+0000","last_became_active":"2024-09-21T05:39:22.818471+0000","last_became_peered":"2024-09-21T05:39:22.818471+0000","last_unstale":"2024-09-21T05:39:23.831080+0000","last_undegraded":"2024-09-21T05:39:23.831080+0000","last_fullsized":"2024-09-21T05:39:23.831080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:11.829615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226338+0000","last_change":"2024-09-21T05:39:22.837584+0000","last_active":"2024-09-21T05:39:24.226338+0000","last_peered":"2024-09-21T05:39:24.226338+0000","last_clean":"2024-09-21T05:39:24.226338+0000","last_became_active":"2024-09-21T05:39:22.837243+0000","last_became_peered":"2024-09-21T05:39:22.837243+0000","last_unstale":"2024-09-21T05:39:24.226338+0000","last_undegraded":"2024-09-21T05:39:24.226338+0000","last_fullsized":"2024-09-21T05:39:24.226338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:46.224414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830586+0000","last_change":"2024-09-21T05:39:22.822050+0000","last_active":"2024-09-21T05:39:23.830586+0000","last_peered":"2024-09-21T05:39:23.830586+0000","last_clean":"2024-09-21T05:39:23.830586+0000","last_became_active":"2024-09-21T05:39:22.821705+0000","last_became_peered":"2024-09-21T05:39:22.821705+0000","last_unstale":"2024-09-21T05:39:23.830586+0000","last_undegraded":"2024-09-21T05:39:23.830586+0000","last_fullsized":"2024-09-21T05:39:23.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:42.204486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831067+0000","last_change":"2024-09-21T05:39:22.820929+0000","last_active":"2024-09-21T05:39:23.831067+0000","last_peered":"2024-09-21T05:39:23.831067+0000","last_clean":"2024-09-21T05:39:23.831067+0000","last_became_active":"2024-09-21T05:39:22.820587+0000","last_became_peered":"2024-09-21T05:39:22.820587+0000","last_unstale":"2024-09-21T05:39:23.831067+0000","last_undegraded":"2024-09-21T05:39:23.831067+0000","last_fullsized":"2024-09-21T05:39:23.831067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:42.698282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226051+0000","last_change":"2024-09-21T05:39:22.838353+0000","last_active":"2024-09-21T05:39:24.226051+0000","last_peered":"2024-09-21T05:39:24.226051+0000","last_clean":"2024-09-21T05:39:24.226051+0000","last_became_active":"2024-09-21T05:39:22.838044+0000","last_became_peered":"2024-09-21T05:39:22.838044+0000","last_unstale":"2024-09-21T05:39:24.226051+0000","last_undegraded":"2024-09-21T05:39:24.226051+0000","last_fullsized":"2024-09-21T05:39:24.226051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:27.463085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225891+0000","last_change":"2024-09-21T05:39:22.833336+0000","last_active":"2024-09-21T05:39:24.225891+0000","last_peered":"2024-09-21T05:39:24.225891+0000","last_clean":"2024-09-21T05:39:24.225891+0000","last_became_active":"2024-09-21T05:39:22.832959+0000","last_became_peered":"2024-09-21T05:39:22.832959+0000","last_unstale":"2024-09-21T05:39:24.225891+0000","last_undegraded":"2024-09-21T05:39:24.225891+0000","last_fullsized":"2024-09-21T05:39:24.225891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.450059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225617+0000","last_change":"2024-09-21T05:39:22.834980+0000","last_active":"2024-09-21T05:39:24.225617+0000","last_peered":"2024-09-21T05:39:24.225617+0000","last_clean":"2024-09-21T05:39:24.225617+0000","last_became_active":"2024-09-21T05:39:22.834075+0000","last_became_peered":"2024-09-21T05:39:22.834075+0000","last_unstale":"2024-09-21T05:39:24.225617+0000","last_undegraded":"2024-09-21T05:39:24.225617+0000","last_fullsized":"2024-09-21T05:39:24.225617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:31.814988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830803+0000","last_change":"2024-09-21T05:39:21.824655+0000","last_active":"2024-09-21T05:39:23.830803+0000","last_peered":"2024-09-21T05:39:23.830803+0000","last_clean":"2024-09-21T05:39:23.830803+0000","last_became_active":"2024-09-21T05:39:21.824309+0000","last_became_peered":"2024-09-21T05:39:21.824309+0000","last_unstale":"2024-09-21T05:39:23.830803+0000","last_undegraded":"2024-09-21T05:39:23.830803+0000","last_fullsized":"2024-09-21T05:39:23.830803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:50.559527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226272+0000","last_change":"2024-09-21T05:39:22.830496+0000","last_active":"2024-09-21T05:39:24.226272+0000","last_peered":"2024-09-21T05:39:24.226272+0000","last_clean":"2024-09-21T05:39:24.226272+0000","last_became_active":"2024-09-21T05:39:22.829859+0000","last_became_peered":"2024-09-21T05:39:22.829859+0000","last_unstale":"2024-09-21T05:39:24.226272+0000","last_undegraded":"2024-09-21T05:39:24.226272+0000","last_fullsized":"2024-09-21T05:39:24.226272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:09.763411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226478+0000","last_change":"2024-09-21T05:39:22.837101+0000","last_active":"2024-09-21T05:39:24.226478+0000","last_peered":"2024-09-21T05:39:24.226478+0000","last_clean":"2024-09-21T05:39:24.226478+0000","last_became_active":"2024-09-21T05:39:22.834638+0000","last_became_peered":"2024-09-21T05:39:22.834638+0000","last_unstale":"2024-09-21T05:39:24.226478+0000","last_undegraded":"2024-09-21T05:39:24.226478+0000","last_fullsized":"2024-09-21T05:39:24.226478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.717004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225997+0000","last_change":"2024-09-21T05:39:22.828178+0000","last_active":"2024-09-21T05:39:24.225997+0000","last_peered":"2024-09-21T05:39:24.225997+0000","last_clean":"2024-09-21T05:39:24.225997+0000","last_became_active":"2024-09-21T05:39:22.827833+0000","last_became_peered":"2024-09-21T05:39:22.827833+0000","last_unstale":"2024-09-21T05:39:24.225997+0000","last_undegraded":"2024-09-21T05:39:24.225997+0000","last_fullsized":"2024-09-21T05:39:24.225997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:58.078709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225514+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:24.225514+0000","last_peered":"2024-09-21T05:39:24.225514+0000","last_clean":"2024-09-21T05:39:24.225514+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:24.225514+0000","last_undegraded":"2024-09-21T05:39:24.225514+0000","last_fullsized":"2024-09-21T05:39:24.225514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225383+0000","last_change":"2024-09-21T05:39:21.833248+0000","last_active":"2024-09-21T05:39:24.225383+0000","last_peered":"2024-09-21T05:39:24.225383+0000","last_clean":"2024-09-21T05:39:24.225383+0000","last_became_active":"2024-09-21T05:39:21.832873+0000","last_became_peered":"2024-09-21T05:39:21.832873+0000","last_unstale":"2024-09-21T05:39:24.225383+0000","last_undegraded":"2024-09-21T05:39:24.225383+0000","last_fullsized":"2024-09-21T05:39:24.225383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:29.420862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830594+0000","last_change":"2024-09-21T05:39:22.831920+0000","last_active":"2024-09-21T05:39:23.830594+0000","last_peered":"2024-09-21T05:39:23.830594+0000","last_clean":"2024-09-21T05:39:23.830594+0000","last_became_active":"2024-09-21T05:39:22.831421+0000","last_became_peered":"2024-09-21T05:39:22.831421+0000","last_unstale":"2024-09-21T05:39:23.830594+0000","last_undegraded":"2024-09-21T05:39:23.830594+0000","last_fullsized":"2024-09-21T05:39:23.830594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:56.775839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226188+0000","last_change":"2024-09-21T05:39:21.831044+0000","last_active":"2024-09-21T05:39:24.226188+0000","last_peered":"2024-09-21T05:39:24.226188+0000","last_clean":"2024-09-21T05:39:24.226188+0000","last_became_active":"2024-09-21T05:39:21.830412+0000","last_became_peered":"2024-09-21T05:39:21.830412+0000","last_unstale":"2024-09-21T05:39:24.226188+0000","last_undegraded":"2024-09-21T05:39:24.226188+0000","last_fullsized":"2024-09-21T05:39:24.226188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:36.640541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225380+0000","last_change":"2024-09-21T05:39:21.833615+0000","last_active":"2024-09-21T05:39:24.225380+0000","last_peered":"2024-09-21T05:39:24.225380+0000","last_clean":"2024-09-21T05:39:24.225380+0000","last_became_active":"2024-09-21T05:39:21.833028+0000","last_became_peered":"2024-09-21T05:39:21.833028+0000","last_unstale":"2024-09-21T05:39:24.225380+0000","last_undegraded":"2024-09-21T05:39:24.225380+0000","last_fullsized":"2024-09-21T05:39:24.225380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:28.358924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829777+0000","last_change":"2024-09-21T05:39:22.827047+0000","last_active":"2024-09-21T05:39:23.829777+0000","last_peered":"2024-09-21T05:39:23.829777+0000","last_clean":"2024-09-21T05:39:23.829777+0000","last_became_active":"2024-09-21T05:39:22.825920+0000","last_became_peered":"2024-09-21T05:39:22.825920+0000","last_unstale":"2024-09-21T05:39:23.829777+0000","last_undegraded":"2024-09-21T05:39:23.829777+0000","last_fullsized":"2024-09-21T05:39:23.829777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:56.401187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226082+0000","last_change":"2024-09-21T05:39:21.829961+0000","last_active":"2024-09-21T05:39:24.226082+0000","last_peered":"2024-09-21T05:39:24.226082+0000","last_clean":"2024-09-21T05:39:24.226082+0000","last_became_active":"2024-09-21T05:39:21.829484+0000","last_became_peered":"2024-09-21T05:39:21.829484+0000","last_unstale":"2024-09-21T05:39:24.226082+0000","last_undegraded":"2024-09-21T05:39:24.226082+0000","last_fullsized":"2024-09-21T05:39:24.226082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:16.446491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225816+0000","last_change":"2024-09-21T05:39:21.830335+0000","last_active":"2024-09-21T05:39:24.225816+0000","last_peered":"2024-09-21T05:39:24.225816+0000","last_clean":"2024-09-21T05:39:24.225816+0000","last_became_active":"2024-09-21T05:39:21.829550+0000","last_became_peered":"2024-09-21T05:39:21.829550+0000","last_unstale":"2024-09-21T05:39:24.225816+0000","last_undegraded":"2024-09-21T05:39:24.225816+0000","last_fullsized":"2024-09-21T05:39:24.225816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:51.148096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226090+0000","last_change":"2024-09-21T05:39:21.830622+0000","last_active":"2024-09-21T05:39:24.226090+0000","last_peered":"2024-09-21T05:39:24.226090+0000","last_clean":"2024-09-21T05:39:24.226090+0000","last_became_active":"2024-09-21T05:39:21.829994+0000","last_became_peered":"2024-09-21T05:39:21.829994+0000","last_unstale":"2024-09-21T05:39:24.226090+0000","last_undegraded":"2024-09-21T05:39:24.226090+0000","last_fullsized":"2024-09-21T05:39:24.226090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:41.692740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":664479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:31.754 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:39:31.755 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T05:39:31.755 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 -- 172.21.15.22:0/354533058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac155ce0 msgr2=0x7fd8ac0b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/354533058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac0b45a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009830 tx=0x7fd8a402a510 comp rx=0 tx=0).stop 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 -- 172.21.15.22:0/354533058 shutdown_connections 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/354533058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac0b45a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/354533058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8ac15c580 0x7fd8ac070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 -- 172.21.15.22:0/354533058 >> 172.21.15.22:0/354533058 conn(0x7fd8ac06def0 msgr2=0x7fd8ac06e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 -- 172.21.15.22:0/354533058 shutdown_connections 2024-09-21T05:39:31.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.906+0000 7fd8b18f9640 1 -- 172.21.15.22:0/354533058 wait complete. 2024-09-21T05:39:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.908+0000 7fd8b18f9640 1 Processor -- start 2024-09-21T05:39:31.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.908+0000 7fd8b18f9640 1 -- start start 2024-09-21T05:39:31.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.909+0000 7fd8b18f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 0x7fd8ac1f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.909+0000 7fd8b18f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac1f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.909+0000 7fd8b18f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8ac158890 0x7fd8ac15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.909+0000 7fd8b18f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac0b5060 con 0x7fd8ac074420 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.909+0000 7fd8b18f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac0b4ee0 con 0x7fd8ac155ce0 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.909+0000 7fd8b18f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac0b51e0 con 0x7fd8ac158890 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 0x7fd8ac1f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 0x7fd8ac1f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46626/0 (socket says 172.21.15.22:46626) 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 -- 172.21.15.22:0/493154905 learned_addr learned my addr 172.21.15.22:0/493154905 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8ab7fe640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8ac158890 0x7fd8ac15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aa7fc640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac1f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 -- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8ac158890 msgr2=0x7fd8ac15a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8ac158890 0x7fd8ac15a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 -- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8ac155ce0 msgr2=0x7fd8ac1f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac1f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:31.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 -- 172.21.15.22:0/493154905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac1f3100 con 0x7fd8ac074420 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aa7fc640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8aaffd640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 0x7fd8ac1f3b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00cbf0 tx=0x7fd89c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c016020 con 0x7fd8ac074420 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd89c00ad90 con 0x7fd8ac074420 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c012680 con 0x7fd8ac074420 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac1f71b0 con 0x7fd8ac074420 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.910+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1f3380 con 0x7fd8ac074420 2024-09-21T05:39:31.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.911+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac0b5700 con 0x7fd8ac074420 2024-09-21T05:39:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.912+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd89c019020 con 0x7fd8ac074420 2024-09-21T05:39:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.913+0000 7fd893fff640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd898078b90 0x7fd89807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.913+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd89c09b430 con 0x7fd8ac074420 2024-09-21T05:39:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.913+0000 7fd8aa7fc640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd898078b90 0x7fd89807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:31.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.913+0000 7fd8aa7fc640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd898078b90 0x7fd89807b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac151340 tx=0x7fd8a0005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:31.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:31.917+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd89c020020 con 0x7fd8ac074420 2024-09-21T05:39:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.106+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd8ac15da80 con 0x7fd8ac074420 2024-09-21T05:39:32.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.106+0000 7fd893fff640 1 -- 172.21.15.22:0/493154905 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fd8ac1f3100 con 0x7fd8ac074420 2024-09-21T05:39:32.108 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd898078b90 msgr2=0x7fd89807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd898078b90 0x7fd89807b0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac151340 tx=0x7fd8a0005f90 comp rx=0 tx=0).stop 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 msgr2=0x7fd8ac1f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 0x7fd8ac1f3b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00cbf0 tx=0x7fd89c00a060 comp rx=0 tx=0).stop 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 shutdown_connections 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd898078b90 0x7fd89807b0a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd8ac158890 0x7fd8ac15a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd8ac155ce0 0x7fd8ac1f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 --2- 172.21.15.22:0/493154905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd8ac074420 0x7fd8ac1f3b80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 >> 172.21.15.22:0/493154905 conn(0x7fd8ac06def0 msgr2=0x7fd8ac15e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 shutdown_connections 2024-09-21T05:39:32.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.109+0000 7fd8b18f9640 1 -- 172.21.15.22:0/493154905 wait complete. 2024-09-21T05:39:32.122 INFO:teuthology.orchestra.run.smithi022.stdout:[{"version":31,"timestamp":"2024-09-21T05:39:09.759958+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"5971.638123"}]}] 2024-09-21T05:39:32.122 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T05:39:32.122 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T05:39:32.122 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T05:39:32.122 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.236+0000 7f6343f57640 1 -- 172.21.15.22:0/3484768907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 msgr2=0x7f6328004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.236+0000 7f6343f57640 1 --2- 172.21.15.22:0/3484768907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f6328004c10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f632c001050 tx=0x7f6328005830 comp rx=0 tx=0).stop 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.236+0000 7f6343f57640 1 -- 172.21.15.22:0/3484768907 shutdown_connections 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.236+0000 7f6343f57640 1 --2- 172.21.15.22:0/3484768907 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f6328004c10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.236+0000 7f6343f57640 1 -- 172.21.15.22:0/3484768907 >> 172.21.15.22:0/3484768907 conn(0x7f633c08bfc0 msgr2=0x7f633c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.237+0000 7f6343f57640 1 -- 172.21.15.22:0/3484768907 shutdown_connections 2024-09-21T05:39:32.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.237+0000 7f6343f57640 1 -- 172.21.15.22:0/3484768907 wait complete. 2024-09-21T05:39:32.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.238+0000 7f6343f57640 1 Processor -- start 2024-09-21T05:39:32.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.238+0000 7f6343f57640 1 -- start start 2024-09-21T05:39:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f633c0c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c14d540 0x7f633c14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f633c14e070 0x7f633c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d1c00 con 0x7f6328004800 2024-09-21T05:39:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f633c0d1a80 con 0x7f633c14d540 2024-09-21T05:39:32.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f633c0d1d80 con 0x7f633c14e070 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f633c0c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f633c0c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46636/0 (socket says 172.21.15.22:46636) 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 -- 172.21.15.22:0/1035415291 learned_addr learned my addr 172.21.15.22:0/1035415291 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63414f2640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c14d540 0x7f633c14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63424f4640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f633c14e070 0x7f633c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 -- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f633c14e070 msgr2=0x7f633c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f633c14e070 0x7f633c14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 -- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c14d540 msgr2=0x7f633c14d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c14d540 0x7f633c14d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 -- 172.21.15.22:0/1035415291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c0c8940 con 0x7f6328004800 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63414f2640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c14d540 0x7f633c14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63424f4640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f633c14e070 0x7f633c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6341cf3640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f633c0c80f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6338011ab0 tx=0x7f6338011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633801e070 con 0x7f6328004800 2024-09-21T05:39:32.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f633800fc70 con 0x7f6328004800 2024-09-21T05:39:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c0c8c20 con 0x7f6328004800 2024-09-21T05:39:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338019670 con 0x7f6328004800 2024-09-21T05:39:32.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.240+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c0c91e0 con 0x7f6328004800 2024-09-21T05:39:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.242+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633c0cf310 con 0x7f6328004800 2024-09-21T05:39:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.242+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6338021070 con 0x7f6328004800 2024-09-21T05:39:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.243+0000 7f63327fc640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6320078bc0 0x7f632007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.243+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f633809b4f0 con 0x7f6328004800 2024-09-21T05:39:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.243+0000 7f63414f2640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6320078bc0 0x7f632007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.243+0000 7f63414f2640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6320078bc0 0x7f632007b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f632c033000 tx=0x7f632c0019f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:32.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.247+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f633800fe60 con 0x7f6328004800 2024-09-21T05:39:32.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.484+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f633c0b51a0 con 0x7f6328004800 2024-09-21T05:39:32.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.487+0000 7f63327fc640 1 -- 172.21.15.22:0/1035415291 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194721 (secure 0 0 0) 0x7f6338063370 con 0x7f6328004800 2024-09-21T05:39:32.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6320078bc0 msgr2=0x7f632007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6320078bc0 0x7f632007b0d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f632c033000 tx=0x7f632c0019f0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 msgr2=0x7f633c0c80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f633c0c80f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6338011ab0 tx=0x7f6338011f70 comp rx=0 tx=0).stop 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 shutdown_connections 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6320078bc0 0x7f632007b0d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f633c14e070 0x7f633c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f633c14d540 0x7f633c14d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.490+0000 7f6343f57640 1 --2- 172.21.15.22:0/1035415291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6328004800 0x7f633c0c80f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.491+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 >> 172.21.15.22:0/1035415291 conn(0x7f633c08bfc0 msgr2=0x7f633c0d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.491+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 shutdown_connections 2024-09-21T05:39:32.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.491+0000 7f6343f57640 1 -- 172.21.15.22:0/1035415291 wait complete. 2024-09-21T05:39:32.504 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":3406357127},{"type":"v1","addr":"172.21.15.134:6801","nonce":3406357127}]},"active_addr":"172.21.15.134:6801/3406357127","active_change":"2024-09-21T05:39:01.727780+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":1318568167}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":459419682}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":2825898736}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":4199298994}]}]} 2024-09-21T05:39:32.507 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T05:39:32.507 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T05:39:32.508 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 -- 172.21.15.22:0/3445806364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780d1a80 msgr2=0x7f76780cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 --2- 172.21.15.22:0/3445806364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780d1a80 0x7f76780cf090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7668009860 tx=0x7f766802a5a0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 -- 172.21.15.22:0/3445806364 shutdown_connections 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 --2- 172.21.15.22:0/3445806364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780d1a80 0x7f76780cf090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 -- 172.21.15.22:0/3445806364 >> 172.21.15.22:0/3445806364 conn(0x7f767808bfc0 msgr2=0x7f767809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 -- 172.21.15.22:0/3445806364 shutdown_connections 2024-09-21T05:39:32.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.622+0000 7f767eb39640 1 -- 172.21.15.22:0/3445806364 wait complete. 2024-09-21T05:39:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.623+0000 7f767eb39640 1 Processor -- start 2024-09-21T05:39:32.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.623+0000 7f767eb39640 1 -- start start 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767eb39640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780cf5d0 0x7f76781f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767eb39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76780d1a80 0x7f7678154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767eb39640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 0x7f767809e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767eb39640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7678162f70 con 0x7f76780d2ed0 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767eb39640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7678162df0 con 0x7f76780d1a80 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767eb39640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f76781630f0 con 0x7f76780cf5d0 2024-09-21T05:39:32.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 0x7f767809e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 0x7f767809e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46644/0 (socket says 172.21.15.22:46644) 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 -- 172.21.15.22:0/1867225566 learned_addr learned my addr 172.21.15.22:0/1867225566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767c8d5640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780cf5d0 0x7f76781f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f7677fff640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76780d1a80 0x7f7678154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 -- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780cf5d0 msgr2=0x7f76781f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780cf5d0 0x7f76781f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 -- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76780d1a80 msgr2=0x7f7678154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76780d1a80 0x7f7678154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f767d0d6640 1 -- 172.21.15.22:0/1867225566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76781f6070 con 0x7f76780d2ed0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.625+0000 7f7677fff640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76780d1a80 0x7f7678154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.626+0000 7f767d0d6640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 0x7f767809e200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7670002230 tx=0x7f767000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.626+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670016020 con 0x7f76780d2ed0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.626+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f767000ace0 con 0x7f76780d2ed0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.626+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670012680 con 0x7f76780d2ed0 2024-09-21T05:39:32.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.626+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76780a0090 con 0x7f76780d2ed0 2024-09-21T05:39:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.626+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781f6860 con 0x7f76780d2ed0 2024-09-21T05:39:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.627+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76781f62f0 con 0x7f76780d2ed0 2024-09-21T05:39:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.628+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7670019020 con 0x7f76780d2ed0 2024-09-21T05:39:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.628+0000 7f76757fa640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7658078b90 0x7f765807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.628+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f767009b380 con 0x7f76780d2ed0 2024-09-21T05:39:32.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.628+0000 7f767c8d5640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7658078b90 0x7f765807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.629+0000 7f767c8d5640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7658078b90 0x7f765807b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7660007fd0 tx=0x7f7660009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:32.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.633+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7670020020 con 0x7f76780d2ed0 2024-09-21T05:39:32.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.822+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76780cf090 con 0x7f76780d2ed0 2024-09-21T05:39:32.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.823+0000 7f76757fa640 1 -- 172.21.15.22:0/1867225566 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7f76700637c0 con 0x7f76780d2ed0 2024-09-21T05:39:32.826 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:32.826 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:32.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7658078b90 msgr2=0x7f765807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7658078b90 0x7f765807b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7660007fd0 tx=0x7f7660009040 comp rx=0 tx=0).stop 2024-09-21T05:39:32.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 msgr2=0x7f767809e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 0x7f767809e200 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7670002230 tx=0x7f767000cf50 comp rx=0 tx=0).stop 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 shutdown_connections 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76780d2ed0 0x7f767809e200 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76780d1a80 0x7f7678154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7658078b90 0x7f765807b0a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 --2- 172.21.15.22:0/1867225566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76780cf5d0 0x7f76781f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 >> 172.21.15.22:0/1867225566 conn(0x7f767808bfc0 msgr2=0x7f76780b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 shutdown_connections 2024-09-21T05:39:32.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.826+0000 7f767eb39640 1 -- 172.21.15.22:0/1867225566 wait complete. 2024-09-21T05:39:32.839 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T05:39:32.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 -- 172.21.15.22:0/1747455152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b3a10 msgr2=0x7f981c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 --2- 172.21.15.22:0/1747455152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b3a10 0x7f981c0cfa20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f981c0d34f0 tx=0x7f981402a890 comp rx=0 tx=0).stop 2024-09-21T05:39:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 -- 172.21.15.22:0/1747455152 shutdown_connections 2024-09-21T05:39:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 --2- 172.21.15.22:0/1747455152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b3a10 0x7f981c0cfa20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 -- 172.21.15.22:0/1747455152 >> 172.21.15.22:0/1747455152 conn(0x7f981c09d8f0 msgr2=0x7f981c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:32.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 -- 172.21.15.22:0/1747455152 shutdown_connections 2024-09-21T05:39:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.954+0000 7f982283f640 1 -- 172.21.15.22:0/1747455152 wait complete. 2024-09-21T05:39:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.956+0000 7f982283f640 1 Processor -- start 2024-09-21T05:39:32.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.956+0000 7f982283f640 1 -- start start 2024-09-21T05:39:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f982283f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f981c0b3a10 0x7f981c0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f982283f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f981c0b9b60 0x7f981c0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f982283f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 0x7f981c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f982283f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f981c0d02f0 con 0x7f981c0b4000 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f982283f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f981c0d0170 con 0x7f981c0b9b60 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f982283f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f981c16a5a0 con 0x7f981c0b3a10 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f9820ddc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 0x7f981c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f9820ddc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 0x7f981c0b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46668/0 (socket says 172.21.15.22:46668) 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f9820ddc640 1 -- 172.21.15.22:0/3419389993 learned_addr learned my addr 172.21.15.22:0/3419389993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f981bfff640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f981c0b3a10 0x7f981c0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.957+0000 7f981b7fe640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f981c0b9b60 0x7f981c0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9820ddc640 1 -- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f981c0b3a10 msgr2=0x7f981c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9820ddc640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f981c0b3a10 0x7f981c0ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9820ddc640 1 -- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f981c0b9b60 msgr2=0x7f981c0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9820ddc640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f981c0b9b60 0x7f981c0cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9820ddc640 1 -- 172.21.15.22:0/3419389993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c0c9f50 con 0x7f981c0b4000 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f981bfff640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f981c0b3a10 0x7f981c0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9820ddc640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 0x7f981c0b4dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f981402ae80 tx=0x7f9814033d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9814031e40 con 0x7f981c0b4000 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9814003a60 con 0x7f981c0b4000 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c0ca230 con 0x7f981c0b4000 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98140330d0 con 0x7f981c0b4000 2024-09-21T05:39:32.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.958+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1f9050 con 0x7f981c0b4000 2024-09-21T05:39:32.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.959+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f981c0ba0d0 con 0x7f981c0b4000 2024-09-21T05:39:32.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.960+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9814033270 con 0x7f981c0b4000 2024-09-21T05:39:32.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.960+0000 7f9818ff9640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9808078b90 0x7f980807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:32.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.960+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f98140bbf70 con 0x7f981c0b4000 2024-09-21T05:39:32.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.961+0000 7f981bfff640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9808078b90 0x7f980807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:32.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.961+0000 7f981bfff640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9808078b90 0x7f980807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f980c007fd0 tx=0x7f980c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:32.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:32.965+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f981403f030 con 0x7f981c0b4000 2024-09-21T05:39:33.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.152+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f981c0ceae0 con 0x7f981c0b4000 2024-09-21T05:39:33.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.153+0000 7f9818ff9640 1 -- 172.21.15.22:0/3419389993 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7f9814031790 con 0x7f981c0b4000 2024-09-21T05:39:33.156 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:33.156 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:33.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9808078b90 msgr2=0x7f980807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9808078b90 0x7f980807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f980c007fd0 tx=0x7f980c009040 comp rx=0 tx=0).stop 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 msgr2=0x7f981c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 0x7f981c0b4dd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f981402ae80 tx=0x7f9814033d80 comp rx=0 tx=0).stop 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 shutdown_connections 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f981c0b4000 0x7f981c0b4dd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f981c0b9b60 0x7f981c0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9808078b90 0x7f980807b0a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 --2- 172.21.15.22:0/3419389993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f981c0b3a10 0x7f981c0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 >> 172.21.15.22:0/3419389993 conn(0x7f981c09d8f0 msgr2=0x7f981c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 shutdown_connections 2024-09-21T05:39:33.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.156+0000 7f982283f640 1 -- 172.21.15.22:0/3419389993 wait complete. 2024-09-21T05:39:33.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T05:39:33.170 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T05:39:33.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T05:39:33.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T05:39:33.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T05:39:33.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T05:39:33.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T05:39:33.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T05:39:33.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e77fff640 1 -- 172.21.15.22:0/1815976448 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7802e050 con 0x7f9e800cac60 2024-09-21T05:39:33.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e87312640 1 -- 172.21.15.22:0/1815976448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e800cac60 msgr2=0x7f9e800cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e87312640 1 --2- 172.21.15.22:0/1815976448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e800cac60 0x7f9e800cee10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e78005fd0 tx=0x7f9e7802a840 comp rx=0 tx=0).stop 2024-09-21T05:39:33.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e87312640 1 -- 172.21.15.22:0/1815976448 shutdown_connections 2024-09-21T05:39:33.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e87312640 1 --2- 172.21.15.22:0/1815976448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e800cac60 0x7f9e800cee10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e87312640 1 --2- 172.21.15.22:0/1815976448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e800d0d70 0x7f9e800b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.401+0000 7f9e87312640 1 -- 172.21.15.22:0/1815976448 >> 172.21.15.22:0/1815976448 conn(0x7f9e80074170 msgr2=0x7f9e800b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.402+0000 7fe8f3fff640 1 -- 172.21.15.22:0/2404559510 reap_dead start 2024-09-21T05:39:33.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.402+0000 7fe8f27fc640 1 -- 172.21.15.22:0/2404559510 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4006b30 con 0x7fe8dc00c260 2024-09-21T05:39:33.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.402+0000 7fe8fa212640 1 -- 172.21.15.22:0/2404559510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 msgr2=0x7fe8dc00c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.402+0000 7fe8fa212640 1 --2- 172.21.15.22:0/2404559510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 0x7fe8dc00c6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4006f10 tx=0x7fe8dc00e040 comp rx=0 tx=0).stop 2024-09-21T05:39:33.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7fe8fa212640 1 -- 172.21.15.22:0/2404559510 shutdown_connections 2024-09-21T05:39:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7fe8fa212640 1 --2- 172.21.15.22:0/2404559510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 0x7fe8dc00c6b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7fe8fa212640 1 -- 172.21.15.22:0/2404559510 >> 172.21.15.22:0/2404559510 conn(0x7fe8f40cd480 msgr2=0x7fe8f40cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f16447ad640 1 -- 172.21.15.22:0/1275175676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1634035930 msgr2=0x7f1634037140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f16447ad640 1 --2- 172.21.15.22:0/1275175676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1634035930 0x7f1634037140 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f162c009830 tx=0x7f162c02a520 comp rx=0 tx=0).stop 2024-09-21T05:39:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7fe8fa212640 1 -- 172.21.15.22:0/2404559510 shutdown_connections 2024-09-21T05:39:33.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7fe8fa212640 1 -- 172.21.15.22:0/2404559510 wait complete. 2024-09-21T05:39:33.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f1f43319640 1 -- 172.21.15.22:0/1838788372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c06f2a0 msgr2=0x7f1f3c0cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f8601a97640 1 -- 172.21.15.22:0/3945583521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40e6bc0 msgr2=0x7f85f40e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f8601a97640 1 --2- 172.21.15.22:0/3945583521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40e6bc0 0x7f85f40e6fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f85fc000b60 tx=0x7f85fc0a6690 comp rx=0 tx=0).stop 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f1f43319640 1 --2- 172.21.15.22:0/1838788372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c06f2a0 0x7f1f3c0cccf0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0092b0 tx=0x7f1f2c02b730 comp rx=0 tx=0).stop 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.403+0000 7f9e87312640 1 -- 172.21.15.22:0/1815976448 shutdown_connections 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f9e87312640 1 -- 172.21.15.22:0/1815976448 wait complete. 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f8601a97640 1 -- 172.21.15.22:0/3945583521 shutdown_connections 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f8601a97640 1 --2- 172.21.15.22:0/3945583521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f40374e0 0x7f85f40378f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f8601a97640 1 --2- 172.21.15.22:0/3945583521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40e6bc0 0x7f85f40e6fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f8601a97640 1 -- 172.21.15.22:0/3945583521 >> 172.21.15.22:0/3945583521 conn(0x7f85f4055e60 msgr2=0x7f85f400e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f16447ad640 1 -- 172.21.15.22:0/1275175676 shutdown_connections 2024-09-21T05:39:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f16447ad640 1 --2- 172.21.15.22:0/1275175676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1634035930 0x7f1634037140 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f16447ad640 1 --2- 172.21.15.22:0/1275175676 >> v2:172.21.15.134:3300/0 conn(0x7f16340e45c0 0x7f16340dda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f16447ad640 1 -- 172.21.15.22:0/1275175676 >> 172.21.15.22:0/1275175676 conn(0x7f163401eb10 msgr2=0x7f163403a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f16447ad640 1 -- 172.21.15.22:0/1275175676 shutdown_connections 2024-09-21T05:39:33.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f8601a97640 1 -- 172.21.15.22:0/3945583521 shutdown_connections 2024-09-21T05:39:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f8601a97640 1 -- 172.21.15.22:0/3945583521 wait complete. 2024-09-21T05:39:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f16447ad640 1 -- 172.21.15.22:0/1275175676 wait complete. 2024-09-21T05:39:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f1f43319640 1 -- 172.21.15.22:0/1838788372 shutdown_connections 2024-09-21T05:39:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f1f43319640 1 --2- 172.21.15.22:0/1838788372 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c06f2a0 0x7f1f3c0cccf0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f1f43319640 1 -- 172.21.15.22:0/1838788372 >> 172.21.15.22:0/1838788372 conn(0x7f1f3c0b21d0 msgr2=0x7f1f3c0b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 -- 172.21.15.22:0/2849629992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58600af630 msgr2=0x7f58600ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 --2- 172.21.15.22:0/2849629992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58600af630 0x7f58600ccb30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f586015e620 tx=0x7f5850032830 comp rx=0 tx=0).stop 2024-09-21T05:39:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 -- 172.21.15.22:0/2849629992 shutdown_connections 2024-09-21T05:39:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 --2- 172.21.15.22:0/2849629992 >> v2:172.21.15.134:3300/0 conn(0x7f58600cf660 0x7f58600cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 --2- 172.21.15.22:0/2849629992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f58600af630 0x7f58600ccb30 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f586015e620 tx=0x7f5850032830 comp rx=0 tx=0).stop 2024-09-21T05:39:33.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 --2- 172.21.15.22:0/2849629992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5860070f20 0x7f58600cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.404+0000 7f586760a640 1 -- 172.21.15.22:0/2849629992 >> 172.21.15.22:0/2849629992 conn(0x7f58600b1cb0 msgr2=0x7f58600b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 -- 172.21.15.22:0/11008278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 msgr2=0x7f67700b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 --2- 172.21.15.22:0/11008278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 0x7f67700b3660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6760009830 tx=0x7f676002a510 comp rx=0 tx=0).stop 2024-09-21T05:39:33.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f1f43319640 1 -- 172.21.15.22:0/1838788372 shutdown_connections 2024-09-21T05:39:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 -- 172.21.15.22:0/11008278 shutdown_connections 2024-09-21T05:39:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 --2- 172.21.15.22:0/11008278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 0x7f67700b3660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 -- 172.21.15.22:0/11008278 >> 172.21.15.22:0/11008278 conn(0x7f677009d680 msgr2=0x7f67700d4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 -- 172.21.15.22:0/11008278 shutdown_connections 2024-09-21T05:39:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f586760a640 1 -- 172.21.15.22:0/2849629992 shutdown_connections 2024-09-21T05:39:33.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f677753f640 1 -- 172.21.15.22:0/11008278 wait complete. 2024-09-21T05:39:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f1d3740e640 1 -- 172.21.15.22:0/316596966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 msgr2=0x7f1d28053b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f1d3740e640 1 --2- 172.21.15.22:0/316596966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28053b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d3006ad50 tx=0x7f1d300a64c0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1d3740e640 1 -- 172.21.15.22:0/316596966 shutdown_connections 2024-09-21T05:39:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1d3740e640 1 --2- 172.21.15.22:0/316596966 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28053b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1d3740e640 1 --2- 172.21.15.22:0/316596966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d280520b0 0x7f1d28007c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1d3740e640 1 -- 172.21.15.22:0/316596966 >> 172.21.15.22:0/316596966 conn(0x7f1d2801e260 msgr2=0x7f1d28039e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7fe8fa212640 1 Processor -- start 2024-09-21T05:39:33.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1d3740e640 1 -- 172.21.15.22:0/316596966 shutdown_connections 2024-09-21T05:39:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1d3740e640 1 -- 172.21.15.22:0/316596966 wait complete. 2024-09-21T05:39:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f8601a97640 1 Processor -- start 2024-09-21T05:39:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f1f43319640 1 -- 172.21.15.22:0/1838788372 wait complete. 2024-09-21T05:39:33.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7f8601a97640 1 -- start start 2024-09-21T05:39:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.406+0000 7fe8fa212640 1 -- start start 2024-09-21T05:39:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.407+0000 7f9e87312640 1 Processor -- start 2024-09-21T05:39:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.407+0000 7f9e87312640 1 -- start start 2024-09-21T05:39:33.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.407+0000 7f16447ad640 1 Processor -- start 2024-09-21T05:39:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.407+0000 7f16447ad640 1 -- start start 2024-09-21T05:39:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.408+0000 7f8601a97640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40374e0 0x7f85f40d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.408+0000 7f8601a97640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f403e650 0x7f85f40da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f8601a97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f40e6bc0 0x7f85f40dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8fa212640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 0x7fe8f4157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f8600a95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40374e0 0x7f85f40d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8fa212640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8f4157060 0x7fe8f414c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8fa212640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8f41551d0 0x7fe8f40b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 0x7fe8f4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 0x7fe8f4157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34202/0 (socket says 172.21.15.22:34202) 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8f4157060 0x7fe8f414c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8f4157060 0x7fe8f414c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54116/0 (socket says 172.21.15.22:54116) 2024-09-21T05:39:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f677753f640 1 Processor -- start 2024-09-21T05:39:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f677753f640 1 -- start start 2024-09-21T05:39:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f8600a95640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40374e0 0x7f85f40d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54102/0 (socket says 172.21.15.22:54102) 2024-09-21T05:39:33.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8f41551d0 0x7fe8f40b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f1d3740e640 1 Processor -- start 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f1f43319640 1 Processor -- start 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8f41551d0 0x7fe8f40b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46734/0 (socket says 172.21.15.22:46734) 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8fa212640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41625a0 con 0x7fe8f41551d0 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8fa212640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4162420 con 0x7fe8f4157060 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8fa212640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f4162720 con 0x7fe8dc00c260 2024-09-21T05:39:33.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7fe8f2ffd640 1 -- 172.21.15.22:0/739487570 learned_addr learned my addr 172.21.15.22:0/739487570 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e87312640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8009aa30 0x7f9e801f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e87312640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e800d0d70 0x7f9e80149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e87312640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e8015a940 0x7f9e8015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e87312640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80162c60 con 0x7f9e8015a940 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f16447ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1634008ed0 0x7f1634052550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f16447ad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340e45c0 0x7f1634177460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f16447ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f163404ed30 0x7f163416a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f1f43319640 1 -- start start 2024-09-21T05:39:33.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f1d3740e640 1 -- start start 2024-09-21T05:39:33.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e87312640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80162ae0 con 0x7f9e8009aa30 2024-09-21T05:39:33.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e87312640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9e80162de0 con 0x7f9e800d0d70 2024-09-21T05:39:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f16447ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1634037a10 con 0x7f163404ed30 2024-09-21T05:39:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f16447ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1634037890 con 0x7f1634008ed0 2024-09-21T05:39:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.405+0000 7f586760a640 1 -- 172.21.15.22:0/2849629992 wait complete. 2024-09-21T05:39:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f16447ad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1634037b90 con 0x7f16340e45c0 2024-09-21T05:39:33.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e858af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e8015a940 0x7f9e8015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f586760a640 1 Processor -- start 2024-09-21T05:39:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.409+0000 7f586760a640 1 -- start start 2024-09-21T05:39:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e858af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e8015a940 0x7f9e8015ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46752/0 (socket says 172.21.15.22:46752) 2024-09-21T05:39:33.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f9e858af640 1 -- 172.21.15.22:0/1670685292 learned_addr learned my addr 172.21.15.22:0/1670685292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642549640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1634008ed0 0x7f1634052550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642d4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f163404ed30 0x7f163416a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642d4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f163404ed30 0x7f163416a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46766/0 (socket says 172.21.15.22:46766) 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642d4a640 1 -- 172.21.15.22:0/1495464910 learned_addr learned my addr 172.21.15.22:0/1495464910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7fe8f2ffd640 1 -- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 msgr2=0x7fe8f4157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7fe8f2ffd640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe8dc00c260 0x7fe8f4157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7fe8f2ffd640 1 -- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8f41551d0 msgr2=0x7fe8f40b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7fe8f2ffd640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe8f41551d0 0x7fe8f40b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7fe8f2ffd640 1 -- 172.21.15.22:0/739487570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f4154730 con 0x7fe8f4157060 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e848ad640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e800d0d70 0x7f9e80149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e850ae640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8009aa30 0x7f9e801f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e858af640 1 -- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e800d0d70 msgr2=0x7f9e80149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e858af640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9e800d0d70 0x7f9e80149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e858af640 1 -- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8009aa30 msgr2=0x7f9e801f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e858af640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9e8009aa30 0x7f9e801f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f9e858af640 1 -- 172.21.15.22:0/1670685292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e801f9410 con 0x7f9e8015a940 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f8601a97640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40588a0 con 0x7f85f40e6bc0 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f8601a97640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85f40574b0 con 0x7f85f40374e0 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f8601a97640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f85f4037e30 con 0x7f85f403e650 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f85fbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f403e650 0x7f85f40da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f85fbfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f403e650 0x7f85f40da6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34194/0 (socket says 172.21.15.22:34194) 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f85fbfff640 1 -- 172.21.15.22:0/3361573812 learned_addr learned my addr 172.21.15.22:0/3361573812 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.410+0000 7f8601296640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f40e6bc0 0x7f85f40dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8601296640 1 -- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f403e650 msgr2=0x7f85f40da6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8601296640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85f403e650 0x7f85f40da6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8601296640 1 -- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40374e0 msgr2=0x7f85f40d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8601296640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40374e0 0x7f85f40d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8601296640 1 -- 172.21.15.22:0/3361573812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f40db8d0 con 0x7f85f40e6bc0 2024-09-21T05:39:33.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8600a95640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f85f40374e0 0x7f85f40d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f8601296640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f40e6bc0 0x7f85f40dac00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fc0 tx=0x7f85e800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7fe8f2ffd640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8f4157060 0x7fe8f414c5e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00efc0 tx=0x7fe8ec00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1641d48640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340e45c0 0x7f1634177460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642549640 1 -- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340e45c0 msgr2=0x7f1634177460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642549640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340e45c0 0x7f1634177460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642549640 1 -- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f163404ed30 msgr2=0x7f163416a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642549640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f163404ed30 0x7f163416a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1642549640 1 -- 172.21.15.22:0/1495464910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1634050910 con 0x7f1634008ed0 2024-09-21T05:39:33.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.411+0000 7f1641d48640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f16340e45c0 0x7f1634177460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:33.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3740e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28048890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677753f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67700b9540 0x7f677015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3740e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280df800 0x7f1d280d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1642549640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1634008ed0 0x7f1634052550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f163800e9f0 tx=0x7f163800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677753f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67700d0bb0 0x7f67700cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3740e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d280333a0 0x7f1d280337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3740e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28006110 con 0x7f1d280df800 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3740e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28005f90 con 0x7f1d28053710 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3740e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1d28006290 con 0x7f1d280333a0 2024-09-21T05:39:33.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677753f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 0x7f67700cb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677753f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67700b4030 con 0x7f67700d4ee0 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677753f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67700b3eb0 con 0x7f67700d0bb0 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677753f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f67700b41b0 con 0x7f67700b9540 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8004220 con 0x7f85f40e6bc0 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3640c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28048890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3640c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28048890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54140/0 (socket says 172.21.15.22:54140) 2024-09-21T05:39:33.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d3640c640 1 -- 172.21.15.22:0/3698768647 learned_addr learned my addr 172.21.15.22:0/3698768647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677653d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67700b9540 0x7f677015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677653d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67700b9540 0x7f677015f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34222/0 (socket says 172.21.15.22:34222) 2024-09-21T05:39:33.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f677653d640 1 -- 172.21.15.22:0/4152786386 learned_addr learned my addr 172.21.15.22:0/4152786386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163800ccc0 con 0x7f1634008ed0 2024-09-21T05:39:33.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f8601a97640 1 -- 172.21.15.22:0/3361573812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f4050c90 con 0x7f85f40e6bc0 2024-09-21T05:39:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1634050bf0 con 0x7f1634008ed0 2024-09-21T05:39:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1634177d60 con 0x7f1634008ed0 2024-09-21T05:39:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d35c0b640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280df800 0x7f1d280d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec004b20 con 0x7fe8f4157060 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f43319640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c06f2a0 0x7f1f3c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f43319640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c06f6b0 0x7f1f3c147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f43319640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c15edd0 0x7f1f3c1f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f43319640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c0cbfe0 con 0x7f1f3c15edd0 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f43319640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c069e40 con 0x7f1f3c06f6b0 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f43319640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c0cc160 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f410b5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c06f2a0 0x7f1f3c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1638004020 con 0x7f1634008ed0 2024-09-21T05:39:33.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638004470 con 0x7f1634008ed0 2024-09-21T05:39:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f410b5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c06f2a0 0x7f1f3c0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34250/0 (socket says 172.21.15.22:34250) 2024-09-21T05:39:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f410b5640 1 -- 172.21.15.22:0/3439785882 learned_addr learned my addr 172.21.15.22:0/3439785882 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1d36c0d640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d280333a0 0x7f1d280337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f8601a97640 1 -- 172.21.15.22:0/3361573812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f4051250 con 0x7f85f40e6bc0 2024-09-21T05:39:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8ec004dd0 con 0x7fe8f4157060 2024-09-21T05:39:33.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f1f408b4640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c06f6b0 0x7f1f3c147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f4157d70 con 0x7fe8f4157060 2024-09-21T05:39:33.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f6775d3c640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67700d0bb0 0x7f67700cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f6776d3e640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 0x7f67700cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f41549d0 con 0x7fe8f4157060 2024-09-21T05:39:33.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85e8005020 con 0x7f85f40e6bc0 2024-09-21T05:39:33.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8005470 con 0x7f85f40e6bc0 2024-09-21T05:39:33.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec003bd0 con 0x7fe8f4157060 2024-09-21T05:39:33.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f586760a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5860070f20 0x7f58600c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f586760a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58600cf660 0x7f58601534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f586760a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58600c6cb0 0x7f5860153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f586760a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f58600b2fb0 con 0x7f5860070f20 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f586760a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f58600cfda0 con 0x7f58600c6cb0 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f586760a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f58600b3130 con 0x7f58600cf660 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f5864ba5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58600cf660 0x7f58601534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f5864ba5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58600cf660 0x7f58601534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34214/0 (socket says 172.21.15.22:34214) 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.412+0000 7f5864ba5640 1 -- 172.21.15.22:0/422941292 learned_addr learned my addr 172.21.15.22:0/422941292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f9e858af640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e8015a940 0x7f9e8015ad50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009b30 tx=0x7f9e78031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f5865ba7640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58600c6cb0 0x7f5860153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f58653a6640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5860070f20 0x7f58600c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f5864ba5640 1 -- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58600c6cb0 msgr2=0x7f5860153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f5864ba5640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f58600c6cb0 0x7f5860153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f5864ba5640 1 -- 172.21.15.22:0/422941292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5860070f20 msgr2=0x7f58600c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f5864ba5640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5860070f20 0x7f58600c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f5864ba5640 1 -- 172.21.15.22:0/422941292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586009e050 con 0x7f58600cf660 2024-09-21T05:39:33.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1624001030 con 0x7f1634008ed0 2024-09-21T05:39:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f58653a6640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5860070f20 0x7f58600c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f8601a97640 1 -- 172.21.15.22:0/3361573812 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85f40d60f0 con 0x7f85f40e6bc0 2024-09-21T05:39:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f1d3640c640 1 -- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d280333a0 msgr2=0x7f1d280337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f1d3640c640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d280333a0 0x7f1d280337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f1d3640c640 1 -- 172.21.15.22:0/3698768647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280df800 msgr2=0x7f1d280d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f1d3640c640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280df800 0x7f1d280d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f1d3640c640 1 -- 172.21.15.22:0/3698768647 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d28033cf0 con 0x7f1d28053710 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f1d3640c640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28048890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000c4d0 tx=0x7f1d2000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20012830 con 0x7f1d28053710 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d20017070 con 0x7f1d28053710 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20012d60 con 0x7f1d28053710 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f1d3740e640 1 -- 172.21.15.22:0/3698768647 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d28049310 con 0x7f1d28053710 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f1d3740e640 1 -- 172.21.15.22:0/3698768647 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d28049da0 con 0x7f1d28053710 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f677653d640 1 -- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67700d0bb0 msgr2=0x7f67700cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f677653d640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67700d0bb0 0x7f67700cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f677653d640 1 -- 172.21.15.22:0/4152786386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 msgr2=0x7f67700cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f677653d640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67700d4ee0 0x7f67700cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.413+0000 7f677653d640 1 -- 172.21.15.22:0/4152786386 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6770160280 con 0x7f67700b9540 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f677653d640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67700b9540 0x7f677015f7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f676c00c3c0 tx=0x7f676c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe8d4001030 con 0x7fe8f4157060 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe8ec01d020 con 0x7fe8f4157060 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78037040 con 0x7f9e8015a940 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f9e87312640 1 -- 172.21.15.22:0/1670685292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8014a430 con 0x7f9e8015a940 2024-09-21T05:39:33.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f9e87312640 1 -- 172.21.15.22:0/1670685292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e800cbd70 con 0x7f9e8015a940 2024-09-21T05:39:33.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e78043070 con 0x7f9e8015a940 2024-09-21T05:39:33.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78031bd0 con 0x7f9e8015a940 2024-09-21T05:39:33.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85e800c7e0 con 0x7f85f40e6bc0 2024-09-21T05:39:33.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f163803bd10 con 0x7f1634008ed0 2024-09-21T05:39:33.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f1632ffd640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1620078b90 0x7f162007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f5864ba5640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58600cf660 0x7f58601534f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5850032e20 tx=0x7f58500045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850005020 con 0x7f58600cf660 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58500052b0 con 0x7f58600cf660 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500047a0 con 0x7f58600cf660 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586009da90 con 0x7f58600cf660 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601e1c10 con 0x7f58600cf660 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f410b5640 1 -- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c06f6b0 msgr2=0x7f1f3c147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f410b5640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f3c06f6b0 0x7f1f3c147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f410b5640 1 -- 172.21.15.22:0/3439785882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c15edd0 msgr2=0x7f1f3c1f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f418b6640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c15edd0 0x7f1f3c1f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f410b5640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c15edd0 0x7f1f3c1f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f410b5640 1 -- 172.21.15.22:0/3439785882 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c0b4040 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f418b6640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f3c15edd0 0x7f1f3c1f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f1f410b5640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c06f2a0 0x7f1f3c0c9e80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800c440 tx=0x7f1f3800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38016020 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f380129e0 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c0b4320 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c011790 con 0x7f67700b9540 2024-09-21T05:39:33.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.414+0000 7f677753f640 1 -- 172.21.15.22:0/4152786386 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6770160560 con 0x7f67700b9540 2024-09-21T05:39:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.415+0000 7f677753f640 1 -- 172.21.15.22:0/4152786386 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6770160b20 con 0x7f67700b9540 2024-09-21T05:39:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.416+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f676c017070 con 0x7f67700b9540 2024-09-21T05:39:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c011c80 con 0x7f67700b9540 2024-09-21T05:39:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7fe8dbfff640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe8e0078c60 0x7fe8e007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe8ec09b1f0 con 0x7fe8f4157060 2024-09-21T05:39:33.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38012e10 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7fe8dbfff640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7fe8e0080f30 0x7fe8e0083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8e0083a50 con 0x7fe8e0080f30 2024-09-21T05:39:33.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe8ec0a2050 con 0x7fe8f4157060 2024-09-21T05:39:33.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c0b48e0 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7f1d3740e640 1 -- 172.21.15.22:0/3698768647 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d28049b90 con 0x7f1d28053710 2024-09-21T05:39:33.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f85f97fa640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85e0078c60 0x7f85e007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f85e80ac6f0 con 0x7f85f40e6bc0 2024-09-21T05:39:33.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58600cf010 con 0x7f58600cf660 2024-09-21T05:39:33.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.417+0000 7f85f97fa640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f85e0080f30 0x7f85e0083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7f85fbfff640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f85e0080f30 0x7f85e0083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.418+0000 7f8600a95640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85e0078c60 0x7f85e007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f3c0cd310 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9e78031d70 con 0x7f9e8015a940 2024-09-21T05:39:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6758001030 con 0x7f67700b9540 2024-09-21T05:39:33.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f163809b5a0 con 0x7f1634008ed0 2024-09-21T05:39:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f8600a95640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85e0078c60 0x7f85e007b170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007ac0 tx=0x7f85f0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f85fbfff640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f85e0080f30 0x7f85e0083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85e0083a50 con 0x7f85e0080f30 2024-09-21T05:39:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1632ffd640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7f1620080e60 0x7f16200832b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1620083980 con 0x7f1620080e60 2024-09-21T05:39:33.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f163809b990 con 0x7f1634008ed0 2024-09-21T05:39:33.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f38019020 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7fe8f3fff640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7fe8e0080f30 0x7fe8e0083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7fe8f37fe640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe8e0078c60 0x7fe8e007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1641d48640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1620078b90 0x7f162007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f676c01c460 con 0x7f67700b9540 2024-09-21T05:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f9e87312640 1 -- 172.21.15.22:0/1670685292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e60001030 con 0x7f9e8015a940 2024-09-21T05:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7fe8f3fff640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7fe8e0080f30 0x7fe8e0083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1f31ffb640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f20078c60 0x7f1f2007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f9e75ffb640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9e64078b90 0x7f9e6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1642d4a640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7f1620080e60 0x7f16200832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f3809a3b0 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1641d48640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1620078b90 0x7f162007b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f163c06b8e0 tx=0x7f163c070040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9e780bb750 con 0x7f9e8015a940 2024-09-21T05:39:33.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1f408b4640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f20078c60 0x7f1f2007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1f31ffb640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7f1f20080ed0 0x7f1f20083320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f200839f0 con 0x7f1f20080ed0 2024-09-21T05:39:33.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1f380a1050 con 0x7f1f3c06f2a0 2024-09-21T05:39:33.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f9e75ffb640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7f9e64080e00 0x7f9e64083250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f9e850ae640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9e64078b90 0x7f9e6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e78044ec0 con 0x7f9e64080e00 2024-09-21T05:39:33.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6766ffd640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6754078c60 0x7f675407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9e780c2050 con 0x7f9e8015a940 2024-09-21T05:39:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f676c09a9a0 con 0x7f67700b9540 2024-09-21T05:39:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f9e848ad640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7f9e64080e00 0x7f9e64083250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.419+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d2001b410 con 0x7f1d28053710 2024-09-21T05:39:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f1d26ffd640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d14078c90 0x7f1d1407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1d2009a540 con 0x7f1d28053710 2024-09-21T05:39:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1d26ffd640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7f1d14080ee0 0x7f1d14083330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d2001bbe0 con 0x7f1d14080ee0 2024-09-21T05:39:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f85e80acae0 con 0x7f85f40e6bc0 2024-09-21T05:39:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.420+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85e0083a50 con 0x7f85e0080f30 2024-09-21T05:39:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6766ffd640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f6754080f30 0x7f6754083380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1d200a1050 con 0x7f1d28053710 2024-09-21T05:39:33.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1642d4a640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7f1620080e60 0x7f16200832b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1d35c0b640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d14078c90 0x7f1d1407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6775d3c640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6754078c60 0x7f675407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f676c01cc30 con 0x7f6754080f30 2024-09-21T05:39:33.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f676c09f2a0 con 0x7f67700b9540 2024-09-21T05:39:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1d36c0d640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7f1d14080ee0 0x7f1d14083330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f1f418b6640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7f1f20080ed0 0x7f1f20083320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f6776d3e640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f6754080f30 0x7f6754083380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8e0083a50 con 0x7fe8e0080f30 2024-09-21T05:39:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f9e848ad640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7f9e64080e00 0x7f9e64083250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f1f408b4640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f20078c60 0x7f1f2007b170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c035000 tx=0x7f1f2c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f1f418b6640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7f1f20080ed0 0x7f1f20083320 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f9e850ae640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9e64078b90 0x7f9e6407b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009ac0 tx=0x7f9e7c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7fe8f37fe640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe8e0078c60 0x7fe8e007b170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4033000 tx=0x7fe8f40c7480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.421+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5850041070 con 0x7f58600cf660 2024-09-21T05:39:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f5855ffb640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5844078be0 0x7f584407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1620083980 con 0x7f1620080e60 2024-09-21T05:39:33.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f58653a6640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5844078be0 0x7f584407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f6775d3c640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6754078c60 0x7f675407b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f6760004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f58500056c0 con 0x7f58600cf660 2024-09-21T05:39:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f1d36c0d640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7f1d14080ee0 0x7f1d14083330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.422+0000 7f5855ffb640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f5844081a60 0x7f5844083f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5844084620 con 0x7f5844081a60 2024-09-21T05:39:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f1d35c0b640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d14078c90 0x7f1d1407b1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c005fd0 tx=0x7f1d2c005da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f585007ee30 con 0x7f58600cf660 2024-09-21T05:39:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f5865ba7640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f5844081a60 0x7f5844083f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f6776d3e640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f6754080f30 0x7f6754083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f58653a6640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5844078be0 0x7f584407b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5860072600 tx=0x7f5858009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e78044ec0 con 0x7f9e64080e00 2024-09-21T05:39:33.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f5865ba7640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f5844081a60 0x7f5844083f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f200839f0 con 0x7f1f20080ed0 2024-09-21T05:39:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.423+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d2001bbe0 con 0x7f1d14080ee0 2024-09-21T05:39:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.425+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5844084620 con 0x7f5844081a60 2024-09-21T05:39:33.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.425+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f676c01cc30 con 0x7f6754080f30 2024-09-21T05:39:33.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.447+0000 7f9e87312640 1 -- 172.21.15.22:0/1670685292 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e60002d50 con 0x7f9e64080e00 2024-09-21T05:39:33.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.448+0000 7f9e75ffb640 1 -- 172.21.15.22:0/1670685292 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9e60002d50 con 0x7f9e64080e00 2024-09-21T05:39:33.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.448+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7f9e64080e00 msgr2=0x7f9e64083250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.448+0000 7f9e6b7fe640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7f9e64080e00 0x7f9e64083250 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9e64078b90 msgr2=0x7f9e6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9e64078b90 0x7f9e6407b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009ac0 tx=0x7f9e7c005f90 comp rx=0 tx=0).stop 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e8015a940 msgr2=0x7f9e8015ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 --2- 172.21.15.22:0/1670685292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9e8015a940 0x7f9e8015ad50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009b30 tx=0x7f9e78031600 comp rx=0 tx=0).stop 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e858af640 1 -- 172.21.15.22:0/1670685292 reap_dead start 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 shutdown_connections 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 >> 172.21.15.22:0/1670685292 conn(0x7f9e80074170 msgr2=0x7f9e8015c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 shutdown_connections 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.450+0000 7f9e6b7fe640 1 -- 172.21.15.22:0/1670685292 wait complete. 2024-09-21T05:39:33.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.453+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe8d4002dd0 con 0x7fe8e0080f30 2024-09-21T05:39:33.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.453+0000 7f8601a97640 1 -- 172.21.15.22:0/3361573812 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85f4050a40 con 0x7f85e0080f30 2024-09-21T05:39:33.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.453+0000 7f1d3740e640 1 -- 172.21.15.22:0/3698768647 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d28007c60 con 0x7f1d14080ee0 2024-09-21T05:39:33.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.453+0000 7fe8dbfff640 1 -- 172.21.15.22:0/739487570 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe8d4002dd0 con 0x7fe8e0080f30 2024-09-21T05:39:33.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.453+0000 7f85f97fa640 1 -- 172.21.15.22:0/3361573812 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f85f4050a40 con 0x7f85e0080f30 2024-09-21T05:39:33.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f1d26ffd640 1 -- 172.21.15.22:0/3698768647 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1d28007c60 con 0x7f1d14080ee0 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f85e0080f30 msgr2=0x7f85e0083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f85e6ffd640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f85e0080f30 0x7f85e0083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85e0078c60 msgr2=0x7f85e007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f85e6ffd640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85e0078c60 0x7f85e007b170 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f85f4007ac0 tx=0x7f85f0008040 comp rx=0 tx=0).stop 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f40e6bc0 msgr2=0x7f85f40dac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f85e6ffd640 1 --2- 172.21.15.22:0/3361573812 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f85f40e6bc0 0x7f85f40dac00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fc0 tx=0x7f85e800ead0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7fe8e0080f30 msgr2=0x7fe8e0083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7fe8fa212640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7fe8e0080f30 0x7fe8e0083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7f1d14080ee0 msgr2=0x7f1d14083330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.454+0000 7f1d24ff9640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7f1d14080ee0 0x7f1d14083330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f8601296640 1 -- 172.21.15.22:0/3361573812 reap_dead start 2024-09-21T05:39:33.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 shutdown_connections 2024-09-21T05:39:33.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 >> 172.21.15.22:0/3361573812 conn(0x7f85f4055e60 msgr2=0x7f85f40e3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d14078c90 msgr2=0x7f1d1407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d14078c90 0x7f1d1407b1a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c005fd0 tx=0x7f1d2c005da0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 msgr2=0x7f1d28048890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 --2- 172.21.15.22:0/3698768647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d28053710 0x7f1d28048890 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000c4d0 tx=0x7f1d2000c990 comp rx=0 tx=0).stop 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d36c0d640 1 -- 172.21.15.22:0/3698768647 reap_dead start 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 shutdown_connections 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 >> 172.21.15.22:0/3698768647 conn(0x7f1d2801e260 msgr2=0x7f1d280067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe8e0078c60 msgr2=0x7fe8e007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8fa212640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe8e0078c60 0x7fe8e007b170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4033000 tx=0x7fe8f40c7480 comp rx=0 tx=0).stop 2024-09-21T05:39:33.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8f4157060 msgr2=0x7fe8f414c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8fa212640 1 --2- 172.21.15.22:0/739487570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe8f4157060 0x7fe8f414c5e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00efc0 tx=0x7fe8ec00c800 comp rx=0 tx=0).stop 2024-09-21T05:39:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8f3fff640 1 -- 172.21.15.22:0/739487570 reap_dead start 2024-09-21T05:39:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f3c15e630 con 0x7f1f20080ed0 2024-09-21T05:39:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 shutdown_connections 2024-09-21T05:39:33.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f1d24ff9640 1 -- 172.21.15.22:0/3698768647 wait complete. 2024-09-21T05:39:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 shutdown_connections 2024-09-21T05:39:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7f85e6ffd640 1 -- 172.21.15.22:0/3361573812 wait complete. 2024-09-21T05:39:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 shutdown_connections 2024-09-21T05:39:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.455+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 >> 172.21.15.22:0/739487570 conn(0x7fe8f40cd480 msgr2=0x7fe8f4070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 shutdown_connections 2024-09-21T05:39:33.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1624002dd0 con 0x7f1620080e60 2024-09-21T05:39:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6758002dd0 con 0x7f6754080f30 2024-09-21T05:39:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7fe8fa212640 1 -- 172.21.15.22:0/739487570 wait complete. 2024-09-21T05:39:33.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f1f31ffb640 1 -- 172.21.15.22:0/3439785882 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f3c15e630 con 0x7f1f20080ed0 2024-09-21T05:39:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f1632ffd640 1 -- 172.21.15.22:0/1495464910 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1624002dd0 con 0x7f1620080e60 2024-09-21T05:39:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7f1f20080ed0 msgr2=0x7f1f20083320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f1f43319640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7f1f20080ed0 0x7f1f20083320 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f20078c60 msgr2=0x7f1f2007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f20078c60 0x7f1f2007b170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c035000 tx=0x7f1f2c004730 comp rx=0 tx=0).stop 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.456+0000 7f6766ffd640 1 -- 172.21.15.22:0/4152786386 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6758002dd0 con 0x7f6754080f30 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c06f2a0 msgr2=0x7f1f3c0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 --2- 172.21.15.22:0/3439785882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f3c06f2a0 0x7f1f3c0c9e80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1f3800c440 tx=0x7f1f3800c900 comp rx=0 tx=0).stop 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f6754080f30 msgr2=0x7f6754083380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f6754080f30 0x7f6754083380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6754078c60 msgr2=0x7f675407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6754078c60 0x7f675407b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6760034000 tx=0x7f6760004730 comp rx=0 tx=0).stop 2024-09-21T05:39:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67700b9540 msgr2=0x7f677015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 --2- 172.21.15.22:0/4152786386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67700b9540 0x7f677015f7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f676c00c3c0 tx=0x7f676c00c880 comp rx=0 tx=0).stop 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6776d3e640 1 -- 172.21.15.22:0/4152786386 reap_dead start 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f418b6640 1 -- 172.21.15.22:0/3439785882 reap_dead start 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 shutdown_connections 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 >> 172.21.15.22:0/3439785882 conn(0x7f1f3c0b21d0 msgr2=0x7f1f3c156000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 shutdown_connections 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f1f43319640 1 -- 172.21.15.22:0/3439785882 wait complete. 2024-09-21T05:39:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 shutdown_connections 2024-09-21T05:39:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 >> 172.21.15.22:0/4152786386 conn(0x7f677009d680 msgr2=0x7f67700b8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.457+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 shutdown_connections 2024-09-21T05:39:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f6764ff9640 1 -- 172.21.15.22:0/4152786386 wait complete. 2024-09-21T05:39:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58600aea90 con 0x7f5844081a60 2024-09-21T05:39:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7f1620080e60 msgr2=0x7f16200832b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f16447ad640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7f1620080e60 0x7f16200832b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1620078b90 msgr2=0x7f162007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f16447ad640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1620078b90 0x7f162007b0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f163c06b8e0 tx=0x7f163c070040 comp rx=0 tx=0).stop 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1634008ed0 msgr2=0x7f1634052550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.458+0000 7f16447ad640 1 --2- 172.21.15.22:0/1495464910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1634008ed0 0x7f1634052550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f163800e9f0 tx=0x7f163800eeb0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.459+0000 7f1642d4a640 1 -- 172.21.15.22:0/1495464910 reap_dead start 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.459+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 shutdown_connections 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.459+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 >> 172.21.15.22:0/1495464910 conn(0x7f163401eb10 msgr2=0x7f16340328a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.459+0000 7f5855ffb640 1 -- 172.21.15.22:0/422941292 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f58600aea90 con 0x7f5844081a60 2024-09-21T05:39:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.459+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 shutdown_connections 2024-09-21T05:39:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.459+0000 7f16447ad640 1 -- 172.21.15.22:0/1495464910 wait complete. 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.460+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f5844081a60 msgr2=0x7f5844083f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.460+0000 7f586760a640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f5844081a60 0x7f5844083f50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5844078be0 msgr2=0x7f584407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5844078be0 0x7f584407b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5860072600 tx=0x7f5858009500 comp rx=0 tx=0).stop 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58600cf660 msgr2=0x7f58601534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 --2- 172.21.15.22:0/422941292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f58600cf660 0x7f58601534f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5850032e20 tx=0x7f58500045e0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f5865ba7640 1 -- 172.21.15.22:0/422941292 reap_dead start 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 shutdown_connections 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 >> 172.21.15.22:0/422941292 conn(0x7f58600b1cb0 msgr2=0x7f58600b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 shutdown_connections 2024-09-21T05:39:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.461+0000 7f586760a640 1 -- 172.21.15.22:0/422941292 wait complete. 2024-09-21T05:39:33.485 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:33.486 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T05:39:33.486 INFO:teuthology.orchestra.run.smithi022.stdout:64424509448 2024-09-21T05:39:33.487 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T05:39:33.487 INFO:teuthology.orchestra.run.smithi022.stdout:60129542153 2024-09-21T05:39:33.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T05:39:33.488 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:33.488 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T05:39:33.490 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:33.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T05:39:33.491 INFO:teuthology.orchestra.run.smithi022.stdout:64424509447 2024-09-21T05:39:33.491 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T05:39:33.492 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-09-21T05:39:33.492 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:33.492 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T05:39:33.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T05:39:33.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.757+0000 7f08bc9e7640 1 -- 172.21.15.22:0/2675917597 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80adc70 con 0x7f08b00e7410 2024-09-21T05:39:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 -- 172.21.15.22:0/2675917597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b00e7410 msgr2=0x7f08b0006320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 --2- 172.21.15.22:0/2675917597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b00e7410 0x7f08b0006320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f08b8051230 tx=0x7f08b80a66e0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 -- 172.21.15.22:0/2675917597 shutdown_connections 2024-09-21T05:39:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 --2- 172.21.15.22:0/2675917597 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b00e7410 0x7f08b0006320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 --2- 172.21.15.22:0/2675917597 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7000 0x7f08b0005de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 --2- 172.21.15.22:0/2675917597 >> v2:172.21.15.134:3300/0 conn(0x7f08b0052360 0x7f08b00e6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.758+0000 7f08be9eb640 1 -- 172.21.15.22:0/2675917597 >> 172.21.15.22:0/2675917597 conn(0x7f08b001d9a0 msgr2=0x7f08b0039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f08be9eb640 1 -- 172.21.15.22:0/2675917597 shutdown_connections 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f9fafda8640 1 -- 172.21.15.22:0/438425102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 msgr2=0x7f9fa80cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f9fafda8640 1 --2- 172.21.15.22:0/438425102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80cf6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009830 tx=0x7f9fa402a510 comp rx=0 tx=0).stop 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f9fafda8640 1 -- 172.21.15.22:0/438425102 shutdown_connections 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f9fafda8640 1 --2- 172.21.15.22:0/438425102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fa80cfc10 0x7f9fa8072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f9fafda8640 1 --2- 172.21.15.22:0/438425102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80cf6d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f9fafda8640 1 -- 172.21.15.22:0/438425102 >> 172.21.15.22:0/438425102 conn(0x7f9fa80b20a0 msgr2=0x7f9fa80b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.760+0000 7f9fafda8640 1 -- 172.21.15.22:0/438425102 shutdown_connections 2024-09-21T05:39:33.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.760+0000 7f9fafda8640 1 -- 172.21.15.22:0/438425102 wait complete. 2024-09-21T05:39:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.760+0000 7fad2db62640 1 -- 172.21.15.22:0/2730106550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d30c0 msgr2=0x7fad280b98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.760+0000 7fad2db62640 1 --2- 172.21.15.22:0/2730106550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d30c0 0x7fad280b98c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009830 tx=0x7fad1c02a560 comp rx=0 tx=0).stop 2024-09-21T05:39:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.760+0000 7f8b70677640 1 -- 172.21.15.22:0/2589619268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b681624d0 msgr2=0x7f8b681628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7f8b70677640 1 --2- 172.21.15.22:0/2589619268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b681624d0 0x7f8b681628e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b64009830 tx=0x7f8b6402a4d0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7f8b70677640 1 -- 172.21.15.22:0/2589619268 shutdown_connections 2024-09-21T05:39:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7fad2db62640 1 -- 172.21.15.22:0/2730106550 shutdown_connections 2024-09-21T05:39:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7f8b70677640 1 --2- 172.21.15.22:0/2589619268 >> v2:172.21.15.134:3300/0 conn(0x7f8b680ccde0 0x7f8b680cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7fad2db62640 1 --2- 172.21.15.22:0/2730106550 >> v2:172.21.15.22:3300/0 conn(0x7fad280d34d0 0x7fad280b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7fad2db62640 1 --2- 172.21.15.22:0/2730106550 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d30c0 0x7fad280b98c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7fad2db62640 1 -- 172.21.15.22:0/2730106550 >> 172.21.15.22:0/2730106550 conn(0x7fad2809d7f0 msgr2=0x7fad280d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7f8b70677640 1 --2- 172.21.15.22:0/2589619268 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b681624d0 0x7f8b681628e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.761+0000 7f8b70677640 1 -- 172.21.15.22:0/2589619268 >> 172.21.15.22:0/2589619268 conn(0x7f8b680b20a0 msgr2=0x7f8b680b24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.759+0000 7f08be9eb640 1 -- 172.21.15.22:0/2675917597 wait complete. 2024-09-21T05:39:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.762+0000 7fad2db62640 1 -- 172.21.15.22:0/2730106550 shutdown_connections 2024-09-21T05:39:33.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.762+0000 7fad2db62640 1 -- 172.21.15.22:0/2730106550 wait complete. 2024-09-21T05:39:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.762+0000 7f8b70677640 1 -- 172.21.15.22:0/2589619268 shutdown_connections 2024-09-21T05:39:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.762+0000 7f8b70677640 1 -- 172.21.15.22:0/2589619268 wait complete. 2024-09-21T05:39:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.762+0000 7f08be9eb640 1 Processor -- start 2024-09-21T05:39:33.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.762+0000 7f08be9eb640 1 -- start start 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3686446085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88009ba0 msgr2=0x7fdd880df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3686446085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88009ba0 0x7fdd880df180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800dad0 tx=0x7fdd8002b860 comp rx=0 tx=0).stop 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3686446085 shutdown_connections 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3686446085 >> v2:172.21.15.22:3300/0 conn(0x7fdd88005c60 0x7fdd8801ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3686446085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88009ba0 0x7fdd880df180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3686446085 >> 172.21.15.22:0/3686446085 conn(0x7fdd88051420 msgr2=0x7fdd88051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/2542193974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 msgr2=0x7f6cb4036430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.763+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/2542193974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4036430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc06af90 tx=0x7f6cbc0a6690 comp rx=0 tx=0).stop 2024-09-21T05:39:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/2542193974 shutdown_connections 2024-09-21T05:39:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/2542193974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4036430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/2542193974 >> 172.21.15.22:0/2542193974 conn(0x7f6cb4051450 msgr2=0x7f6cb4051890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7f9fafda8640 1 Processor -- start 2024-09-21T05:39:33.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3686446085 shutdown_connections 2024-09-21T05:39:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/2542193974 shutdown_connections 2024-09-21T05:39:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3686446085 wait complete. 2024-09-21T05:39:33.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/2542193974 wait complete. 2024-09-21T05:39:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 -- 172.21.15.22:0/3494258262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 msgr2=0x7fc650154e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 --2- 172.21.15.22:0/3494258262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650154e20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc640005fd0 tx=0x7fc64002b250 comp rx=0 tx=0).stop 2024-09-21T05:39:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 -- 172.21.15.22:0/3494258262 shutdown_connections 2024-09-21T05:39:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 --2- 172.21.15.22:0/3494258262 >> v2:172.21.15.134:3300/0 conn(0x7fc6500cf040 0x7fc6500cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 --2- 172.21.15.22:0/3494258262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650154e20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 -- 172.21.15.22:0/3494258262 >> 172.21.15.22:0/3494258262 conn(0x7fc6500b21d0 msgr2=0x7fc6500b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 -- 172.21.15.22:0/3494258262 shutdown_connections 2024-09-21T05:39:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.764+0000 7fc65558b640 1 -- 172.21.15.22:0/3494258262 wait complete. 2024-09-21T05:39:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08be9eb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 0x7f08b00dd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7fad2db62640 1 Processor -- start 2024-09-21T05:39:33.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08be9eb640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00e7000 0x7f08b004fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 -- 172.21.15.22:0/3837006917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc0d3340 msgr2=0x7f20dc06eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 --2- 172.21.15.22:0/3837006917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc0d3340 0x7f20dc06eaf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009830 tx=0x7f20c402a480 comp rx=0 tx=0).stop 2024-09-21T05:39:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08be9eb640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7410 0x7f08b00e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08bd9e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 0x7f08b00dd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7fad2db62640 1 -- start start 2024-09-21T05:39:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08bd9e9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 0x7f08b00dd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46870/0 (socket says 172.21.15.22:46870) 2024-09-21T05:39:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08bd9e9640 1 -- 172.21.15.22:0/1630598993 learned_addr learned my addr 172.21.15.22:0/1630598993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f8b70677640 1 Processor -- start 2024-09-21T05:39:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 -- 172.21.15.22:0/3837006917 shutdown_connections 2024-09-21T05:39:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 --2- 172.21.15.22:0/3837006917 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc0d3340 0x7f20dc06eaf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 -- 172.21.15.22:0/3837006917 >> 172.21.15.22:0/3837006917 conn(0x7f20dc09d8c0 msgr2=0x7f20dc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:33.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 -- 172.21.15.22:0/3837006917 shutdown_connections 2024-09-21T05:39:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00e3590 con 0x7f08b0052360 2024-09-21T05:39:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08b00e3410 con 0x7f08b00e7000 2024-09-21T05:39:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f8b70677640 1 -- start start 2024-09-21T05:39:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08b0054720 con 0x7f08b00e7410 2024-09-21T05:39:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f9fafda8640 1 -- start start 2024-09-21T05:39:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.765+0000 7f20e08fa640 1 -- 172.21.15.22:0/3837006917 wait complete. 2024-09-21T05:39:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd1e8640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00e7000 0x7f08b004fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08be1ea640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7410 0x7f08b00e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd9e9640 1 -- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7410 msgr2=0x7f08b00e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd9e9640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7410 0x7f08b00e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd9e9640 1 -- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00e7000 msgr2=0x7f08b004fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd9e9640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00e7000 0x7f08b004fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd9e9640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b00e64c0 con 0x7f08b0052360 2024-09-21T05:39:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7fc65558b640 1 Processor -- start 2024-09-21T05:39:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08be1ea640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7410 0x7f08b00e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:33.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7fc65558b640 1 -- start start 2024-09-21T05:39:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7fdd96a1c640 1 Processor -- start 2024-09-21T05:39:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7fdd96a1c640 1 -- start start 2024-09-21T05:39:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad2db62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 0x7fad2815f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f6cc0cd5640 1 Processor -- start 2024-09-21T05:39:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.767+0000 7f08bd9e9640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 0x7f08b00dd5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f08a800d690 tx=0x7f08a800db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad2db62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad280d34d0 0x7fad280c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f6cc0cd5640 1 -- start start 2024-09-21T05:39:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad2db62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d4ca0 0x7fad280c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8017070 con 0x7f08b0052360 2024-09-21T05:39:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a8009b10 con 0x7f08b0052360 2024-09-21T05:39:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8010460 con 0x7f08b0052360 2024-09-21T05:39:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad277fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 0x7fad2815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b004ee60 con 0x7f08b0052360 2024-09-21T05:39:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad277fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 0x7fad2815f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54222/0 (socket says 172.21.15.22:54222) 2024-09-21T05:39:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f8b70677640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b680ccde0 0x7f8b681515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b004f420 con 0x7f08b0052360 2024-09-21T05:39:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad277fe640 1 -- 172.21.15.22:0/2255466298 learned_addr learned my addr 172.21.15.22:0/2255466298 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad27fff640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d4ca0 0x7fad280c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad2db62640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad280b6870 con 0x7fad280d34d0 2024-09-21T05:39:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad2db62640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad280b66f0 con 0x7fad280d30c0 2024-09-21T05:39:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.768+0000 7fad2db62640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad2806dea0 con 0x7fad280d4ca0 2024-09-21T05:39:33.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7fad26ffd640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad280d34d0 0x7fad280c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f8b70677640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6815f9a0 0x7f8b680b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f8b70677640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 0x7f8b6809b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f9fafda8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f8b70677640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6815c120 con 0x7f8b6815f9a0 2024-09-21T05:39:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f9fafda8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fa808b690 0x7f9fa808a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f8b70677640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6815bfa0 con 0x7f8b681624d0 2024-09-21T05:39:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f9fafda8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 0x7f9fa8203930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f8b70677640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6815c2a0 con 0x7f8b680ccde0 2024-09-21T05:39:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f9fadb44640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7fad277fe640 1 -- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d4ca0 msgr2=0x7fad280c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7fad277fe640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d4ca0 0x7fad280c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7fc65558b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc650073260 0x7fc6500b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7fad277fe640 1 -- 172.21.15.22:0/2255466298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad280d34d0 msgr2=0x7fad280c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7fad277fe640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad280d34d0 0x7fad280c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f8b6ec14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 0x7f8b6809b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fad277fe640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad281fd810 con 0x7fad280d30c0 2024-09-21T05:39:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc65558b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f20e08fa640 1 Processor -- start 2024-09-21T05:39:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f20e08fa640 1 -- start start 2024-09-21T05:39:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.769+0000 7f9fae345640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 0x7f9fa8203930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fae345640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 0x7f9fa8203930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46882/0 (socket says 172.21.15.22:46882) 2024-09-21T05:39:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fadb44640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54230/0 (socket says 172.21.15.22:54230) 2024-09-21T05:39:33.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc65558b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc650147e70 0x7fc65015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fae345640 1 -- 172.21.15.22:0/3553338987 learned_addr learned my addr 172.21.15.22:0/3553338987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f8b6ec14640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 0x7f8b6809b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54224/0 (socket says 172.21.15.22:54224) 2024-09-21T05:39:33.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f8b6ec14640 1 -- 172.21.15.22:0/794945431 learned_addr learned my addr 172.21.15.22:0/794945431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fafda8640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80cb180 con 0x7f9fa80cfc10 2024-09-21T05:39:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fafda8640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa80cd750 con 0x7f9fa808b140 2024-09-21T05:39:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fafda8640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa80cb300 con 0x7f9fa808b690 2024-09-21T05:39:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fad27fff640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d4ca0 0x7fad280c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fdd96a1c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 0x7fdd88031f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64e7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc65558b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500cbea0 con 0x7fc650147e70 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc65558b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6500cbd20 con 0x7fc650073260 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc65558b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6500cc020 con 0x7fc65015bb20 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64e7fc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34348/0 (socket says 172.21.15.22:34348) 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64e7fc640 1 -- 172.21.15.22:0/1292633918 learned_addr learned my addr 172.21.15.22:0/1292633918 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64effd640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc650073260 0x7fc6500b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fdd96a1c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd88009ba0 0x7fdd880e1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64f7fe640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc650147e70 0x7fc65015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fdd95a1a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 0x7fdd88031f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f6cc0cd5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4032400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fad277fe640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 0x7fad2815f020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c7a0 tx=0x7fad2000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fdd95a1a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 0x7fdd88031f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54262/0 (socket says 172.21.15.22:54262) 2024-09-21T05:39:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fad26ffd640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad280d34d0 0x7fad280c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:39:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64effd640 1 -- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 msgr2=0x7fc650147070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64effd640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650147070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64effd640 1 -- 172.21.15.22:0/1292633918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc650147e70 msgr2=0x7fc65015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64effd640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc650147e70 0x7fc65015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7fc64effd640 1 -- 172.21.15.22:0/1292633918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc65015d360 con 0x7fc650073260 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd95a1a640 1 -- 172.21.15.22:0/3316204122 learned_addr learned my addr 172.21.15.22:0/3316204122 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd8800bc30 0x7fdd880e1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f8b6e413640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b680ccde0 0x7f8b681515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8801d380 con 0x7fdd88009ba0 2024-09-21T05:39:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fad343640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fa808b690 0x7f9fa808a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fae345640 1 -- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fa808b690 msgr2=0x7f9fa808a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fae345640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fa808b690 0x7f9fa808a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fae345640 1 -- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 msgr2=0x7f9fa80b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.770+0000 7f9fae345640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f8b6ec14640 1 -- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b680ccde0 msgr2=0x7f8b681515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f8b6ec14640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b680ccde0 0x7f8b681515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f8b6ec14640 1 -- 172.21.15.22:0/794945431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6815f9a0 msgr2=0x7f8b680b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fc64effd640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc650073260 0x7fc6500b3cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6500cb4d0 tx=0x7fc640003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f9fae345640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa80b38b0 con 0x7f9fa80cfc10 2024-09-21T05:39:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f9fae345640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 0x7f9fa8203930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000cae0 tx=0x7f9fa000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f9fadb44640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:33.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f8b6ec14640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6815f9a0 0x7f8b680b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f8b6ec14640 1 -- 172.21.15.22:0/794945431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b68151af0 con 0x7f8b681624d0 2024-09-21T05:39:33.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd9621b640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd8800bc30 0x7fdd880e1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20017070 con 0x7fad280d30c0 2024-09-21T05:39:33.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640003e20 con 0x7fc650073260 2024-09-21T05:39:33.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd95219640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd88009ba0 0x7fdd880e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad20005070 con 0x7fad280d30c0 2024-09-21T05:39:33.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fad2db62640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad281fdaf0 con 0x7fad280d30c0 2024-09-21T05:39:33.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8801d200 con 0x7fdd88005c60 2024-09-21T05:39:33.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdd88036430 con 0x7fdd8800bc30 2024-09-21T05:39:33.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fc65558b640 1 -- 172.21.15.22:0/1292633918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc65015d640 con 0x7fc650073260 2024-09-21T05:39:33.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cc0cd5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb4009bd0 0x7f6cb4030aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cc0cd5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cb400bc60 0x7f6cb40d4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cc0cd5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4036d00 con 0x7f6cb4005c90 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cc0cd5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4036b80 con 0x7f6cb4009bd0 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cc0cd5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb401c5e0 con 0x7f6cb400bc60 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f8b6ec14640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 0x7f8b6809b6b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ea80 tx=0x7f8b6000ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cba59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4032400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cba59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4032400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46908/0 (socket says 172.21.15.22:46908) 2024-09-21T05:39:33.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f6cba59c640 1 -- 172.21.15.22:0/1159052676 learned_addr learned my addr 172.21.15.22:0/1159052676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb4009bd0 0x7f6cb4030aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7fc65558b640 1 -- 172.21.15.22:0/1292633918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc65015bf50 con 0x7fc650073260 2024-09-21T05:39:33.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7fad2db62640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281fe0b0 con 0x7fad280d30c0 2024-09-21T05:39:33.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20005500 con 0x7fad280d30c0 2024-09-21T05:39:33.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cbad9d640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cb400bc60 0x7f6cb40d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7f20e08fa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc06e6e0 0x7f20dc15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20e08fa640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20dc0b96d0 0x7f20dc159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20e08fa640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20dc0d3340 0x7f20dc0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 -- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cb400bc60 msgr2=0x7f6cb40d4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0016020 con 0x7f9fa80cfc10 2024-09-21T05:39:33.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cb400bc60 0x7f6cb40d4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f9fafda8640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa80b3b90 con 0x7f9fa80cfc10 2024-09-21T05:39:33.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.771+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc64003e970 con 0x7fc650073260 2024-09-21T05:39:33.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64003edc0 con 0x7fc650073260 2024-09-21T05:39:33.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f9fafda8640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa80b4150 con 0x7f9fa80cfc10 2024-09-21T05:39:33.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 -- 172.21.15.22:0/1159052676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 msgr2=0x7f6cb4032400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4032400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 -- 172.21.15.22:0/1159052676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb4175f50 con 0x7f6cb4009bd0 2024-09-21T05:39:33.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cba59c640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4032400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:33.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f6cb9d9b640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb4009bd0 0x7f6cb4030aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c3c0 tx=0x7f6cb000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b6000ce40 con 0x7f8b681624d0 2024-09-21T05:39:33.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6815e7d0 con 0x7f8b681624d0 2024-09-21T05:39:33.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b00e6ac0 con 0x7f08b0052360 2024-09-21T05:39:33.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.773+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6809aeb0 con 0x7f8b681624d0 2024-09-21T05:39:33.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.773+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b60004b20 con 0x7f8b681624d0 2024-09-21T05:39:33.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.773+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b60018470 con 0x7f8b681624d0 2024-09-21T05:39:33.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.773+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0017070 con 0x7f6cb4009bd0 2024-09-21T05:39:33.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20e08fa640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc16e3f0 con 0x7f20dc06e6e0 2024-09-21T05:39:33.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.773+0000 7fc65558b640 1 -- 172.21.15.22:0/1292633918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6500cb7e0 con 0x7fc650073260 2024-09-21T05:39:33.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20e08fa640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc16e270 con 0x7f20dc0b96d0 2024-09-21T05:39:33.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20e08fa640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f20dc16e570 con 0x7f20dc0d3340 2024-09-21T05:39:33.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20dad9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20dc0d3340 0x7f20dc0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20dad9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20dc0d3340 0x7f20dc0b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:34370/0 (socket says 172.21.15.22:34370) 2024-09-21T05:39:33.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20dad9d640 1 -- 172.21.15.22:0/1015110312 learned_addr learned my addr 172.21.15.22:0/1015110312 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:33.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.772+0000 7f20da59c640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc06e6e0 0x7f20dc15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.773+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad280b98c0 con 0x7fad280d30c0 2024-09-21T05:39:33.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f20da59c640 1 -- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20dc0d3340 msgr2=0x7f20dc0b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f20da59c640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20dc0d3340 0x7f20dc0b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cb00053e0 con 0x7f6cb4009bd0 2024-09-21T05:39:33.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f20da59c640 1 -- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20dc0b96d0 msgr2=0x7f20dc159960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T05:39:33.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fa000acd0 con 0x7f9fa80cfc10 2024-09-21T05:39:33.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0012680 con 0x7f9fa80cfc10 2024-09-21T05:39:33.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0003850 con 0x7f6cb4009bd0 2024-09-21T05:39:33.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb402d200 con 0x7f6cb4009bd0 2024-09-21T05:39:33.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb402cf20 con 0x7f6cb4009bd0 2024-09-21T05:39:33.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f20da59c640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20dc0b96d0 0x7f20dc159960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f20da59c640 1 -- 172.21.15.22:0/1015110312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc153570 con 0x7f20dc06e6e0 2024-09-21T05:39:33.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc640046030 con 0x7fc650073260 2024-09-21T05:39:33.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fc6377fe640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc63c078b90 0x7fc63c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7f20da59c640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc06e6e0 0x7f20dc15d1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c410 tx=0x7f20d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc6400c2990 con 0x7fc650073260 2024-09-21T05:39:33.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fc64e7fc640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc63c078b90 0x7fc63c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fdd95a1a640 1 -- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd8800bc30 msgr2=0x7fdd880e1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fdd95a1a640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd8800bc30 0x7fdd880e1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fdd95a1a640 1 -- 172.21.15.22:0/3316204122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd88009ba0 msgr2=0x7fdd880e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.775+0000 7fdd95a1a640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd88009ba0 0x7fdd880e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7fdd95a1a640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd88032460 con 0x7fdd88005c60 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b50005290 con 0x7f8b681624d0 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.774+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08a801a070 con 0x7f08b0052360 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f08ae7fc640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f089c078c90 0x7f089c07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f08a809ae30 con 0x7f08b0052360 2024-09-21T05:39:33.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b60018610 con 0x7f8b681624d0 2024-09-21T05:39:33.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7fdd95a1a640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 0x7fdd88031f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c390 tx=0x7fdd8c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cb4036430 con 0x7f6cb4009bd0 2024-09-21T05:39:33.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f08bd1e8640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f089c078c90 0x7f089c07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa80ad6f0 con 0x7f9fa80cfc10 2024-09-21T05:39:33.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.776+0000 7fc64e7fc640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc63c078b90 0x7fc63c07b0a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6500b1f00 tx=0x7fc644009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7f8b5effd640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b4c078b90 0x7f8b4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad2001a070 con 0x7fad280d30c0 2024-09-21T05:39:33.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7f08bd1e8640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f089c078c90 0x7f089c07b1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005880 tx=0x7f08b4005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8b6009b9f0 con 0x7f8b681624d0 2024-09-21T05:39:33.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7f8b6e413640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b4c078b90 0x7f8b4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fa0019020 con 0x7f9fa80cfc10 2024-09-21T05:39:33.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7fad0ffff640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad14078bc0 0x7fad1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.777+0000 7f8b6e413640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b4c078b90 0x7f8b4c07b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000f60 tx=0x7f8b58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.778+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad2009a2a0 con 0x7fad280d30c0 2024-09-21T05:39:33.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.778+0000 7fad26ffd640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad14078bc0 0x7fad1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.778+0000 7f9f9e7fc640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9f8c078be0 0x7f9f8c07b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.778+0000 7f9fadb44640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9f8c078be0 0x7f9f8c07b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.778+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9fa009ae50 con 0x7f9fa80cfc10 2024-09-21T05:39:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.778+0000 7fad26ffd640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad14078bc0 0x7fad1407b0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad1c034000 tx=0x7fad1c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.779+0000 7f9fadb44640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9f8c078be0 0x7f9f8c07b0f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4034000 tx=0x7f9fa4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.781+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc64008a6e0 con 0x7fc650073260 2024-09-21T05:39:33.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.781+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4017070 con 0x7f20dc06e6e0 2024-09-21T05:39:33.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.781+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc153850 con 0x7f20dc06e6e0 2024-09-21T05:39:33.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.781+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f08a8063250 con 0x7f08b0052360 2024-09-21T05:39:33.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.781+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc152ee0 con 0x7f20dc06e6e0 2024-09-21T05:39:33.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.782+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20dc06eaf0 con 0x7f20dc06e6e0 2024-09-21T05:39:33.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.783+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6cb001a070 con 0x7f6cb4009bd0 2024-09-21T05:39:33.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.784+0000 7f6caaffd640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c9c078b90 0x7f6c9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.784+0000 7f6cba59c640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c9c078b90 0x7f6c9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.784+0000 7f6cba59c640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c9c078b90 0x7f6c9c07b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cac001780 tx=0x7f6cac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.786+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad20062190 con 0x7fad280d30c0 2024-09-21T05:39:33.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.787+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20d4007020 con 0x7f20dc06e6e0 2024-09-21T05:39:33.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.787+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4021060 con 0x7f20dc06e6e0 2024-09-21T05:39:33.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.787+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fa0020030 con 0x7f9fa80cfc10 2024-09-21T05:39:33.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.789+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20d401a070 con 0x7f20dc06e6e0 2024-09-21T05:39:33.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.790+0000 7f20d2ffd640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20bc078bc0 0x7f20bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.790+0000 7f20d9d9b640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20bc078bc0 0x7f20bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.790+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f20d409a440 con 0x7f20dc06e6e0 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c011a30 con 0x7fdd88005c60 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd8c011ce0 con 0x7fdd88005c60 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c019650 con 0x7fdd88005c60 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd881783b0 con 0x7fdd88005c60 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd88178970 con 0x7fdd88005c60 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b60063eb0 con 0x7f8b681624d0 2024-09-21T05:39:33.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.792+0000 7f20d9d9b640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20bc078bc0 0x7f20bc07b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20dc1fd510 tx=0x7f20cc0015f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.791+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6cb005e2a0 con 0x7f6cb4009bd0 2024-09-21T05:39:33.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.793+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd880df180 con 0x7fdd88005c60 2024-09-21T05:39:33.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.794+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20d4007450 con 0x7f20dc06e6e0 2024-09-21T05:39:33.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.796+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cb0021080 con 0x7f6cb4009bd0 2024-09-21T05:39:33.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.798+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd8c0197f0 con 0x7fdd88005c60 2024-09-21T05:39:33.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.799+0000 7fdd867fc640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd74078c60 0x7fdd7407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:33.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.799+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdd8c09eea0 con 0x7fdd88005c60 2024-09-21T05:39:33.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.800+0000 7fdd95219640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd74078c60 0x7fdd7407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:33.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.800+0000 7fdd95219640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd74078c60 0x7fdd7407b170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd80036000 tx=0x7fdd80004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:33.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:33.801+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdd88006070 con 0x7fdd88005c60 2024-09-21T05:39:34.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.212+0000 7f08be9eb640 1 -- 172.21.15.22:0/1630598993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f08b00e1520 con 0x7f08b0052360 2024-09-21T05:39:34.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.213+0000 7f08ae7fc640 1 -- 172.21.15.22:0/1630598993 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f08a8062b50 con 0x7f08b0052360 2024-09-21T05:39:34.215 INFO:teuthology.orchestra.run.smithi022.stdout:64424509447 2024-09-21T05:39:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.217+0000 7f089bfff640 1 -- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f089c078c90 msgr2=0x7f089c07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.217+0000 7f089bfff640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f089c078c90 0x7f089c07b1a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f08b0005880 tx=0x7f08b4005f90 comp rx=0 tx=0).stop 2024-09-21T05:39:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.217+0000 7f089bfff640 1 -- 172.21.15.22:0/1630598993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 msgr2=0x7f08b00dd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.217+0000 7f089bfff640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 0x7f08b00dd5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f08a800d690 tx=0x7f08a800db50 comp rx=0 tx=0).stop 2024-09-21T05:39:34.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 -- 172.21.15.22:0/1630598993 shutdown_connections 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f089c078c90 0x7f089c07b1a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08b00e7410 0x7f08b00e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08b00e7000 0x7f08b004fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 --2- 172.21.15.22:0/1630598993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08b0052360 0x7f08b00dd5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 -- 172.21.15.22:0/1630598993 >> 172.21.15.22:0/1630598993 conn(0x7f08b001d9a0 msgr2=0x7f08b0054b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 -- 172.21.15.22:0/1630598993 shutdown_connections 2024-09-21T05:39:34.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.218+0000 7f089bfff640 1 -- 172.21.15.22:0/1630598993 wait complete. 2024-09-21T05:39:34.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.220+0000 7fc65558b640 1 -- 172.21.15.22:0/1292633918 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc6500cc730 con 0x7fc650073260 2024-09-21T05:39:34.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.222+0000 7fc6377fe640 1 -- 172.21.15.22:0/1292633918 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc64008f110 con 0x7fc650073260 2024-09-21T05:39:34.224 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.225+0000 7fc6357fa640 1 -- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc63c078b90 msgr2=0x7fc63c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.225+0000 7fc6357fa640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc63c078b90 0x7fc63c07b0a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6500b1f00 tx=0x7fc644009040 comp rx=0 tx=0).stop 2024-09-21T05:39:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.225+0000 7fc6357fa640 1 -- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc650073260 msgr2=0x7fc6500b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.225+0000 7fc6357fa640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc650073260 0x7fc6500b3cd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6500cb4d0 tx=0x7fc640003820 comp rx=0 tx=0).stop 2024-09-21T05:39:34.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 -- 172.21.15.22:0/1292633918 shutdown_connections 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc650147e70 0x7fc65015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc63c078b90 0x7fc63c07b0a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc65015bb20 0x7fc650147070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 --2- 172.21.15.22:0/1292633918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc650073260 0x7fc6500b3cd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 -- 172.21.15.22:0/1292633918 >> 172.21.15.22:0/1292633918 conn(0x7fc6500b21d0 msgr2=0x7fc6500cca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 -- 172.21.15.22:0/1292633918 shutdown_connections 2024-09-21T05:39:34.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.226+0000 7fc6357fa640 1 -- 172.21.15.22:0/1292633918 wait complete. 2024-09-21T05:39:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.229+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fad280b9040 con 0x7fad280d30c0 2024-09-21T05:39:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.230+0000 7fad0ffff640 1 -- 172.21.15.22:0/2255466298 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fad20066bc0 con 0x7fad280d30c0 2024-09-21T05:39:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.230+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8b50005580 con 0x7f8b681624d0 2024-09-21T05:39:34.232 INFO:teuthology.orchestra.run.smithi022.stdout:60129542153 2024-09-21T05:39:34.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.231+0000 7f8b5effd640 1 -- 172.21.15.22:0/794945431 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b600637b0 con 0x7f8b681624d0 2024-09-21T05:39:34.233 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad14078bc0 msgr2=0x7fad1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad14078bc0 0x7fad1407b0d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fad1c034000 tx=0x7fad1c004730 comp rx=0 tx=0).stop 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 msgr2=0x7fad2815f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 0x7fad2815f020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c7a0 tx=0x7fad2000cc60 comp rx=0 tx=0).stop 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 shutdown_connections 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad14078bc0 0x7fad1407b0d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad280d4ca0 0x7fad280c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad280d34d0 0x7fad280c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 --2- 172.21.15.22:0/2255466298 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad280d30c0 0x7fad2815f020 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.234+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 >> 172.21.15.22:0/2255466298 conn(0x7fad2809d7f0 msgr2=0x7fad280b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 shutdown_connections 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7fad0dffb640 1 -- 172.21.15.22:0/2255466298 wait complete. 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b4c078b90 msgr2=0x7f8b4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b4c078b90 0x7f8b4c07b0a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8b58000f60 tx=0x7f8b58009040 comp rx=0 tx=0).stop 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 msgr2=0x7f8b6809b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 0x7f8b6809b6b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000ea80 tx=0x7f8b6000ef40 comp rx=0 tx=0).stop 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 shutdown_connections 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b681624d0 0x7f8b6809b6b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b6815f9a0 0x7f8b680b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b4c078b90 0x7f8b4c07b0a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 --2- 172.21.15.22:0/794945431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b680ccde0 0x7f8b681515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 >> 172.21.15.22:0/794945431 conn(0x7f8b680b20a0 msgr2=0x7f8b680d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.235+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f20dc0d5360 con 0x7f20dc06e6e0 2024-09-21T05:39:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.236+0000 7f20d2ffd640 1 -- 172.21.15.22:0/1015110312 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f20d4062360 con 0x7f20dc06e6e0 2024-09-21T05:39:34.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.236+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 shutdown_connections 2024-09-21T05:39:34.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.236+0000 7f8b70677640 1 -- 172.21.15.22:0/794945431 wait complete. 2024-09-21T05:39:34.240 INFO:teuthology.orchestra.run.smithi022.stdout:60129542150 2024-09-21T05:39:34.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-21T05:39:34.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.238+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6cb4050c10 con 0x7f6cb4009bd0 2024-09-21T05:39:34.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.238+0000 7f6caaffd640 1 -- 172.21.15.22:0/1159052676 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6cb40060a0 con 0x7f6cb4009bd0 2024-09-21T05:39:34.241 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.242 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20bc078bc0 msgr2=0x7f20bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20bc078bc0 0x7f20bc07b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20dc1fd510 tx=0x7f20cc0015f0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc06e6e0 msgr2=0x7f20dc15d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc06e6e0 0x7f20dc15d1a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f20d400c410 tx=0x7f20d400c8d0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 shutdown_connections 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20bc078bc0 0x7f20bc07b0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f20dc0d3340 0x7f20dc0b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f20dc0b96d0 0x7f20dc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 --2- 172.21.15.22:0/1015110312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f20dc06e6e0 0x7f20dc15d1a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 >> 172.21.15.22:0/1015110312 conn(0x7f20dc09d8c0 msgr2=0x7f20dc0d15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 shutdown_connections 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.240+0000 7f20e08fa640 1 -- 172.21.15.22:0/1015110312 wait complete. 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.241+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9fa80b3610 con 0x7f9fa80cfc10 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.242+0000 7f9f9e7fc640 1 -- 172.21.15.22:0/3553338987 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fa0063370 con 0x7f9fa80cfc10 2024-09-21T05:39:34.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.242+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdd8802acb0 con 0x7fdd88005c60 2024-09-21T05:39:34.245 INFO:teuthology.orchestra.run.smithi022.stdout:60129542151 2024-09-21T05:39:34.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.242+0000 7fdd867fc640 1 -- 172.21.15.22:0/3316204122 <== mon.1 v2:172.21.15.134:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdd8c066d80 con 0x7fdd88005c60 2024-09-21T05:39:34.245 INFO:teuthology.orchestra.run.smithi022.stdout:64424509448 2024-09-21T05:39:34.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.243+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c9c078b90 msgr2=0x7f6c9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.243+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c9c078b90 0x7f6c9c07b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cac001780 tx=0x7f6cac009040 comp rx=0 tx=0).stop 2024-09-21T05:39:34.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.243+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb4009bd0 msgr2=0x7f6cb4030aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.243+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb4009bd0 0x7f6cb4030aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c3c0 tx=0x7f6cb000c880 comp rx=0 tx=0).stop 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 shutdown_connections 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c9c078b90 0x7f6c9c07b0a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6cb400bc60 0x7f6cb40d4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6cb4009bd0 0x7f6cb4030aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 --2- 172.21.15.22:0/1159052676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6cb4005c90 0x7f6cb4032400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 >> 172.21.15.22:0/1159052676 conn(0x7f6cb4051450 msgr2=0x7f6cb4052520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 shutdown_connections 2024-09-21T05:39:34.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.244+0000 7f6cc0cd5640 1 -- 172.21.15.22:0/1159052676 wait complete. 2024-09-21T05:39:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9f8c078be0 msgr2=0x7f9f8c07b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9f8c078be0 0x7f9f8c07b0f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4034000 tx=0x7f9fa4004730 comp rx=0 tx=0).stop 2024-09-21T05:39:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd74078c60 msgr2=0x7fdd7407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd74078c60 0x7fdd7407b170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd80036000 tx=0x7fdd80004730 comp rx=0 tx=0).stop 2024-09-21T05:39:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 msgr2=0x7fdd88031f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 0x7fdd88031f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00c390 tx=0x7fdd8c00c850 comp rx=0 tx=0).stop 2024-09-21T05:39:34.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 msgr2=0x7f9fa8203930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 0x7f9fa8203930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9fa000cae0 tx=0x7f9fa000a060 comp rx=0 tx=0).stop 2024-09-21T05:39:34.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 shutdown_connections 2024-09-21T05:39:34.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa80cfc10 0x7f9fa8203930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 shutdown_connections 2024-09-21T05:39:34.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd74078c60 0x7fdd7407b170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd8800bc30 0x7fdd880e1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd88009ba0 0x7fdd880e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 --2- 172.21.15.22:0/3316204122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd88005c60 0x7fdd88031f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 >> 172.21.15.22:0/3316204122 conn(0x7fdd88051420 msgr2=0x7fdd88034b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.246+0000 7f9f93fff640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9f8c078be0 0x7f9f8c07b0f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 shutdown_connections 2024-09-21T05:39:34.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7fdd96a1c640 1 -- 172.21.15.22:0/3316204122 wait complete. 2024-09-21T05:39:34.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2024-09-21T05:39:34.252 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7f9f93fff640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fa808b690 0x7f9fa808a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7f9f93fff640 1 --2- 172.21.15.22:0/3553338987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa808b140 0x7f9fa80b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 >> 172.21.15.22:0/3553338987 conn(0x7f9fa80b20a0 msgr2=0x7f9fa80d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 shutdown_connections 2024-09-21T05:39:34.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.247+0000 7f9f93fff640 1 -- 172.21.15.22:0/3553338987 wait complete. 2024-09-21T05:39:34.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.5 2024-09-21T05:39:34.258 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.2 2024-09-21T05:39:34.258 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-09-21T05:39:34.265 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-09-21T05:39:34.269 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.6 2024-09-21T05:39:34.269 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-09-21T05:39:34.270 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:34.270 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:39:34.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 -- 172.21.15.22:0/2116166218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c0cfae0 msgr2=0x7fa57c15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 --2- 172.21.15.22:0/2116166218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c0cfae0 0x7fa57c15ef10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009830 tx=0x7fa56c02a3f0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 -- 172.21.15.22:0/2116166218 shutdown_connections 2024-09-21T05:39:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 --2- 172.21.15.22:0/2116166218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c0cfae0 0x7fa57c15ef10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 --2- 172.21.15.22:0/2116166218 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 0x7fa57c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 -- 172.21.15.22:0/2116166218 >> 172.21.15.22:0/2116166218 conn(0x7fa57c06d7d0 msgr2=0x7fa57c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 -- 172.21.15.22:0/2116166218 shutdown_connections 2024-09-21T05:39:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.423+0000 7fa583c27640 1 -- 172.21.15.22:0/2116166218 wait complete. 2024-09-21T05:39:34.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.425+0000 7fa583c27640 1 Processor -- start 2024-09-21T05:39:34.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.425+0000 7fa583c27640 1 -- start start 2024-09-21T05:39:34.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa583c27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 0x7fa57c1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa583c27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 0x7fa57c157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa583c27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57c0cfae0 0x7fa57c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa583c27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c06fb30 con 0x7fa57c06ee90 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa583c27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c06f9b0 con 0x7fa57c0cfae0 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa583c27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c0cc970 con 0x7fa57c070f20 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 0x7fa57c1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 0x7fa57c1518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38632/0 (socket says 172.21.15.22:38632) 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 -- 172.21.15.22:0/3036717343 learned_addr learned my addr 172.21.15.22:0/3036717343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5811c2640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 0x7fa57c157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5821c4640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57c0cfae0 0x7fa57c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 -- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 msgr2=0x7fa57c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 0x7fa57c157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 -- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57c0cfae0 msgr2=0x7fa57c158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57c0cfae0 0x7fa57c158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5819c3640 1 -- 172.21.15.22:0/3036717343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c1f4590 con 0x7fa57c06ee90 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5811c2640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 0x7fa57c157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.426+0000 7fa5821c4640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57c0cfae0 0x7fa57c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.427+0000 7fa5819c3640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 0x7fa57c1518a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa57800d6d0 tx=0x7fa57800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.427+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578017070 con 0x7fa57c06ee90 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.427+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa578005330 con 0x7fa57c06ee90 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.427+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c1f4870 con 0x7fa57c06ee90 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.427+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa578009820 con 0x7fa57c06ee90 2024-09-21T05:39:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.427+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c157510 con 0x7fa57c06ee90 2024-09-21T05:39:34.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.428+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c15a9b0 con 0x7fa57c06ee90 2024-09-21T05:39:34.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.429+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa578009bd0 con 0x7fa57c06ee90 2024-09-21T05:39:34.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.429+0000 7fa5727fc640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa560078b90 0x7fa56007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.430+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa57809b1b0 con 0x7fa57c06ee90 2024-09-21T05:39:34.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.430+0000 7fa5811c2640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa560078b90 0x7fa56007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.430+0000 7fa5811c2640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa560078b90 0x7fa56007b0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa56c034000 tx=0x7fa56c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:34.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.434+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa578063550 con 0x7fa57c06ee90 2024-09-21T05:39:34.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.619+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa57c15f230 con 0x7fa560078b90 2024-09-21T05:39:34.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.624+0000 7fa5727fc640 1 -- 172.21.15.22:0/3036717343 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fa57c15f230 con 0x7fa560078b90 2024-09-21T05:39:34.629 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:34.630 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa560078b90 msgr2=0x7fa56007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa560078b90 0x7fa56007b0a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa56c034000 tx=0x7fa56c004730 comp rx=0 tx=0).stop 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 msgr2=0x7fa57c1518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 0x7fa57c1518a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa57800d6d0 tx=0x7fa57800db90 comp rx=0 tx=0).stop 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 shutdown_connections 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa57c0cfae0 0x7fa57c158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa560078b90 0x7fa56007b0a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa57c070f20 0x7fa57c157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 --2- 172.21.15.22:0/3036717343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa57c06ee90 0x7fa57c1518a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.627+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 >> 172.21.15.22:0/3036717343 conn(0x7fa57c06d7d0 msgr2=0x7fa57c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.628+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 shutdown_connections 2024-09-21T05:39:34.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.628+0000 7fa583c27640 1 -- 172.21.15.22:0/3036717343 wait complete. 2024-09-21T05:39:34.641 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-21T05:39:33.739686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239896,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734824,"statfs":{"total":773094113280,"available":772848459776,"internally_reserved":0,"allocated":2924544,"data_stored":1773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002740"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226309+0000","last_change":"2024-09-21T05:39:22.834732+0000","last_active":"2024-09-21T05:39:24.226309+0000","last_peered":"2024-09-21T05:39:24.226309+0000","last_clean":"2024-09-21T05:39:24.226309+0000","last_became_active":"2024-09-21T05:39:22.833098+0000","last_became_peered":"2024-09-21T05:39:22.833098+0000","last_unstale":"2024-09-21T05:39:24.226309+0000","last_undegraded":"2024-09-21T05:39:24.226309+0000","last_fullsized":"2024-09-21T05:39:24.226309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.341720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225995+0000","last_change":"2024-09-21T05:39:22.838816+0000","last_active":"2024-09-21T05:39:24.225995+0000","last_peered":"2024-09-21T05:39:24.225995+0000","last_clean":"2024-09-21T05:39:24.225995+0000","last_became_active":"2024-09-21T05:39:22.838511+0000","last_became_peered":"2024-09-21T05:39:22.838511+0000","last_unstale":"2024-09-21T05:39:24.225995+0000","last_undegraded":"2024-09-21T05:39:24.225995+0000","last_fullsized":"2024-09-21T05:39:24.225995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:27.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225652+0000","last_change":"2024-09-21T05:39:22.843633+0000","last_active":"2024-09-21T05:39:24.225652+0000","last_peered":"2024-09-21T05:39:24.225652+0000","last_clean":"2024-09-21T05:39:24.225652+0000","last_became_active":"2024-09-21T05:39:22.843200+0000","last_became_peered":"2024-09-21T05:39:22.843200+0000","last_unstale":"2024-09-21T05:39:24.225652+0000","last_undegraded":"2024-09-21T05:39:24.225652+0000","last_fullsized":"2024-09-21T05:39:24.225652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:30.085035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226390+0000","last_change":"2024-09-21T05:39:22.836751+0000","last_active":"2024-09-21T05:39:24.226390+0000","last_peered":"2024-09-21T05:39:24.226390+0000","last_clean":"2024-09-21T05:39:24.226390+0000","last_became_active":"2024-09-21T05:39:22.833532+0000","last_became_peered":"2024-09-21T05:39:22.833532+0000","last_unstale":"2024-09-21T05:39:24.226390+0000","last_undegraded":"2024-09-21T05:39:24.226390+0000","last_fullsized":"2024-09-21T05:39:24.226390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:07.246445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831080+0000","last_change":"2024-09-21T05:39:22.821659+0000","last_active":"2024-09-21T05:39:23.831080+0000","last_peered":"2024-09-21T05:39:23.831080+0000","last_clean":"2024-09-21T05:39:23.831080+0000","last_became_active":"2024-09-21T05:39:22.818471+0000","last_became_peered":"2024-09-21T05:39:22.818471+0000","last_unstale":"2024-09-21T05:39:23.831080+0000","last_undegraded":"2024-09-21T05:39:23.831080+0000","last_fullsized":"2024-09-21T05:39:23.831080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:11.829615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226338+0000","last_change":"2024-09-21T05:39:22.837584+0000","last_active":"2024-09-21T05:39:24.226338+0000","last_peered":"2024-09-21T05:39:24.226338+0000","last_clean":"2024-09-21T05:39:24.226338+0000","last_became_active":"2024-09-21T05:39:22.837243+0000","last_became_peered":"2024-09-21T05:39:22.837243+0000","last_unstale":"2024-09-21T05:39:24.226338+0000","last_undegraded":"2024-09-21T05:39:24.226338+0000","last_fullsized":"2024-09-21T05:39:24.226338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:46.224414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830586+0000","last_change":"2024-09-21T05:39:22.822050+0000","last_active":"2024-09-21T05:39:23.830586+0000","last_peered":"2024-09-21T05:39:23.830586+0000","last_clean":"2024-09-21T05:39:23.830586+0000","last_became_active":"2024-09-21T05:39:22.821705+0000","last_became_peered":"2024-09-21T05:39:22.821705+0000","last_unstale":"2024-09-21T05:39:23.830586+0000","last_undegraded":"2024-09-21T05:39:23.830586+0000","last_fullsized":"2024-09-21T05:39:23.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:42.204486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831067+0000","last_change":"2024-09-21T05:39:22.820929+0000","last_active":"2024-09-21T05:39:23.831067+0000","last_peered":"2024-09-21T05:39:23.831067+0000","last_clean":"2024-09-21T05:39:23.831067+0000","last_became_active":"2024-09-21T05:39:22.820587+0000","last_became_peered":"2024-09-21T05:39:22.820587+0000","last_unstale":"2024-09-21T05:39:23.831067+0000","last_undegraded":"2024-09-21T05:39:23.831067+0000","last_fullsized":"2024-09-21T05:39:23.831067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:42.698282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226051+0000","last_change":"2024-09-21T05:39:22.838353+0000","last_active":"2024-09-21T05:39:24.226051+0000","last_peered":"2024-09-21T05:39:24.226051+0000","last_clean":"2024-09-21T05:39:24.226051+0000","last_became_active":"2024-09-21T05:39:22.838044+0000","last_became_peered":"2024-09-21T05:39:22.838044+0000","last_unstale":"2024-09-21T05:39:24.226051+0000","last_undegraded":"2024-09-21T05:39:24.226051+0000","last_fullsized":"2024-09-21T05:39:24.226051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:27.463085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225891+0000","last_change":"2024-09-21T05:39:22.833336+0000","last_active":"2024-09-21T05:39:24.225891+0000","last_peered":"2024-09-21T05:39:24.225891+0000","last_clean":"2024-09-21T05:39:24.225891+0000","last_became_active":"2024-09-21T05:39:22.832959+0000","last_became_peered":"2024-09-21T05:39:22.832959+0000","last_unstale":"2024-09-21T05:39:24.225891+0000","last_undegraded":"2024-09-21T05:39:24.225891+0000","last_fullsized":"2024-09-21T05:39:24.225891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.450059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225617+0000","last_change":"2024-09-21T05:39:22.834980+0000","last_active":"2024-09-21T05:39:24.225617+0000","last_peered":"2024-09-21T05:39:24.225617+0000","last_clean":"2024-09-21T05:39:24.225617+0000","last_became_active":"2024-09-21T05:39:22.834075+0000","last_became_peered":"2024-09-21T05:39:22.834075+0000","last_unstale":"2024-09-21T05:39:24.225617+0000","last_undegraded":"2024-09-21T05:39:24.225617+0000","last_fullsized":"2024-09-21T05:39:24.225617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:31.814988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830803+0000","last_change":"2024-09-21T05:39:21.824655+0000","last_active":"2024-09-21T05:39:23.830803+0000","last_peered":"2024-09-21T05:39:23.830803+0000","last_clean":"2024-09-21T05:39:23.830803+0000","last_became_active":"2024-09-21T05:39:21.824309+0000","last_became_peered":"2024-09-21T05:39:21.824309+0000","last_unstale":"2024-09-21T05:39:23.830803+0000","last_undegraded":"2024-09-21T05:39:23.830803+0000","last_fullsized":"2024-09-21T05:39:23.830803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:50.559527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226272+0000","last_change":"2024-09-21T05:39:22.830496+0000","last_active":"2024-09-21T05:39:24.226272+0000","last_peered":"2024-09-21T05:39:24.226272+0000","last_clean":"2024-09-21T05:39:24.226272+0000","last_became_active":"2024-09-21T05:39:22.829859+0000","last_became_peered":"2024-09-21T05:39:22.829859+0000","last_unstale":"2024-09-21T05:39:24.226272+0000","last_undegraded":"2024-09-21T05:39:24.226272+0000","last_fullsized":"2024-09-21T05:39:24.226272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:09.763411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226478+0000","last_change":"2024-09-21T05:39:22.837101+0000","last_active":"2024-09-21T05:39:24.226478+0000","last_peered":"2024-09-21T05:39:24.226478+0000","last_clean":"2024-09-21T05:39:24.226478+0000","last_became_active":"2024-09-21T05:39:22.834638+0000","last_became_peered":"2024-09-21T05:39:22.834638+0000","last_unstale":"2024-09-21T05:39:24.226478+0000","last_undegraded":"2024-09-21T05:39:24.226478+0000","last_fullsized":"2024-09-21T05:39:24.226478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.717004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225997+0000","last_change":"2024-09-21T05:39:22.828178+0000","last_active":"2024-09-21T05:39:24.225997+0000","last_peered":"2024-09-21T05:39:24.225997+0000","last_clean":"2024-09-21T05:39:24.225997+0000","last_became_active":"2024-09-21T05:39:22.827833+0000","last_became_peered":"2024-09-21T05:39:22.827833+0000","last_unstale":"2024-09-21T05:39:24.225997+0000","last_undegraded":"2024-09-21T05:39:24.225997+0000","last_fullsized":"2024-09-21T05:39:24.225997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:58.078709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225514+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:24.225514+0000","last_peered":"2024-09-21T05:39:24.225514+0000","last_clean":"2024-09-21T05:39:24.225514+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:24.225514+0000","last_undegraded":"2024-09-21T05:39:24.225514+0000","last_fullsized":"2024-09-21T05:39:24.225514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225383+0000","last_change":"2024-09-21T05:39:21.833248+0000","last_active":"2024-09-21T05:39:24.225383+0000","last_peered":"2024-09-21T05:39:24.225383+0000","last_clean":"2024-09-21T05:39:24.225383+0000","last_became_active":"2024-09-21T05:39:21.832873+0000","last_became_peered":"2024-09-21T05:39:21.832873+0000","last_unstale":"2024-09-21T05:39:24.225383+0000","last_undegraded":"2024-09-21T05:39:24.225383+0000","last_fullsized":"2024-09-21T05:39:24.225383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:29.420862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830594+0000","last_change":"2024-09-21T05:39:22.831920+0000","last_active":"2024-09-21T05:39:23.830594+0000","last_peered":"2024-09-21T05:39:23.830594+0000","last_clean":"2024-09-21T05:39:23.830594+0000","last_became_active":"2024-09-21T05:39:22.831421+0000","last_became_peered":"2024-09-21T05:39:22.831421+0000","last_unstale":"2024-09-21T05:39:23.830594+0000","last_undegraded":"2024-09-21T05:39:23.830594+0000","last_fullsized":"2024-09-21T05:39:23.830594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:56.775839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226188+0000","last_change":"2024-09-21T05:39:21.831044+0000","last_active":"2024-09-21T05:39:24.226188+0000","last_peered":"2024-09-21T05:39:24.226188+0000","last_clean":"2024-09-21T05:39:24.226188+0000","last_became_active":"2024-09-21T05:39:21.830412+0000","last_became_peered":"2024-09-21T05:39:21.830412+0000","last_unstale":"2024-09-21T05:39:24.226188+0000","last_undegraded":"2024-09-21T05:39:24.226188+0000","last_fullsized":"2024-09-21T05:39:24.226188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:36.640541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225380+0000","last_change":"2024-09-21T05:39:21.833615+0000","last_active":"2024-09-21T05:39:24.225380+0000","last_peered":"2024-09-21T05:39:24.225380+0000","last_clean":"2024-09-21T05:39:24.225380+0000","last_became_active":"2024-09-21T05:39:21.833028+0000","last_became_peered":"2024-09-21T05:39:21.833028+0000","last_unstale":"2024-09-21T05:39:24.225380+0000","last_undegraded":"2024-09-21T05:39:24.225380+0000","last_fullsized":"2024-09-21T05:39:24.225380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:28.358924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829777+0000","last_change":"2024-09-21T05:39:22.827047+0000","last_active":"2024-09-21T05:39:23.829777+0000","last_peered":"2024-09-21T05:39:23.829777+0000","last_clean":"2024-09-21T05:39:23.829777+0000","last_became_active":"2024-09-21T05:39:22.825920+0000","last_became_peered":"2024-09-21T05:39:22.825920+0000","last_unstale":"2024-09-21T05:39:23.829777+0000","last_undegraded":"2024-09-21T05:39:23.829777+0000","last_fullsized":"2024-09-21T05:39:23.829777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:56.401187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226082+0000","last_change":"2024-09-21T05:39:21.829961+0000","last_active":"2024-09-21T05:39:24.226082+0000","last_peered":"2024-09-21T05:39:24.226082+0000","last_clean":"2024-09-21T05:39:24.226082+0000","last_became_active":"2024-09-21T05:39:21.829484+0000","last_became_peered":"2024-09-21T05:39:21.829484+0000","last_unstale":"2024-09-21T05:39:24.226082+0000","last_undegraded":"2024-09-21T05:39:24.226082+0000","last_fullsized":"2024-09-21T05:39:24.226082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:16.446491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225816+0000","last_change":"2024-09-21T05:39:21.830335+0000","last_active":"2024-09-21T05:39:24.225816+0000","last_peered":"2024-09-21T05:39:24.225816+0000","last_clean":"2024-09-21T05:39:24.225816+0000","last_became_active":"2024-09-21T05:39:21.829550+0000","last_became_peered":"2024-09-21T05:39:21.829550+0000","last_unstale":"2024-09-21T05:39:24.225816+0000","last_undegraded":"2024-09-21T05:39:24.225816+0000","last_fullsized":"2024-09-21T05:39:24.225816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:51.148096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226090+0000","last_change":"2024-09-21T05:39:21.830622+0000","last_active":"2024-09-21T05:39:24.226090+0000","last_peered":"2024-09-21T05:39:24.226090+0000","last_clean":"2024-09-21T05:39:24.226090+0000","last_became_active":"2024-09-21T05:39:21.829994+0000","last_became_peered":"2024-09-21T05:39:21.829994+0000","last_unstale":"2024-09-21T05:39:24.226090+0000","last_undegraded":"2024-09-21T05:39:24.226090+0000","last_fullsized":"2024-09-21T05:39:24.226090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:41.692740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":664479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:34.644 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.795+0000 7f8d953c3640 1 -- 172.21.15.22:0/729384361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d78003c10 msgr2=0x7f8d78004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.795+0000 7f8d953c3640 1 --2- 172.21.15.22:0/729384361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d78003c10 0x7f8d78004060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009b40 tx=0x7f8d78005830 comp rx=0 tx=0).stop 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.795+0000 7f8d8f7fe640 1 -- 172.21.15.22:0/729384361 reap_dead start 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.795+0000 7f8d953c3640 1 -- 172.21.15.22:0/729384361 shutdown_connections 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.795+0000 7f8d953c3640 1 -- 172.21.15.22:0/729384361 >> 172.21.15.22:0/729384361 conn(0x7f8d9009d7f0 msgr2=0x7f8d900d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.795+0000 7f8d953c3640 1 -- 172.21.15.22:0/729384361 shutdown_connections 2024-09-21T05:39:34.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.796+0000 7f8d953c3640 1 -- 172.21.15.22:0/729384361 wait complete. 2024-09-21T05:39:34.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.797+0000 7f8d953c3640 1 Processor -- start 2024-09-21T05:39:34.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.797+0000 7f8d953c3640 1 -- start start 2024-09-21T05:39:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d953c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78003c10 0x7f8d901601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d953c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d900b4e50 0x7f8d900b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d953c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 0x7f8d900ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d953c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d900b3aa0 con 0x7f8d900ca040 2024-09-21T05:39:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d953c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d900b3920 con 0x7f8d900b4e50 2024-09-21T05:39:34.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d953c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d900b3c20 con 0x7f8d78003c10 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 0x7f8d900ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 0x7f8d900ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38650/0 (socket says 172.21.15.22:38650) 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 -- 172.21.15.22:0/685899442 learned_addr learned my addr 172.21.15.22:0/685899442 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8effd640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78003c10 0x7f8d901601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8e7fc640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d900b4e50 0x7f8d900b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 -- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78003c10 msgr2=0x7f8d901601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78003c10 0x7f8d901601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 -- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d900b4e50 msgr2=0x7f8d900b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d900b4e50 0x7f8d900b5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 -- 172.21.15.22:0/685899442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d901f6ad0 con 0x7f8d900ca040 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.799+0000 7f8d8f7fe640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 0x7f8d900ca450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d9008d920 tx=0x7f8d8800eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.800+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8800c9d0 con 0x7f8d900ca040 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.800+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d88018070 con 0x7f8d900ca040 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.800+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88011460 con 0x7f8d900ca040 2024-09-21T05:39:34.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.800+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d901f5f50 con 0x7f8d900ca040 2024-09-21T05:39:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.800+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901f6510 con 0x7f8d900ca040 2024-09-21T05:39:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.801+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d9006e4f0 con 0x7f8d900ca040 2024-09-21T05:39:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.801+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d88011600 con 0x7f8d900ca040 2024-09-21T05:39:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.802+0000 7f8d777fe640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8d7c078bc0 0x7f8d7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:34.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.802+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8d8809b030 con 0x7f8d900ca040 2024-09-21T05:39:34.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.802+0000 7f8d8effd640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8d7c078bc0 0x7f8d7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:34.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.803+0000 7f8d8effd640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8d7c078bc0 0x7f8d7c07b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d80035000 tx=0x7f8d8002d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:34.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.807+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8d88062d80 con 0x7f8d900ca040 2024-09-21T05:39:34.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.993+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d900d2ac0 con 0x7f8d7c078bc0 2024-09-21T05:39:34.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:34.998+0000 7f8d777fe640 1 -- 172.21.15.22:0/685899442 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f8d900d2ac0 con 0x7f8d7c078bc0 2024-09-21T05:39:35.002 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:35.003 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8d7c078bc0 msgr2=0x7f8d7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8d7c078bc0 0x7f8d7c07b0d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8d80035000 tx=0x7f8d8002d040 comp rx=0 tx=0).stop 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 msgr2=0x7f8d900ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 0x7f8d900ca450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d9008d920 tx=0x7f8d8800eae0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 shutdown_connections 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d900ca040 0x7f8d900ca450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d900b4e50 0x7f8d900b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8d7c078bc0 0x7f8d7c07b0d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 --2- 172.21.15.22:0/685899442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d78003c10 0x7f8d901601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 >> 172.21.15.22:0/685899442 conn(0x7f8d9009d7f0 msgr2=0x7f8d9015e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:35.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 shutdown_connections 2024-09-21T05:39:35.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.001+0000 7f8d953c3640 1 -- 172.21.15.22:0/685899442 wait complete. 2024-09-21T05:39:35.014 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-21T05:39:33.739686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239896,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754734824,"statfs":{"total":773094113280,"available":772848459776,"internally_reserved":0,"allocated":2924544,"data_stored":1773788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002740"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226309+0000","last_change":"2024-09-21T05:39:22.834732+0000","last_active":"2024-09-21T05:39:24.226309+0000","last_peered":"2024-09-21T05:39:24.226309+0000","last_clean":"2024-09-21T05:39:24.226309+0000","last_became_active":"2024-09-21T05:39:22.833098+0000","last_became_peered":"2024-09-21T05:39:22.833098+0000","last_unstale":"2024-09-21T05:39:24.226309+0000","last_undegraded":"2024-09-21T05:39:24.226309+0000","last_fullsized":"2024-09-21T05:39:24.226309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:17.341720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828533+0000","last_change":"2024-09-21T05:39:22.828684+0000","last_active":"2024-09-21T05:39:23.828533+0000","last_peered":"2024-09-21T05:39:23.828533+0000","last_clean":"2024-09-21T05:39:23.828533+0000","last_became_active":"2024-09-21T05:39:22.828364+0000","last_became_peered":"2024-09-21T05:39:22.828364+0000","last_unstale":"2024-09-21T05:39:23.828533+0000","last_undegraded":"2024-09-21T05:39:23.828533+0000","last_fullsized":"2024-09-21T05:39:23.828533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:48.651070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225779+0000","last_change":"2024-09-21T05:39:22.834011+0000","last_active":"2024-09-21T05:39:24.225779+0000","last_peered":"2024-09-21T05:39:24.225779+0000","last_clean":"2024-09-21T05:39:24.225779+0000","last_became_active":"2024-09-21T05:39:22.833666+0000","last_became_peered":"2024-09-21T05:39:22.833666+0000","last_unstale":"2024-09-21T05:39:24.225779+0000","last_undegraded":"2024-09-21T05:39:24.225779+0000","last_fullsized":"2024-09-21T05:39:24.225779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099169+0000","last_change":"2024-09-21T05:39:22.839961+0000","last_active":"2024-09-21T05:39:24.099169+0000","last_peered":"2024-09-21T05:39:24.099169+0000","last_clean":"2024-09-21T05:39:24.099169+0000","last_became_active":"2024-09-21T05:39:22.839652+0000","last_became_peered":"2024-09-21T05:39:22.839652+0000","last_unstale":"2024-09-21T05:39:24.099169+0000","last_undegraded":"2024-09-21T05:39:24.099169+0000","last_fullsized":"2024-09-21T05:39:24.099169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:57.013650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225995+0000","last_change":"2024-09-21T05:39:22.838816+0000","last_active":"2024-09-21T05:39:24.225995+0000","last_peered":"2024-09-21T05:39:24.225995+0000","last_clean":"2024-09-21T05:39:24.225995+0000","last_became_active":"2024-09-21T05:39:22.838511+0000","last_became_peered":"2024-09-21T05:39:22.838511+0000","last_unstale":"2024-09-21T05:39:24.225995+0000","last_undegraded":"2024-09-21T05:39:24.225995+0000","last_fullsized":"2024-09-21T05:39:24.225995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:27.029278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098579+0000","last_change":"2024-09-21T05:39:22.836373+0000","last_active":"2024-09-21T05:39:24.098579+0000","last_peered":"2024-09-21T05:39:24.098579+0000","last_clean":"2024-09-21T05:39:24.098579+0000","last_became_active":"2024-09-21T05:39:22.836041+0000","last_became_peered":"2024-09-21T05:39:22.836041+0000","last_unstale":"2024-09-21T05:39:24.098579+0000","last_undegraded":"2024-09-21T05:39:24.098579+0000","last_fullsized":"2024-09-21T05:39:24.098579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225652+0000","last_change":"2024-09-21T05:39:22.843633+0000","last_active":"2024-09-21T05:39:24.225652+0000","last_peered":"2024-09-21T05:39:24.225652+0000","last_clean":"2024-09-21T05:39:24.225652+0000","last_became_active":"2024-09-21T05:39:22.843200+0000","last_became_peered":"2024-09-21T05:39:22.843200+0000","last_unstale":"2024-09-21T05:39:24.225652+0000","last_undegraded":"2024-09-21T05:39:24.225652+0000","last_fullsized":"2024-09-21T05:39:24.225652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:30.085035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179867+0000","last_change":"2024-09-21T05:39:22.844140+0000","last_active":"2024-09-21T05:39:24.179867+0000","last_peered":"2024-09-21T05:39:24.179867+0000","last_clean":"2024-09-21T05:39:24.179867+0000","last_became_active":"2024-09-21T05:39:22.843811+0000","last_became_peered":"2024-09-21T05:39:22.843811+0000","last_unstale":"2024-09-21T05:39:24.179867+0000","last_undegraded":"2024-09-21T05:39:24.179867+0000","last_fullsized":"2024-09-21T05:39:24.179867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:38.810365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226390+0000","last_change":"2024-09-21T05:39:22.836751+0000","last_active":"2024-09-21T05:39:24.226390+0000","last_peered":"2024-09-21T05:39:24.226390+0000","last_clean":"2024-09-21T05:39:24.226390+0000","last_became_active":"2024-09-21T05:39:22.833532+0000","last_became_peered":"2024-09-21T05:39:22.833532+0000","last_unstale":"2024-09-21T05:39:24.226390+0000","last_undegraded":"2024-09-21T05:39:24.226390+0000","last_fullsized":"2024-09-21T05:39:24.226390+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:07.246445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829937+0000","last_change":"2024-09-21T05:39:22.824483+0000","last_active":"2024-09-21T05:39:23.829937+0000","last_peered":"2024-09-21T05:39:23.829937+0000","last_clean":"2024-09-21T05:39:23.829937+0000","last_became_active":"2024-09-21T05:39:22.821704+0000","last_became_peered":"2024-09-21T05:39:22.821704+0000","last_unstale":"2024-09-21T05:39:23.829937+0000","last_undegraded":"2024-09-21T05:39:23.829937+0000","last_fullsized":"2024-09-21T05:39:23.829937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829753+0000","last_change":"2024-09-21T05:39:22.821974+0000","last_active":"2024-09-21T05:39:23.829753+0000","last_peered":"2024-09-21T05:39:23.829753+0000","last_clean":"2024-09-21T05:39:23.829753+0000","last_became_active":"2024-09-21T05:39:22.821490+0000","last_became_peered":"2024-09-21T05:39:22.821490+0000","last_unstale":"2024-09-21T05:39:23.829753+0000","last_undegraded":"2024-09-21T05:39:23.829753+0000","last_fullsized":"2024-09-21T05:39:23.829753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831080+0000","last_change":"2024-09-21T05:39:22.821659+0000","last_active":"2024-09-21T05:39:23.831080+0000","last_peered":"2024-09-21T05:39:23.831080+0000","last_clean":"2024-09-21T05:39:23.831080+0000","last_became_active":"2024-09-21T05:39:22.818471+0000","last_became_peered":"2024-09-21T05:39:22.818471+0000","last_unstale":"2024-09-21T05:39:23.831080+0000","last_undegraded":"2024-09-21T05:39:23.831080+0000","last_fullsized":"2024-09-21T05:39:23.831080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:11.829615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179986+0000","last_change":"2024-09-21T05:39:22.840543+0000","last_active":"2024-09-21T05:39:24.179986+0000","last_peered":"2024-09-21T05:39:24.179986+0000","last_clean":"2024-09-21T05:39:24.179986+0000","last_became_active":"2024-09-21T05:39:22.840189+0000","last_became_peered":"2024-09-21T05:39:22.840189+0000","last_unstale":"2024-09-21T05:39:24.179986+0000","last_undegraded":"2024-09-21T05:39:24.179986+0000","last_fullsized":"2024-09-21T05:39:24.179986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:20.353753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225595+0000","last_change":"2024-09-21T05:39:22.827094+0000","last_active":"2024-09-21T05:39:24.225595+0000","last_peered":"2024-09-21T05:39:24.225595+0000","last_clean":"2024-09-21T05:39:24.225595+0000","last_became_active":"2024-09-21T05:39:22.826767+0000","last_became_peered":"2024-09-21T05:39:22.826767+0000","last_unstale":"2024-09-21T05:39:24.225595+0000","last_undegraded":"2024-09-21T05:39:24.225595+0000","last_fullsized":"2024-09-21T05:39:24.225595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:33.104170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827936+0000","last_change":"2024-09-21T05:39:22.827707+0000","last_active":"2024-09-21T05:39:23.827936+0000","last_peered":"2024-09-21T05:39:23.827936+0000","last_clean":"2024-09-21T05:39:23.827936+0000","last_became_active":"2024-09-21T05:39:22.827386+0000","last_became_peered":"2024-09-21T05:39:22.827386+0000","last_unstale":"2024-09-21T05:39:23.827936+0000","last_undegraded":"2024-09-21T05:39:23.827936+0000","last_fullsized":"2024-09-21T05:39:23.827936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:48.862228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098373+0000","last_change":"2024-09-21T05:39:22.835934+0000","last_active":"2024-09-21T05:39:24.098373+0000","last_peered":"2024-09-21T05:39:24.098373+0000","last_clean":"2024-09-21T05:39:24.098373+0000","last_became_active":"2024-09-21T05:39:22.835588+0000","last_became_peered":"2024-09-21T05:39:22.835588+0000","last_unstale":"2024-09-21T05:39:24.098373+0000","last_undegraded":"2024-09-21T05:39:24.098373+0000","last_fullsized":"2024-09-21T05:39:24.098373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099480+0000","last_change":"2024-09-21T05:39:22.839332+0000","last_active":"2024-09-21T05:39:24.099480+0000","last_peered":"2024-09-21T05:39:24.099480+0000","last_clean":"2024-09-21T05:39:24.099480+0000","last_became_active":"2024-09-21T05:39:22.838996+0000","last_became_peered":"2024-09-21T05:39:22.838996+0000","last_unstale":"2024-09-21T05:39:24.099480+0000","last_undegraded":"2024-09-21T05:39:24.099480+0000","last_fullsized":"2024-09-21T05:39:24.099480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:17.322114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225282+0000","last_change":"2024-09-21T05:39:22.833783+0000","last_active":"2024-09-21T05:39:24.225282+0000","last_peered":"2024-09-21T05:39:24.225282+0000","last_clean":"2024-09-21T05:39:24.225282+0000","last_became_active":"2024-09-21T05:39:22.833447+0000","last_became_peered":"2024-09-21T05:39:22.833447+0000","last_unstale":"2024-09-21T05:39:24.225282+0000","last_undegraded":"2024-09-21T05:39:24.225282+0000","last_fullsized":"2024-09-21T05:39:24.225282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:33.466410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226338+0000","last_change":"2024-09-21T05:39:22.837584+0000","last_active":"2024-09-21T05:39:24.226338+0000","last_peered":"2024-09-21T05:39:24.226338+0000","last_clean":"2024-09-21T05:39:24.226338+0000","last_became_active":"2024-09-21T05:39:22.837243+0000","last_became_peered":"2024-09-21T05:39:22.837243+0000","last_unstale":"2024-09-21T05:39:24.226338+0000","last_undegraded":"2024-09-21T05:39:24.226338+0000","last_fullsized":"2024-09-21T05:39:24.226338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:26:46.224414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829591+0000","last_change":"2024-09-21T05:39:22.820628+0000","last_active":"2024-09-21T05:39:23.829591+0000","last_peered":"2024-09-21T05:39:23.829591+0000","last_clean":"2024-09-21T05:39:23.829591+0000","last_became_active":"2024-09-21T05:39:22.820187+0000","last_became_peered":"2024-09-21T05:39:22.820187+0000","last_unstale":"2024-09-21T05:39:23.829591+0000","last_undegraded":"2024-09-21T05:39:23.829591+0000","last_fullsized":"2024-09-21T05:39:23.829591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225824+0000","last_change":"2024-09-21T05:39:22.843566+0000","last_active":"2024-09-21T05:39:24.225824+0000","last_peered":"2024-09-21T05:39:24.225824+0000","last_clean":"2024-09-21T05:39:24.225824+0000","last_became_active":"2024-09-21T05:39:22.843220+0000","last_became_peered":"2024-09-21T05:39:22.843220+0000","last_unstale":"2024-09-21T05:39:24.225824+0000","last_undegraded":"2024-09-21T05:39:24.225824+0000","last_fullsized":"2024-09-21T05:39:24.225824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098548+0000","last_change":"2024-09-21T05:39:22.833899+0000","last_active":"2024-09-21T05:39:24.098548+0000","last_peered":"2024-09-21T05:39:24.098548+0000","last_clean":"2024-09-21T05:39:24.098548+0000","last_became_active":"2024-09-21T05:39:22.833573+0000","last_became_peered":"2024-09-21T05:39:22.833573+0000","last_unstale":"2024-09-21T05:39:24.098548+0000","last_undegraded":"2024-09-21T05:39:24.098548+0000","last_fullsized":"2024-09-21T05:39:24.098548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179770+0000","last_change":"2024-09-21T05:39:22.836999+0000","last_active":"2024-09-21T05:39:24.179770+0000","last_peered":"2024-09-21T05:39:24.179770+0000","last_clean":"2024-09-21T05:39:24.179770+0000","last_became_active":"2024-09-21T05:39:22.836450+0000","last_became_peered":"2024-09-21T05:39:22.836450+0000","last_unstale":"2024-09-21T05:39:24.179770+0000","last_undegraded":"2024-09-21T05:39:24.179770+0000","last_fullsized":"2024-09-21T05:39:24.179770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:04.144295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098448+0000","last_change":"2024-09-21T05:39:22.839783+0000","last_active":"2024-09-21T05:39:24.098448+0000","last_peered":"2024-09-21T05:39:24.098448+0000","last_clean":"2024-09-21T05:39:24.098448+0000","last_became_active":"2024-09-21T05:39:22.839153+0000","last_became_peered":"2024-09-21T05:39:22.839153+0000","last_unstale":"2024-09-21T05:39:24.098448+0000","last_undegraded":"2024-09-21T05:39:24.098448+0000","last_fullsized":"2024-09-21T05:39:24.098448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180129+0000","last_change":"2024-09-21T05:39:22.835443+0000","last_active":"2024-09-21T05:39:24.180129+0000","last_peered":"2024-09-21T05:39:24.180129+0000","last_clean":"2024-09-21T05:39:24.180129+0000","last_became_active":"2024-09-21T05:39:22.835084+0000","last_became_peered":"2024-09-21T05:39:22.835084+0000","last_unstale":"2024-09-21T05:39:24.180129+0000","last_undegraded":"2024-09-21T05:39:24.180129+0000","last_fullsized":"2024-09-21T05:39:24.180129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:34.337288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830167+0000","last_change":"2024-09-21T05:39:22.818885+0000","last_active":"2024-09-21T05:39:23.830167+0000","last_peered":"2024-09-21T05:39:23.830167+0000","last_clean":"2024-09-21T05:39:23.830167+0000","last_became_active":"2024-09-21T05:39:22.818513+0000","last_became_peered":"2024-09-21T05:39:22.818513+0000","last_unstale":"2024-09-21T05:39:23.830167+0000","last_undegraded":"2024-09-21T05:39:23.830167+0000","last_fullsized":"2024-09-21T05:39:23.830167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828239+0000","last_change":"2024-09-21T05:39:22.822287+0000","last_active":"2024-09-21T05:39:23.828239+0000","last_peered":"2024-09-21T05:39:23.828239+0000","last_clean":"2024-09-21T05:39:23.828239+0000","last_became_active":"2024-09-21T05:39:22.821830+0000","last_became_peered":"2024-09-21T05:39:22.821830+0000","last_unstale":"2024-09-21T05:39:23.828239+0000","last_undegraded":"2024-09-21T05:39:23.828239+0000","last_fullsized":"2024-09-21T05:39:23.828239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.093796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180382+0000","last_change":"2024-09-21T05:39:22.837014+0000","last_active":"2024-09-21T05:39:24.180382+0000","last_peered":"2024-09-21T05:39:24.180382+0000","last_clean":"2024-09-21T05:39:24.180382+0000","last_became_active":"2024-09-21T05:39:22.836611+0000","last_became_peered":"2024-09-21T05:39:22.836611+0000","last_unstale":"2024-09-21T05:39:24.180382+0000","last_undegraded":"2024-09-21T05:39:24.180382+0000","last_fullsized":"2024-09-21T05:39:24.180382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225876+0000","last_change":"2024-09-21T05:39:22.829730+0000","last_active":"2024-09-21T05:39:24.225876+0000","last_peered":"2024-09-21T05:39:24.225876+0000","last_clean":"2024-09-21T05:39:24.225876+0000","last_became_active":"2024-09-21T05:39:22.829409+0000","last_became_peered":"2024-09-21T05:39:22.829409+0000","last_unstale":"2024-09-21T05:39:24.225876+0000","last_undegraded":"2024-09-21T05:39:24.225876+0000","last_fullsized":"2024-09-21T05:39:24.225876+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:03.851352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099187+0000","last_change":"2024-09-21T05:39:22.831806+0000","last_active":"2024-09-21T05:39:24.099187+0000","last_peered":"2024-09-21T05:39:24.099187+0000","last_clean":"2024-09-21T05:39:24.099187+0000","last_became_active":"2024-09-21T05:39:22.831323+0000","last_became_peered":"2024-09-21T05:39:22.831323+0000","last_unstale":"2024-09-21T05:39:24.099187+0000","last_undegraded":"2024-09-21T05:39:24.099187+0000","last_fullsized":"2024-09-21T05:39:24.099187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:52:01.850133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225161+0000","last_change":"2024-09-21T05:39:22.825392+0000","last_active":"2024-09-21T05:39:24.225161+0000","last_peered":"2024-09-21T05:39:24.225161+0000","last_clean":"2024-09-21T05:39:24.225161+0000","last_became_active":"2024-09-21T05:39:22.824959+0000","last_became_peered":"2024-09-21T05:39:22.824959+0000","last_unstale":"2024-09-21T05:39:24.225161+0000","last_undegraded":"2024-09-21T05:39:24.225161+0000","last_fullsized":"2024-09-21T05:39:24.225161+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:31.928406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829111+0000","last_change":"2024-09-21T05:39:22.835623+0000","last_active":"2024-09-21T05:39:23.829111+0000","last_peered":"2024-09-21T05:39:23.829111+0000","last_clean":"2024-09-21T05:39:23.829111+0000","last_became_active":"2024-09-21T05:39:22.834944+0000","last_became_peered":"2024-09-21T05:39:22.834944+0000","last_unstale":"2024-09-21T05:39:23.829111+0000","last_undegraded":"2024-09-21T05:39:23.829111+0000","last_fullsized":"2024-09-21T05:39:23.829111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:56.814069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830586+0000","last_change":"2024-09-21T05:39:22.822050+0000","last_active":"2024-09-21T05:39:23.830586+0000","last_peered":"2024-09-21T05:39:23.830586+0000","last_clean":"2024-09-21T05:39:23.830586+0000","last_became_active":"2024-09-21T05:39:22.821705+0000","last_became_peered":"2024-09-21T05:39:22.821705+0000","last_unstale":"2024-09-21T05:39:23.830586+0000","last_undegraded":"2024-09-21T05:39:23.830586+0000","last_fullsized":"2024-09-21T05:39:23.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:42.204486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180307+0000","last_change":"2024-09-21T05:39:21.823571+0000","last_active":"2024-09-21T05:39:24.180307+0000","last_peered":"2024-09-21T05:39:24.180307+0000","last_clean":"2024-09-21T05:39:24.180307+0000","last_became_active":"2024-09-21T05:39:21.823229+0000","last_became_peered":"2024-09-21T05:39:21.823229+0000","last_unstale":"2024-09-21T05:39:24.180307+0000","last_undegraded":"2024-09-21T05:39:24.180307+0000","last_fullsized":"2024-09-21T05:39:24.180307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:28.122438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226088+0000","last_change":"2024-09-21T05:39:22.832497+0000","last_active":"2024-09-21T05:39:24.226088+0000","last_peered":"2024-09-21T05:39:24.226088+0000","last_clean":"2024-09-21T05:39:24.226088+0000","last_became_active":"2024-09-21T05:39:22.831525+0000","last_became_peered":"2024-09-21T05:39:22.831525+0000","last_unstale":"2024-09-21T05:39:24.226088+0000","last_undegraded":"2024-09-21T05:39:24.226088+0000","last_fullsized":"2024-09-21T05:39:24.226088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:11.541093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829477+0000","last_change":"2024-09-21T05:39:21.830086+0000","last_active":"2024-09-21T05:39:23.829477+0000","last_peered":"2024-09-21T05:39:23.829477+0000","last_clean":"2024-09-21T05:39:23.829477+0000","last_became_active":"2024-09-21T05:39:21.829220+0000","last_became_peered":"2024-09-21T05:39:21.829220+0000","last_unstale":"2024-09-21T05:39:23.829477+0000","last_undegraded":"2024-09-21T05:39:23.829477+0000","last_fullsized":"2024-09-21T05:39:23.829477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179838+0000","last_change":"2024-09-21T05:39:21.827177+0000","last_active":"2024-09-21T05:39:24.179838+0000","last_peered":"2024-09-21T05:39:24.179838+0000","last_clean":"2024-09-21T05:39:24.179838+0000","last_became_active":"2024-09-21T05:39:21.826828+0000","last_became_peered":"2024-09-21T05:39:21.826828+0000","last_unstale":"2024-09-21T05:39:24.179838+0000","last_undegraded":"2024-09-21T05:39:24.179838+0000","last_fullsized":"2024-09-21T05:39:24.179838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.179804+0000","last_change":"2024-09-21T05:39:22.838677+0000","last_active":"2024-09-21T05:39:24.179804+0000","last_peered":"2024-09-21T05:39:24.179804+0000","last_clean":"2024-09-21T05:39:24.179804+0000","last_became_active":"2024-09-21T05:39:22.837655+0000","last_became_peered":"2024-09-21T05:39:22.837655+0000","last_unstale":"2024-09-21T05:39:24.179804+0000","last_undegraded":"2024-09-21T05:39:24.179804+0000","last_fullsized":"2024-09-21T05:39:24.179804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:35.249819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098562+0000","last_change":"2024-09-21T05:39:21.826596+0000","last_active":"2024-09-21T05:39:24.098562+0000","last_peered":"2024-09-21T05:39:24.098562+0000","last_clean":"2024-09-21T05:39:24.098562+0000","last_became_active":"2024-09-21T05:39:21.825792+0000","last_became_peered":"2024-09-21T05:39:21.825792+0000","last_unstale":"2024-09-21T05:39:24.098562+0000","last_undegraded":"2024-09-21T05:39:24.098562+0000","last_fullsized":"2024-09-21T05:39:24.098562+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:17.928160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827940+0000","last_change":"2024-09-21T05:39:22.835121+0000","last_active":"2024-09-21T05:39:23.827940+0000","last_peered":"2024-09-21T05:39:23.827940+0000","last_clean":"2024-09-21T05:39:23.827940+0000","last_became_active":"2024-09-21T05:39:22.834781+0000","last_became_peered":"2024-09-21T05:39:22.834781+0000","last_unstale":"2024-09-21T05:39:23.827940+0000","last_undegraded":"2024-09-21T05:39:23.827940+0000","last_fullsized":"2024-09-21T05:39:23.827940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:23:26.906027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830916+0000","last_change":"2024-09-21T05:39:21.829656+0000","last_active":"2024-09-21T05:39:23.830916+0000","last_peered":"2024-09-21T05:39:23.830916+0000","last_clean":"2024-09-21T05:39:23.830916+0000","last_became_active":"2024-09-21T05:39:21.829309+0000","last_became_peered":"2024-09-21T05:39:21.829309+0000","last_unstale":"2024-09-21T05:39:23.830916+0000","last_undegraded":"2024-09-21T05:39:23.830916+0000","last_fullsized":"2024-09-21T05:39:23.830916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098740+0000","last_change":"2024-09-21T05:39:22.833912+0000","last_active":"2024-09-21T05:39:24.098740+0000","last_peered":"2024-09-21T05:39:24.098740+0000","last_clean":"2024-09-21T05:39:24.098740+0000","last_became_active":"2024-09-21T05:39:22.833586+0000","last_became_peered":"2024-09-21T05:39:22.833586+0000","last_unstale":"2024-09-21T05:39:24.098740+0000","last_undegraded":"2024-09-21T05:39:24.098740+0000","last_fullsized":"2024-09-21T05:39:24.098740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:48.003768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099230+0000","last_change":"2024-09-21T05:39:21.831932+0000","last_active":"2024-09-21T05:39:24.099230+0000","last_peered":"2024-09-21T05:39:24.099230+0000","last_clean":"2024-09-21T05:39:24.099230+0000","last_became_active":"2024-09-21T05:39:21.831625+0000","last_became_peered":"2024-09-21T05:39:21.831625+0000","last_unstale":"2024-09-21T05:39:24.099230+0000","last_undegraded":"2024-09-21T05:39:24.099230+0000","last_fullsized":"2024-09-21T05:39:24.099230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225959+0000","last_change":"2024-09-21T05:39:22.837596+0000","last_active":"2024-09-21T05:39:24.225959+0000","last_peered":"2024-09-21T05:39:24.225959+0000","last_clean":"2024-09-21T05:39:24.225959+0000","last_became_active":"2024-09-21T05:39:22.837271+0000","last_became_peered":"2024-09-21T05:39:22.837271+0000","last_unstale":"2024-09-21T05:39:24.225959+0000","last_undegraded":"2024-09-21T05:39:24.225959+0000","last_fullsized":"2024-09-21T05:39:24.225959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828162+0000","last_change":"2024-09-21T05:39:21.827640+0000","last_active":"2024-09-21T05:39:23.828162+0000","last_peered":"2024-09-21T05:39:23.828162+0000","last_clean":"2024-09-21T05:39:23.828162+0000","last_became_active":"2024-09-21T05:39:21.825961+0000","last_became_peered":"2024-09-21T05:39:21.825961+0000","last_unstale":"2024-09-21T05:39:23.828162+0000","last_undegraded":"2024-09-21T05:39:23.828162+0000","last_fullsized":"2024-09-21T05:39:23.828162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:55.395645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.831067+0000","last_change":"2024-09-21T05:39:22.820929+0000","last_active":"2024-09-21T05:39:23.831067+0000","last_peered":"2024-09-21T05:39:23.831067+0000","last_clean":"2024-09-21T05:39:23.831067+0000","last_became_active":"2024-09-21T05:39:22.820587+0000","last_became_peered":"2024-09-21T05:39:22.820587+0000","last_unstale":"2024-09-21T05:39:23.831067+0000","last_undegraded":"2024-09-21T05:39:23.831067+0000","last_fullsized":"2024-09-21T05:39:23.831067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:42.698282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829673+0000","last_change":"2024-09-21T05:39:21.831555+0000","last_active":"2024-09-21T05:39:23.829673+0000","last_peered":"2024-09-21T05:39:23.829673+0000","last_clean":"2024-09-21T05:39:23.829673+0000","last_became_active":"2024-09-21T05:39:21.831127+0000","last_became_peered":"2024-09-21T05:39:21.831127+0000","last_unstale":"2024-09-21T05:39:23.829673+0000","last_undegraded":"2024-09-21T05:39:23.829673+0000","last_fullsized":"2024-09-21T05:39:23.829673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:37.429773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226051+0000","last_change":"2024-09-21T05:39:22.838353+0000","last_active":"2024-09-21T05:39:24.226051+0000","last_peered":"2024-09-21T05:39:24.226051+0000","last_clean":"2024-09-21T05:39:24.226051+0000","last_became_active":"2024-09-21T05:39:22.838044+0000","last_became_peered":"2024-09-21T05:39:22.838044+0000","last_unstale":"2024-09-21T05:39:24.226051+0000","last_undegraded":"2024-09-21T05:39:24.226051+0000","last_fullsized":"2024-09-21T05:39:24.226051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:27.463085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226025+0000","last_change":"2024-09-21T05:39:21.826178+0000","last_active":"2024-09-21T05:39:24.226025+0000","last_peered":"2024-09-21T05:39:24.226025+0000","last_clean":"2024-09-21T05:39:24.226025+0000","last_became_active":"2024-09-21T05:39:21.825740+0000","last_became_peered":"2024-09-21T05:39:21.825740+0000","last_unstale":"2024-09-21T05:39:24.226025+0000","last_undegraded":"2024-09-21T05:39:24.226025+0000","last_fullsized":"2024-09-21T05:39:24.226025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:39.408480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225626+0000","last_change":"2024-09-21T05:39:22.841735+0000","last_active":"2024-09-21T05:39:24.225626+0000","last_peered":"2024-09-21T05:39:24.225626+0000","last_clean":"2024-09-21T05:39:24.225626+0000","last_became_active":"2024-09-21T05:39:22.841396+0000","last_became_peered":"2024-09-21T05:39:22.841396+0000","last_unstale":"2024-09-21T05:39:24.225626+0000","last_undegraded":"2024-09-21T05:39:24.225626+0000","last_fullsized":"2024-09-21T05:39:24.225626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:04.153620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098784+0000","last_change":"2024-09-21T05:39:21.824947+0000","last_active":"2024-09-21T05:39:24.098784+0000","last_peered":"2024-09-21T05:39:24.098784+0000","last_clean":"2024-09-21T05:39:24.098784+0000","last_became_active":"2024-09-21T05:39:21.824607+0000","last_became_peered":"2024-09-21T05:39:21.824607+0000","last_unstale":"2024-09-21T05:39:24.098784+0000","last_undegraded":"2024-09-21T05:39:24.098784+0000","last_fullsized":"2024-09-21T05:39:24.098784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:24.875470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225891+0000","last_change":"2024-09-21T05:39:22.833336+0000","last_active":"2024-09-21T05:39:24.225891+0000","last_peered":"2024-09-21T05:39:24.225891+0000","last_clean":"2024-09-21T05:39:24.225891+0000","last_became_active":"2024-09-21T05:39:22.832959+0000","last_became_peered":"2024-09-21T05:39:22.832959+0000","last_unstale":"2024-09-21T05:39:24.225891+0000","last_undegraded":"2024-09-21T05:39:24.225891+0000","last_fullsized":"2024-09-21T05:39:24.225891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:49.450059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829277+0000","last_change":"2024-09-21T05:39:21.833323+0000","last_active":"2024-09-21T05:39:23.829277+0000","last_peered":"2024-09-21T05:39:23.829277+0000","last_clean":"2024-09-21T05:39:23.829277+0000","last_became_active":"2024-09-21T05:39:21.832881+0000","last_became_peered":"2024-09-21T05:39:21.832881+0000","last_unstale":"2024-09-21T05:39:23.829277+0000","last_undegraded":"2024-09-21T05:39:23.829277+0000","last_fullsized":"2024-09-21T05:39:23.829277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829845+0000","last_change":"2024-09-21T05:39:22.835947+0000","last_active":"2024-09-21T05:39:23.829845+0000","last_peered":"2024-09-21T05:39:23.829845+0000","last_clean":"2024-09-21T05:39:23.829845+0000","last_became_active":"2024-09-21T05:39:22.835332+0000","last_became_peered":"2024-09-21T05:39:22.835332+0000","last_unstale":"2024-09-21T05:39:23.829845+0000","last_undegraded":"2024-09-21T05:39:23.829845+0000","last_fullsized":"2024-09-21T05:39:23.829845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:45.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.224899+0000","last_change":"2024-09-21T05:39:21.819239+0000","last_active":"2024-09-21T05:39:24.224899+0000","last_peered":"2024-09-21T05:39:24.224899+0000","last_clean":"2024-09-21T05:39:24.224899+0000","last_became_active":"2024-09-21T05:39:21.818811+0000","last_became_peered":"2024-09-21T05:39:21.818811+0000","last_unstale":"2024-09-21T05:39:24.224899+0000","last_undegraded":"2024-09-21T05:39:24.224899+0000","last_fullsized":"2024-09-21T05:39:24.224899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:45.833179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225617+0000","last_change":"2024-09-21T05:39:22.834980+0000","last_active":"2024-09-21T05:39:24.225617+0000","last_peered":"2024-09-21T05:39:24.225617+0000","last_clean":"2024-09-21T05:39:24.225617+0000","last_became_active":"2024-09-21T05:39:22.834075+0000","last_became_peered":"2024-09-21T05:39:22.834075+0000","last_unstale":"2024-09-21T05:39:24.225617+0000","last_undegraded":"2024-09-21T05:39:24.225617+0000","last_fullsized":"2024-09-21T05:39:24.225617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:56:31.814988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830803+0000","last_change":"2024-09-21T05:39:21.824655+0000","last_active":"2024-09-21T05:39:23.830803+0000","last_peered":"2024-09-21T05:39:23.830803+0000","last_clean":"2024-09-21T05:39:23.830803+0000","last_became_active":"2024-09-21T05:39:21.824309+0000","last_became_peered":"2024-09-21T05:39:21.824309+0000","last_unstale":"2024-09-21T05:39:23.830803+0000","last_undegraded":"2024-09-21T05:39:23.830803+0000","last_fullsized":"2024-09-21T05:39:23.830803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:54:50.559527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226272+0000","last_change":"2024-09-21T05:39:22.830496+0000","last_active":"2024-09-21T05:39:24.226272+0000","last_peered":"2024-09-21T05:39:24.226272+0000","last_clean":"2024-09-21T05:39:24.226272+0000","last_became_active":"2024-09-21T05:39:22.829859+0000","last_became_peered":"2024-09-21T05:39:22.829859+0000","last_unstale":"2024-09-21T05:39:24.226272+0000","last_undegraded":"2024-09-21T05:39:24.226272+0000","last_fullsized":"2024-09-21T05:39:24.226272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:09.763411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225284+0000","last_change":"2024-09-21T05:39:21.821434+0000","last_active":"2024-09-21T05:39:24.225284+0000","last_peered":"2024-09-21T05:39:24.225284+0000","last_clean":"2024-09-21T05:39:24.225284+0000","last_became_active":"2024-09-21T05:39:21.820776+0000","last_became_peered":"2024-09-21T05:39:21.820776+0000","last_unstale":"2024-09-21T05:39:24.225284+0000","last_undegraded":"2024-09-21T05:39:24.225284+0000","last_fullsized":"2024-09-21T05:39:24.225284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:36.521472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226478+0000","last_change":"2024-09-21T05:39:22.837101+0000","last_active":"2024-09-21T05:39:24.226478+0000","last_peered":"2024-09-21T05:39:24.226478+0000","last_clean":"2024-09-21T05:39:24.226478+0000","last_became_active":"2024-09-21T05:39:22.834638+0000","last_became_peered":"2024-09-21T05:39:22.834638+0000","last_unstale":"2024-09-21T05:39:24.226478+0000","last_undegraded":"2024-09-21T05:39:24.226478+0000","last_fullsized":"2024-09-21T05:39:24.226478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:37.717004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828293+0000","last_change":"2024-09-21T05:39:21.828822+0000","last_active":"2024-09-21T05:39:23.828293+0000","last_peered":"2024-09-21T05:39:23.828293+0000","last_clean":"2024-09-21T05:39:23.828293+0000","last_became_active":"2024-09-21T05:39:21.825429+0000","last_became_peered":"2024-09-21T05:39:21.825429+0000","last_unstale":"2024-09-21T05:39:23.828293+0000","last_undegraded":"2024-09-21T05:39:23.828293+0000","last_fullsized":"2024-09-21T05:39:23.828293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:41.198943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099322+0000","last_change":"2024-09-21T05:39:22.840046+0000","last_active":"2024-09-21T05:39:24.099322+0000","last_peered":"2024-09-21T05:39:24.099322+0000","last_clean":"2024-09-21T05:39:24.099322+0000","last_became_active":"2024-09-21T05:39:22.839635+0000","last_became_peered":"2024-09-21T05:39:22.839635+0000","last_unstale":"2024-09-21T05:39:24.099322+0000","last_undegraded":"2024-09-21T05:39:24.099322+0000","last_fullsized":"2024-09-21T05:39:24.099322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:41.782301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225194+0000","last_change":"2024-09-21T05:39:21.823827+0000","last_active":"2024-09-21T05:39:24.225194+0000","last_peered":"2024-09-21T05:39:24.225194+0000","last_clean":"2024-09-21T05:39:24.225194+0000","last_became_active":"2024-09-21T05:39:21.823408+0000","last_became_peered":"2024-09-21T05:39:21.823408+0000","last_unstale":"2024-09-21T05:39:24.225194+0000","last_undegraded":"2024-09-21T05:39:24.225194+0000","last_fullsized":"2024-09-21T05:39:24.225194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:08:31.079319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225997+0000","last_change":"2024-09-21T05:39:22.828178+0000","last_active":"2024-09-21T05:39:24.225997+0000","last_peered":"2024-09-21T05:39:24.225997+0000","last_clean":"2024-09-21T05:39:24.225997+0000","last_became_active":"2024-09-21T05:39:22.827833+0000","last_became_peered":"2024-09-21T05:39:22.827833+0000","last_unstale":"2024-09-21T05:39:24.225997+0000","last_undegraded":"2024-09-21T05:39:24.225997+0000","last_fullsized":"2024-09-21T05:39:24.225997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:58.078709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225514+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:24.225514+0000","last_peered":"2024-09-21T05:39:24.225514+0000","last_clean":"2024-09-21T05:39:24.225514+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:24.225514+0000","last_undegraded":"2024-09-21T05:39:24.225514+0000","last_fullsized":"2024-09-21T05:39:24.225514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098343+0000","last_change":"2024-09-21T05:39:21.824181+0000","last_active":"2024-09-21T05:39:24.098343+0000","last_peered":"2024-09-21T05:39:24.098343+0000","last_clean":"2024-09-21T05:39:24.098343+0000","last_became_active":"2024-09-21T05:39:21.823710+0000","last_became_peered":"2024-09-21T05:39:21.823710+0000","last_unstale":"2024-09-21T05:39:24.098343+0000","last_undegraded":"2024-09-21T05:39:24.098343+0000","last_fullsized":"2024-09-21T05:39:24.098343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:37.294612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180058+0000","last_change":"2024-09-21T05:39:22.843587+0000","last_active":"2024-09-21T05:39:24.180058+0000","last_peered":"2024-09-21T05:39:24.180058+0000","last_clean":"2024-09-21T05:39:24.180058+0000","last_became_active":"2024-09-21T05:39:22.843247+0000","last_became_peered":"2024-09-21T05:39:22.843247+0000","last_unstale":"2024-09-21T05:39:24.180058+0000","last_undegraded":"2024-09-21T05:39:24.180058+0000","last_fullsized":"2024-09-21T05:39:24.180058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:36.511157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225383+0000","last_change":"2024-09-21T05:39:21.833248+0000","last_active":"2024-09-21T05:39:24.225383+0000","last_peered":"2024-09-21T05:39:24.225383+0000","last_clean":"2024-09-21T05:39:24.225383+0000","last_became_active":"2024-09-21T05:39:21.832873+0000","last_became_peered":"2024-09-21T05:39:21.832873+0000","last_unstale":"2024-09-21T05:39:24.225383+0000","last_undegraded":"2024-09-21T05:39:24.225383+0000","last_fullsized":"2024-09-21T05:39:24.225383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:29.420862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225527+0000","last_change":"2024-09-21T05:39:22.832001+0000","last_active":"2024-09-21T05:39:24.225527+0000","last_peered":"2024-09-21T05:39:24.225527+0000","last_clean":"2024-09-21T05:39:24.225527+0000","last_became_active":"2024-09-21T05:39:22.831653+0000","last_became_peered":"2024-09-21T05:39:22.831653+0000","last_unstale":"2024-09-21T05:39:24.225527+0000","last_undegraded":"2024-09-21T05:39:24.225527+0000","last_fullsized":"2024-09-21T05:39:24.225527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:54.378593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225070+0000","last_change":"2024-09-21T05:39:21.821394+0000","last_active":"2024-09-21T05:39:24.225070+0000","last_peered":"2024-09-21T05:39:24.225070+0000","last_clean":"2024-09-21T05:39:24.225070+0000","last_became_active":"2024-09-21T05:39:21.820818+0000","last_became_peered":"2024-09-21T05:39:21.820818+0000","last_unstale":"2024-09-21T05:39:24.225070+0000","last_undegraded":"2024-09-21T05:39:24.225070+0000","last_fullsized":"2024-09-21T05:39:24.225070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:30.692448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.827925+0000","last_change":"2024-09-21T05:39:22.823276+0000","last_active":"2024-09-21T05:39:23.827925+0000","last_peered":"2024-09-21T05:39:23.827925+0000","last_clean":"2024-09-21T05:39:23.827925+0000","last_became_active":"2024-09-21T05:39:22.822887+0000","last_became_peered":"2024-09-21T05:39:22.822887+0000","last_unstale":"2024-09-21T05:39:23.827925+0000","last_undegraded":"2024-09-21T05:39:23.827925+0000","last_fullsized":"2024-09-21T05:39:23.827925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:24.744307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099295+0000","last_change":"2024-09-21T05:39:21.830642+0000","last_active":"2024-09-21T05:39:24.099295+0000","last_peered":"2024-09-21T05:39:24.099295+0000","last_clean":"2024-09-21T05:39:24.099295+0000","last_became_active":"2024-09-21T05:39:21.830329+0000","last_became_peered":"2024-09-21T05:39:21.830329+0000","last_unstale":"2024-09-21T05:39:24.099295+0000","last_undegraded":"2024-09-21T05:39:24.099295+0000","last_fullsized":"2024-09-21T05:39:24.099295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830594+0000","last_change":"2024-09-21T05:39:22.831920+0000","last_active":"2024-09-21T05:39:23.830594+0000","last_peered":"2024-09-21T05:39:23.830594+0000","last_clean":"2024-09-21T05:39:23.830594+0000","last_became_active":"2024-09-21T05:39:22.831421+0000","last_became_peered":"2024-09-21T05:39:22.831421+0000","last_unstale":"2024-09-21T05:39:23.830594+0000","last_undegraded":"2024-09-21T05:39:23.830594+0000","last_fullsized":"2024-09-21T05:39:23.830594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:56.775839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828257+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:23.828257+0000","last_peered":"2024-09-21T05:39:23.828257+0000","last_clean":"2024-09-21T05:39:23.828257+0000","last_became_active":"2024-09-21T05:39:21.827102+0000","last_became_peered":"2024-09-21T05:39:21.827102+0000","last_unstale":"2024-09-21T05:39:23.828257+0000","last_undegraded":"2024-09-21T05:39:23.828257+0000","last_fullsized":"2024-09-21T05:39:23.828257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:10.824073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099174+0000","last_change":"2024-09-21T05:39:22.840475+0000","last_active":"2024-09-21T05:39:24.099174+0000","last_peered":"2024-09-21T05:39:24.099174+0000","last_clean":"2024-09-21T05:39:24.099174+0000","last_became_active":"2024-09-21T05:39:22.840169+0000","last_became_peered":"2024-09-21T05:39:22.840169+0000","last_unstale":"2024-09-21T05:39:24.099174+0000","last_undegraded":"2024-09-21T05:39:24.099174+0000","last_fullsized":"2024-09-21T05:39:24.099174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:35.382143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226188+0000","last_change":"2024-09-21T05:39:21.831044+0000","last_active":"2024-09-21T05:39:24.226188+0000","last_peered":"2024-09-21T05:39:24.226188+0000","last_clean":"2024-09-21T05:39:24.226188+0000","last_became_active":"2024-09-21T05:39:21.830412+0000","last_became_peered":"2024-09-21T05:39:21.830412+0000","last_unstale":"2024-09-21T05:39:24.226188+0000","last_undegraded":"2024-09-21T05:39:24.226188+0000","last_fullsized":"2024-09-21T05:39:24.226188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:36.640541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828152+0000","last_change":"2024-09-21T05:39:22.824145+0000","last_active":"2024-09-21T05:39:23.828152+0000","last_peered":"2024-09-21T05:39:23.828152+0000","last_clean":"2024-09-21T05:39:23.828152+0000","last_became_active":"2024-09-21T05:39:22.823776+0000","last_became_peered":"2024-09-21T05:39:22.823776+0000","last_unstale":"2024-09-21T05:39:23.828152+0000","last_undegraded":"2024-09-21T05:39:23.828152+0000","last_fullsized":"2024-09-21T05:39:23.828152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:25:01.425945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.815471+0000","last_change":"2024-09-21T05:39:21.828699+0000","last_active":"2024-09-21T05:39:24.815471+0000","last_peered":"2024-09-21T05:39:24.815471+0000","last_clean":"2024-09-21T05:39:24.815471+0000","last_became_active":"2024-09-21T05:39:21.828330+0000","last_became_peered":"2024-09-21T05:39:21.828330+0000","last_unstale":"2024-09-21T05:39:24.815471+0000","last_undegraded":"2024-09-21T05:39:24.815471+0000","last_fullsized":"2024-09-21T05:39:24.815471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:55.770297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180312+0000","last_change":"2024-09-21T05:39:22.841936+0000","last_active":"2024-09-21T05:39:24.180312+0000","last_peered":"2024-09-21T05:39:24.180312+0000","last_clean":"2024-09-21T05:39:24.180312+0000","last_became_active":"2024-09-21T05:39:22.841601+0000","last_became_peered":"2024-09-21T05:39:22.841601+0000","last_unstale":"2024-09-21T05:39:24.180312+0000","last_undegraded":"2024-09-21T05:39:24.180312+0000","last_fullsized":"2024-09-21T05:39:24.180312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.830270+0000","last_change":"2024-09-21T05:39:21.834507+0000","last_active":"2024-09-21T05:39:23.830270+0000","last_peered":"2024-09-21T05:39:23.830270+0000","last_clean":"2024-09-21T05:39:23.830270+0000","last_became_active":"2024-09-21T05:39:21.834148+0000","last_became_peered":"2024-09-21T05:39:21.834148+0000","last_unstale":"2024-09-21T05:39:23.830270+0000","last_undegraded":"2024-09-21T05:39:23.830270+0000","last_fullsized":"2024-09-21T05:39:23.830270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:03.660025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225981+0000","last_change":"2024-09-21T05:39:22.826846+0000","last_active":"2024-09-21T05:39:24.225981+0000","last_peered":"2024-09-21T05:39:24.225981+0000","last_clean":"2024-09-21T05:39:24.225981+0000","last_became_active":"2024-09-21T05:39:22.825449+0000","last_became_peered":"2024-09-21T05:39:22.825449+0000","last_unstale":"2024-09-21T05:39:24.225981+0000","last_undegraded":"2024-09-21T05:39:24.225981+0000","last_fullsized":"2024-09-21T05:39:24.225981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:30.254009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225380+0000","last_change":"2024-09-21T05:39:21.833615+0000","last_active":"2024-09-21T05:39:24.225380+0000","last_peered":"2024-09-21T05:39:24.225380+0000","last_clean":"2024-09-21T05:39:24.225380+0000","last_became_active":"2024-09-21T05:39:21.833028+0000","last_became_peered":"2024-09-21T05:39:21.833028+0000","last_unstale":"2024-09-21T05:39:24.225380+0000","last_undegraded":"2024-09-21T05:39:24.225380+0000","last_fullsized":"2024-09-21T05:39:24.225380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:28.358924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829777+0000","last_change":"2024-09-21T05:39:22.827047+0000","last_active":"2024-09-21T05:39:23.829777+0000","last_peered":"2024-09-21T05:39:23.829777+0000","last_clean":"2024-09-21T05:39:23.829777+0000","last_became_active":"2024-09-21T05:39:22.825920+0000","last_became_peered":"2024-09-21T05:39:22.825920+0000","last_unstale":"2024-09-21T05:39:23.829777+0000","last_undegraded":"2024-09-21T05:39:23.829777+0000","last_fullsized":"2024-09-21T05:39:23.829777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:56.401187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828051+0000","last_change":"2024-09-21T05:39:21.824232+0000","last_active":"2024-09-21T05:39:23.828051+0000","last_peered":"2024-09-21T05:39:23.828051+0000","last_clean":"2024-09-21T05:39:23.828051+0000","last_became_active":"2024-09-21T05:39:21.823849+0000","last_became_peered":"2024-09-21T05:39:21.823849+0000","last_unstale":"2024-09-21T05:39:23.828051+0000","last_undegraded":"2024-09-21T05:39:23.828051+0000","last_fullsized":"2024-09-21T05:39:23.828051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:08.636418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180080+0000","last_change":"2024-09-21T05:39:22.836324+0000","last_active":"2024-09-21T05:39:24.180080+0000","last_peered":"2024-09-21T05:39:24.180080+0000","last_clean":"2024-09-21T05:39:24.180080+0000","last_became_active":"2024-09-21T05:39:22.835950+0000","last_became_peered":"2024-09-21T05:39:22.835950+0000","last_unstale":"2024-09-21T05:39:24.180080+0000","last_undegraded":"2024-09-21T05:39:24.180080+0000","last_fullsized":"2024-09-21T05:39:24.180080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:42.672915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.828025+0000","last_change":"2024-09-21T05:39:21.828537+0000","last_active":"2024-09-21T05:39:23.828025+0000","last_peered":"2024-09-21T05:39:23.828025+0000","last_clean":"2024-09-21T05:39:23.828025+0000","last_became_active":"2024-09-21T05:39:21.828218+0000","last_became_peered":"2024-09-21T05:39:21.828218+0000","last_unstale":"2024-09-21T05:39:23.828025+0000","last_undegraded":"2024-09-21T05:39:23.828025+0000","last_fullsized":"2024-09-21T05:39:23.828025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:45:44.171042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180194+0000","last_change":"2024-09-21T05:39:22.833172+0000","last_active":"2024-09-21T05:39:24.180194+0000","last_peered":"2024-09-21T05:39:24.180194+0000","last_clean":"2024-09-21T05:39:24.180194+0000","last_became_active":"2024-09-21T05:39:22.832820+0000","last_became_peered":"2024-09-21T05:39:22.832820+0000","last_unstale":"2024-09-21T05:39:24.180194+0000","last_undegraded":"2024-09-21T05:39:24.180194+0000","last_fullsized":"2024-09-21T05:39:24.180194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:54.408926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226082+0000","last_change":"2024-09-21T05:39:21.829961+0000","last_active":"2024-09-21T05:39:24.226082+0000","last_peered":"2024-09-21T05:39:24.226082+0000","last_clean":"2024-09-21T05:39:24.226082+0000","last_became_active":"2024-09-21T05:39:21.829484+0000","last_became_peered":"2024-09-21T05:39:21.829484+0000","last_unstale":"2024-09-21T05:39:24.226082+0000","last_undegraded":"2024-09-21T05:39:24.226082+0000","last_fullsized":"2024-09-21T05:39:24.226082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:16.446491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:23.829396+0000","last_change":"2024-09-21T05:39:22.820710+0000","last_active":"2024-09-21T05:39:23.829396+0000","last_peered":"2024-09-21T05:39:23.829396+0000","last_clean":"2024-09-21T05:39:23.829396+0000","last_became_active":"2024-09-21T05:39:22.820283+0000","last_became_peered":"2024-09-21T05:39:22.820283+0000","last_unstale":"2024-09-21T05:39:23.829396+0000","last_undegraded":"2024-09-21T05:39:23.829396+0000","last_fullsized":"2024-09-21T05:39:23.829396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:38.841858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.099220+0000","last_change":"2024-09-21T05:39:22.830616+0000","last_active":"2024-09-21T05:39:24.099220+0000","last_peered":"2024-09-21T05:39:24.099220+0000","last_clean":"2024-09-21T05:39:24.099220+0000","last_became_active":"2024-09-21T05:39:22.830300+0000","last_became_peered":"2024-09-21T05:39:22.830300+0000","last_unstale":"2024-09-21T05:39:24.099220+0000","last_undegraded":"2024-09-21T05:39:24.099220+0000","last_fullsized":"2024-09-21T05:39:24.099220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:24.955220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225816+0000","last_change":"2024-09-21T05:39:21.830335+0000","last_active":"2024-09-21T05:39:24.225816+0000","last_peered":"2024-09-21T05:39:24.225816+0000","last_clean":"2024-09-21T05:39:24.225816+0000","last_became_active":"2024-09-21T05:39:21.829550+0000","last_became_peered":"2024-09-21T05:39:21.829550+0000","last_unstale":"2024-09-21T05:39:24.225816+0000","last_undegraded":"2024-09-21T05:39:24.225816+0000","last_fullsized":"2024-09-21T05:39:24.225816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:51.148096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.098657+0000","last_change":"2024-09-21T05:39:22.828643+0000","last_active":"2024-09-21T05:39:24.098657+0000","last_peered":"2024-09-21T05:39:24.098657+0000","last_clean":"2024-09-21T05:39:24.098657+0000","last_became_active":"2024-09-21T05:39:22.828285+0000","last_became_peered":"2024-09-21T05:39:22.828285+0000","last_unstale":"2024-09-21T05:39:24.098657+0000","last_undegraded":"2024-09-21T05:39:24.098657+0000","last_fullsized":"2024-09-21T05:39:24.098657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:30.943550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226090+0000","last_change":"2024-09-21T05:39:21.830622+0000","last_active":"2024-09-21T05:39:24.226090+0000","last_peered":"2024-09-21T05:39:24.226090+0000","last_clean":"2024-09-21T05:39:24.226090+0000","last_became_active":"2024-09-21T05:39:21.829994+0000","last_became_peered":"2024-09-21T05:39:21.829994+0000","last_unstale":"2024-09-21T05:39:24.226090+0000","last_undegraded":"2024-09-21T05:39:24.226090+0000","last_fullsized":"2024-09-21T05:39:24.226090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:41.692740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.225955+0000","last_change":"2024-09-21T05:39:22.839590+0000","last_active":"2024-09-21T05:39:24.225955+0000","last_peered":"2024-09-21T05:39:24.225955+0000","last_clean":"2024-09-21T05:39:24.225955+0000","last_became_active":"2024-09-21T05:39:22.839256+0000","last_became_peered":"2024-09-21T05:39:22.839256+0000","last_unstale":"2024-09-21T05:39:24.225955+0000","last_undegraded":"2024-09-21T05:39:24.225955+0000","last_fullsized":"2024-09-21T05:39:24.225955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:47:19.873556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180020+0000","last_change":"2024-09-21T05:39:21.828673+0000","last_active":"2024-09-21T05:39:24.180020+0000","last_peered":"2024-09-21T05:39:24.180020+0000","last_clean":"2024-09-21T05:39:24.180020+0000","last_became_active":"2024-09-21T05:39:21.828338+0000","last_became_peered":"2024-09-21T05:39:21.828338+0000","last_unstale":"2024-09-21T05:39:24.180020+0000","last_undegraded":"2024-09-21T05:39:24.180020+0000","last_fullsized":"2024-09-21T05:39:24.180020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:17.200153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.180243+0000","last_change":"2024-09-21T05:39:22.840586+0000","last_active":"2024-09-21T05:39:24.180243+0000","last_peered":"2024-09-21T05:39:24.180243+0000","last_clean":"2024-09-21T05:39:24.180243+0000","last_became_active":"2024-09-21T05:39:22.840272+0000","last_became_peered":"2024-09-21T05:39:22.840272+0000","last_unstale":"2024-09-21T05:39:24.180243+0000","last_undegraded":"2024-09-21T05:39:24.180243+0000","last_fullsized":"2024-09-21T05:39:24.180243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:21.797734+0000","last_clean_scrub_stamp":"2024-09-21T05:39:21.797734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:00:07.194072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-21T05:39:24.226115+0000","last_change":"2024-09-21T05:39:21.833573+0000","last_active":"2024-09-21T05:39:24.226115+0000","last_peered":"2024-09-21T05:39:24.226115+0000","last_clean":"2024-09-21T05:39:24.226115+0000","last_became_active":"2024-09-21T05:39:21.833215+0000","last_became_peered":"2024-09-21T05:39:21.833215+0000","last_unstale":"2024-09-21T05:39:24.226115+0000","last_undegraded":"2024-09-21T05:39:24.226115+0000","last_fullsized":"2024-09-21T05:39:24.226115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:20.792192+0000","last_clean_scrub_stamp":"2024-09-21T05:39:20.792192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:24.424685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":664479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":73655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":73619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":73529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35560,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336280,"statfs":{"total":96636764160,"available":96600350720,"internally_reserved":0,"allocated":217088,"data_stored":73689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31988,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339852,"statfs":{"total":96636764160,"available":96604008448,"internally_reserved":0,"allocated":815104,"data_stored":664535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:39:35.018 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:39:35.018 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T05:39:35.018 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T05:39:35.018 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:39:35.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a99b12640 1 -- 172.21.15.22:0/2374626388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c004800 msgr2=0x7f0a7c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a99b12640 1 --2- 172.21.15.22:0/2374626388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a7c004800 0x7f0a7c004c10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009b20 tx=0x7f0a7c005830 comp rx=0 tx=0).stop 2024-09-21T05:39:35.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a93fff640 1 -- 172.21.15.22:0/2374626388 reap_dead start 2024-09-21T05:39:35.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a99b12640 1 -- 172.21.15.22:0/2374626388 shutdown_connections 2024-09-21T05:39:35.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a99b12640 1 -- 172.21.15.22:0/2374626388 >> 172.21.15.22:0/2374626388 conn(0x7f0a9409d7f0 msgr2=0x7f0a940d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:35.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a99b12640 1 -- 172.21.15.22:0/2374626388 shutdown_connections 2024-09-21T05:39:35.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.168+0000 7f0a99b12640 1 -- 172.21.15.22:0/2374626388 wait complete. 2024-09-21T05:39:35.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.170+0000 7f0a99b12640 1 Processor -- start 2024-09-21T05:39:35.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.170+0000 7f0a99b12640 1 -- start start 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a99b12640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a7c004800 0x7f0a9409eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a99b12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 0x7f0a940ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a99b12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a940cddc0 0x7f0a940ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a99b12640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9406f0e0 con 0x7f0a940ca400 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a99b12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9406ef60 con 0x7f0a940cddc0 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a99b12640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a9406f260 con 0x7f0a7c004800 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.171+0000 7f0a92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 0x7f0a940ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 0x7f0a940ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38660/0 (socket says 172.21.15.22:38660) 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 -- 172.21.15.22:0/1424829964 learned_addr learned my addr 172.21.15.22:0/1424829964 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 -- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a7c004800 msgr2=0x7f0a9409eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a7c004800 0x7f0a9409eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 -- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a940cddc0 msgr2=0x7f0a940ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a940cddc0 0x7f0a940ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 -- 172.21.15.22:0/1424829964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9409f580 con 0x7f0a940ca400 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a92ffd640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 0x7f0a940ca810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009a70 tx=0x7f0a88033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88043070 con 0x7f0a940ca400 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a88004320 con 0x7f0a940ca400 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a941f5750 con 0x7f0a940ca400 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a88038040 con 0x7f0a940ca400 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.172+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a941f5d10 con 0x7f0a940ca400 2024-09-21T05:39:35.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.173+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a940d4d80 con 0x7f0a940ca400 2024-09-21T05:39:35.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.174+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a88046070 con 0x7f0a940ca400 2024-09-21T05:39:35.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.174+0000 7f0a7bfff640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a80078b90 0x7f0a8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.175+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a880bb2a0 con 0x7f0a940ca400 2024-09-21T05:39:35.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.175+0000 7f0a937fe640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a80078b90 0x7f0a8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.175+0000 7f0a937fe640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a80078b90 0x7f0a8007b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a84001180 tx=0x7f0a840012d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:35.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.179+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a88083090 con 0x7f0a940ca400 2024-09-21T05:39:35.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.405+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0a940b6a20 con 0x7f0a940ca400 2024-09-21T05:39:35.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.405+0000 7f0a7bfff640 1 -- 172.21.15.22:0/1424829964 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0a880bb0c0 con 0x7f0a940ca400 2024-09-21T05:39:35.408 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:35.408 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a80078b90 msgr2=0x7f0a8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a80078b90 0x7f0a8007b0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a84001180 tx=0x7f0a840012d0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 msgr2=0x7f0a940ca810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 0x7f0a940ca810 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009a70 tx=0x7f0a88033670 comp rx=0 tx=0).stop 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 shutdown_connections 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a940cddc0 0x7f0a940ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a940ca400 0x7f0a940ca810 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a80078b90 0x7f0a8007b0a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 --2- 172.21.15.22:0/1424829964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a7c004800 0x7f0a9409eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 >> 172.21.15.22:0/1424829964 conn(0x7f0a9409d7f0 msgr2=0x7f0a940b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:35.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.407+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 shutdown_connections 2024-09-21T05:39:35.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.408+0000 7f0a99b12640 1 -- 172.21.15.22:0/1424829964 wait complete. 2024-09-21T05:39:35.421 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T05:39:35.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:35.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 -- 172.21.15.22:0/2718266339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80d16c0 msgr2=0x7f19b80cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 --2- 172.21.15.22:0/2718266339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80d16c0 0x7f19b80cf310 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19b80d0910 tx=0x7f19ac02a560 comp rx=0 tx=0).stop 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 -- 172.21.15.22:0/2718266339 shutdown_connections 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 --2- 172.21.15.22:0/2718266339 >> v2:172.21.15.134:3300/0 conn(0x7f19b80b3a10 0x7f19b80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 --2- 172.21.15.22:0/2718266339 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80d16c0 0x7f19b80cf310 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 -- 172.21.15.22:0/2718266339 >> 172.21.15.22:0/2718266339 conn(0x7f19b809d8f0 msgr2=0x7f19b80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 -- 172.21.15.22:0/2718266339 shutdown_connections 2024-09-21T05:39:35.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.535+0000 7f19be32b640 1 -- 172.21.15.22:0/2718266339 wait complete. 2024-09-21T05:39:35.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.537+0000 7f19be32b640 1 Processor -- start 2024-09-21T05:39:35.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.537+0000 7f19be32b640 1 -- start start 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19be32b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b80b3a10 0x7f19b80b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19be32b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 0x7f19b81f9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19be32b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19b80caa50 0x7f19b81f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19be32b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80d0120 con 0x7f19b80b9b60 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19be32b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f19b80cffa0 con 0x7f19b80b3a10 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19be32b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f19b80d02a0 con 0x7f19b80caa50 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 0x7f19b81f9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.538+0000 7f19b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 0x7f19b81f9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38668/0 (socket says 172.21.15.22:38668) 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 -- 172.21.15.22:0/2965240646 learned_addr learned my addr 172.21.15.22:0/2965240646 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:35.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19bc8c8640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19b80caa50 0x7f19b81f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b7fff640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b80b3a10 0x7f19b80b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 -- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19b80caa50 msgr2=0x7f19b81f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19b80caa50 0x7f19b81f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 -- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b80b3a10 msgr2=0x7f19b80b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b80b3a10 0x7f19b80b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 -- 172.21.15.22:0/2965240646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b80b6680 con 0x7f19b80b9b60 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b7fff640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b80b3a10 0x7f19b80b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19bc8c8640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19b80caa50 0x7f19b81f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b77fe640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 0x7f19b81f9df0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0099c0 tx=0x7f19ac0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac03e030 con 0x7f19b80b9b60 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19ac0047a0 con 0x7f19b80b9b60 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ac035040 con 0x7f19b80b9b60 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b80b6960 con 0x7f19b80b9b60 2024-09-21T05:39:35.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.539+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b80b2990 con 0x7f19b80b9b60 2024-09-21T05:39:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.540+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b80b4570 con 0x7f19b80b9b60 2024-09-21T05:39:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.541+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f19ac041030 con 0x7f19b80b9b60 2024-09-21T05:39:35.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.542+0000 7f19b4ff9640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19a4078bc0 0x7f19a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.542+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19ac0bb510 con 0x7f19b80b9b60 2024-09-21T05:39:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.542+0000 7f19b7fff640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19a4078bc0 0x7f19a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.542+0000 7f19b7fff640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19a4078bc0 0x7f19a407b0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19a8007fd0 tx=0x7f19a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:35.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.546+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19b80b4570 con 0x7f19b80b9b60 2024-09-21T05:39:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.774+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19b80b1250 con 0x7f19b80b9b60 2024-09-21T05:39:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.775+0000 7f19b4ff9640 1 -- 172.21.15.22:0/2965240646 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f19ac083960 con 0x7f19b80b9b60 2024-09-21T05:39:35.777 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:35.777 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:35.778 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:35.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19a4078bc0 msgr2=0x7f19a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19a4078bc0 0x7f19a407b0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f19a8007fd0 tx=0x7f19a8009040 comp rx=0 tx=0).stop 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 msgr2=0x7f19b81f9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 0x7f19b81f9df0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0099c0 tx=0x7f19ac0039e0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 shutdown_connections 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19a4078bc0 0x7f19a407b0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f19b80caa50 0x7f19b81f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f19b80b9b60 0x7f19b81f9df0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 --2- 172.21.15.22:0/2965240646 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f19b80b3a10 0x7f19b80b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 >> 172.21.15.22:0/2965240646 conn(0x7f19b809d8f0 msgr2=0x7f19b806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 shutdown_connections 2024-09-21T05:39:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.777+0000 7f19be32b640 1 -- 172.21.15.22:0/2965240646 wait complete. 2024-09-21T05:39:35.791 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:35.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 -- 172.21.15.22:0/3942904646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006e170 msgr2=0x7f27400b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 --2- 172.21.15.22:0/3942904646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006e170 0x7f27400b3540 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2730009830 tx=0x7f273002a480 comp rx=0 tx=0).stop 2024-09-21T05:39:35.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 -- 172.21.15.22:0/3942904646 shutdown_connections 2024-09-21T05:39:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 --2- 172.21.15.22:0/3942904646 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006e170 0x7f27400b3540 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 -- 172.21.15.22:0/3942904646 >> 172.21.15.22:0/3942904646 conn(0x7f274008bfc0 msgr2=0x7f274009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 -- 172.21.15.22:0/3942904646 shutdown_connections 2024-09-21T05:39:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.905+0000 7f2745d52640 1 -- 172.21.15.22:0/3942904646 wait complete. 2024-09-21T05:39:35.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.907+0000 7f2745d52640 1 Processor -- start 2024-09-21T05:39:35.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.907+0000 7f2745d52640 1 -- start start 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f2745d52640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 0x7f274014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f2745d52640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27400ceb40 0x7f2740150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f2745d52640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274015e210 0x7f2740150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f2745d52640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27400d10b0 con 0x7f274006f4b0 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f2745d52640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27400d0f30 con 0x7f27400ceb40 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f2745d52640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27400d1230 con 0x7f274015e210 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 0x7f274014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 0x7f274014da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38684/0 (socket says 172.21.15.22:38684) 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 -- 172.21.15.22:0/1862070275 learned_addr learned my addr 172.21.15.22:0/1862070275 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273effd640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27400ceb40 0x7f2740150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273ffff640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274015e210 0x7f2740150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 -- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274015e210 msgr2=0x7f2740150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274015e210 0x7f2740150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 -- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27400ceb40 msgr2=0x7f2740150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27400ceb40 0x7f2740150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273f7fe640 1 -- 172.21.15.22:0/1862070275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2740160070 con 0x7f274006f4b0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273effd640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27400ceb40 0x7f2740150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.908+0000 7f273ffff640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274015e210 0x7f2740150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.909+0000 7f273f7fe640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 0x7f274014da20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2730009950 tx=0x7f2730033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.909+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730031d60 con 0x7f274006f4b0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.909+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2730033e10 con 0x7f274006f4b0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.909+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274015fab0 con 0x7f274006f4b0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.909+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730005260 con 0x7f274006f4b0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.909+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27401e8370 con 0x7f274006f4b0 2024-09-21T05:39:35.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.910+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f274006e170 con 0x7f274006f4b0 2024-09-21T05:39:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.911+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2730005400 con 0x7f274006f4b0 2024-09-21T05:39:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.911+0000 7f2727fff640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f272c078bc0 0x7f272c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.911+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f27300bbfa0 con 0x7f274006f4b0 2024-09-21T05:39:35.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.912+0000 7f273effd640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f272c078bc0 0x7f272c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:35.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.912+0000 7f273effd640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f272c078bc0 0x7f272c07b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27401ea210 tx=0x7f2734008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:35.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:35.916+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f273003f030 con 0x7f274006f4b0 2024-09-21T05:39:36.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.143+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27400b1af0 con 0x7f274006f4b0 2024-09-21T05:39:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.144+0000 7f2727fff640 1 -- 172.21.15.22:0/1862070275 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f2730084470 con 0x7f274006f4b0 2024-09-21T05:39:36.146 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:36.146 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:36.146 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f272c078bc0 msgr2=0x7f272c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f272c078bc0 0x7f272c07b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f27401ea210 tx=0x7f2734008040 comp rx=0 tx=0).stop 2024-09-21T05:39:36.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 msgr2=0x7f274014da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 0x7f274014da20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2730009950 tx=0x7f2730033c20 comp rx=0 tx=0).stop 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 shutdown_connections 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f272c078bc0 0x7f272c07b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f274015e210 0x7f2740150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27400ceb40 0x7f2740150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 --2- 172.21.15.22:0/1862070275 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f274006f4b0 0x7f274014da20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 >> 172.21.15.22:0/1862070275 conn(0x7f274008bfc0 msgr2=0x7f2740161ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 shutdown_connections 2024-09-21T05:39:36.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.146+0000 7f2745d52640 1 -- 172.21.15.22:0/1862070275 wait complete. 2024-09-21T05:39:36.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf8415640 1 -- 172.21.15.22:0/1319133875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd80041c0 msgr2=0x7f0bd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf8415640 1 --2- 172.21.15.22:0/1319133875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bd80041c0 0x7f0bd8004800 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009cd0 tx=0x7f0bd8005830 comp rx=0 tx=0).stop 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf69b2640 1 -- 172.21.15.22:0/1319133875 reap_dead start 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf8415640 1 -- 172.21.15.22:0/1319133875 shutdown_connections 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf8415640 1 -- 172.21.15.22:0/1319133875 >> 172.21.15.22:0/1319133875 conn(0x7f0bf006d4f0 msgr2=0x7f0bf0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf8415640 1 -- 172.21.15.22:0/1319133875 shutdown_connections 2024-09-21T05:39:36.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.274+0000 7f0bf8415640 1 -- 172.21.15.22:0/1319133875 wait complete. 2024-09-21T05:39:36.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.275+0000 7f0bf8415640 1 Processor -- start 2024-09-21T05:39:36.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.276+0000 7f0bf8415640 1 -- start start 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf8415640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bd80041c0 0x7f0bf00c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf8415640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0bf015d5c0 0x7f0bf015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf8415640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 0x7f0bf01f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf8415640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf00d1210 con 0x7f0bf015df10 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf8415640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf00d1090 con 0x7f0bd80041c0 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf8415640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0bf00d1390 con 0x7f0bf015d5c0 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 0x7f0bf01f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 0x7f0bf01f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38710/0 (socket says 172.21.15.22:38710) 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 -- 172.21.15.22:0/2419241365 learned_addr learned my addr 172.21.15.22:0/2419241365 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf61b1640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bd80041c0 0x7f0bf00c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf59b0640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0bf015d5c0 0x7f0bf015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 -- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0bf015d5c0 msgr2=0x7f0bf015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0bf015d5c0 0x7f0bf015d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 -- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bd80041c0 msgr2=0x7f0bf00c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bd80041c0 0x7f0bf00c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 -- 172.21.15.22:0/2419241365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf0153d70 con 0x7f0bf015df10 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf61b1640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bd80041c0 0x7f0bf00c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.277+0000 7f0bf69b2640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 0x7f0bf01f6f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0be8000c00 tx=0x7f0be80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.278+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8011840 con 0x7f0bf015df10 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.278+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0be8018070 con 0x7f0bf015df10 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.278+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be8011c40 con 0x7f0bf015df10 2024-09-21T05:39:36.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.278+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf015cb20 con 0x7f0bf015df10 2024-09-21T05:39:36.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.278+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf015cdc0 con 0x7f0bf015df10 2024-09-21T05:39:36.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.279+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf01569d0 con 0x7f0bf015df10 2024-09-21T05:39:36.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.279+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0be801d460 con 0x7f0bf015df10 2024-09-21T05:39:36.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.280+0000 7f0beeffd640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.280+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0be809b490 con 0x7f0bf015df10 2024-09-21T05:39:36.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.280+0000 7f0bf61b1640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.281+0000 7f0bf61b1640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be0001ad0 tx=0x7f0be0006e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:36.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.285+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bf0153d70 con 0x7f0bf015df10 2024-09-21T05:39:36.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.472+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bf00d18f0 con 0x7f0bf015df10 2024-09-21T05:39:36.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.474+0000 7f0beeffd640 1 -- 172.21.15.22:0/2419241365 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7f0be8063960 con 0x7f0bf015df10 2024-09-21T05:39:36.476 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:36.476 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:36.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0bdc078b90 msgr2=0x7f0bdc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be0001ad0 tx=0x7f0be0006e90 comp rx=0 tx=0).stop 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 msgr2=0x7f0bf01f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 0x7f0bf01f6f70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0be8000c00 tx=0x7f0be80039c0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 shutdown_connections 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0bf015df10 0x7f0bf01f6f70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0bdc078b90 0x7f0bdc07b0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0bf015d5c0 0x7f0bf015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 --2- 172.21.15.22:0/2419241365 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0bd80041c0 0x7f0bf00c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 >> 172.21.15.22:0/2419241365 conn(0x7f0bf006d4f0 msgr2=0x7f0bf0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 shutdown_connections 2024-09-21T05:39:36.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.476+0000 7f0bf8415640 1 -- 172.21.15.22:0/2419241365 wait complete. 2024-09-21T05:39:36.490 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.603+0000 7f625fdba640 1 -- 172.21.15.22:0/3724432316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 msgr2=0x7f62580d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.603+0000 7f625fdba640 1 --2- 172.21.15.22:0/3724432316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62580d14f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62580d4470 tx=0x7f625402a4d0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 -- 172.21.15.22:0/3724432316 shutdown_connections 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 --2- 172.21.15.22:0/3724432316 >> v2:172.21.15.134:3300/0 conn(0x7f6258162430 0x7f6258162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 --2- 172.21.15.22:0/3724432316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62580d14f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 --2- 172.21.15.22:0/3724432316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62580b9b60 0x7f62580d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 -- 172.21.15.22:0/3724432316 >> 172.21.15.22:0/3724432316 conn(0x7f625809d8f0 msgr2=0x7f62580d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:36.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 -- 172.21.15.22:0/3724432316 shutdown_connections 2024-09-21T05:39:36.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.604+0000 7f625fdba640 1 -- 172.21.15.22:0/3724432316 wait complete. 2024-09-21T05:39:36.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.605+0000 7f625fdba640 1 Processor -- start 2024-09-21T05:39:36.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.605+0000 7f625fdba640 1 -- start start 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625fdba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62581e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625fdba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62580b9b60 0x7f62581e8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625fdba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6258162430 0x7f62580b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625fdba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6258163110 con 0x7f62580b3a10 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625fdba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6258162f90 con 0x7f6258162430 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625fdba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6258163290 con 0x7f62580b9b60 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62581e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62581e97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38726/0 (socket says 172.21.15.22:38726) 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 -- 172.21.15.22:0/171651016 learned_addr learned my addr 172.21.15.22:0/171651016 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625e357640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6258162430 0x7f62580b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625d355640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62580b9b60 0x7f62581e8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 -- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62580b9b60 msgr2=0x7f62581e8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62580b9b60 0x7f62581e8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 -- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6258162430 msgr2=0x7f62580b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6258162430 0x7f62580b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 -- 172.21.15.22:0/171651016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62580caf20 con 0x7f62580b3a10 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.607+0000 7f625db56640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62581e97a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f624800d750 tx=0x7f624800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.608+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6248017070 con 0x7f62580b3a10 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.608+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6248009cc0 con 0x7f62580b3a10 2024-09-21T05:39:36.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.608+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62580c8490 con 0x7f62580b3a10 2024-09-21T05:39:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.608+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62480106c0 con 0x7f62580b3a10 2024-09-21T05:39:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.608+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62580ca930 con 0x7f62580b3a10 2024-09-21T05:39:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.609+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6258162840 con 0x7f62580b3a10 2024-09-21T05:39:36.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.609+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f624801a070 con 0x7f62580b3a10 2024-09-21T05:39:36.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.610+0000 7f624e7fc640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f623c078b90 0x7f623c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:36.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.610+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f624809ad00 con 0x7f62580b3a10 2024-09-21T05:39:36.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.610+0000 7f625d355640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f623c078b90 0x7f623c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:36.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.611+0000 7f625d355640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f623c078b90 0x7f623c07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6254033000 tx=0x7f6254004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:36.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.614+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f624800de60 con 0x7f62580b3a10 2024-09-21T05:39:36.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.841+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62580d0d90 con 0x7f62580b3a10 2024-09-21T05:39:36.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.841+0000 7f624e7fc640 1 -- 172.21.15.22:0/171651016 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f6248062b80 con 0x7f62580b3a10 2024-09-21T05:39:36.843 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:36.844 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:36.844 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f623c078b90 msgr2=0x7f623c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f623c078b90 0x7f623c07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6254033000 tx=0x7f6254004730 comp rx=0 tx=0).stop 2024-09-21T05:39:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 msgr2=0x7f62581e97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62581e97a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f624800d750 tx=0x7f624800dc10 comp rx=0 tx=0).stop 2024-09-21T05:39:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 shutdown_connections 2024-09-21T05:39:36.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6258162430 0x7f62580b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f623c078b90 0x7f623c07b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62580b9b60 0x7f62581e8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 --2- 172.21.15.22:0/171651016 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62580b3a10 0x7f62581e97a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:36.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 >> 172.21.15.22:0/171651016 conn(0x7f625809d8f0 msgr2=0x7f62580d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:36.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 shutdown_connections 2024-09-21T05:39:36.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:36.844+0000 7f625fdba640 1 -- 172.21.15.22:0/171651016 wait complete. 2024-09-21T05:39:37.259 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:39:22.195858+0000', 'modified': '2024-09-21T05:39:25.213840+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4395': {'gid': 4395, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6835/1656544793', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 1656544793}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 1656544793}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4395, 'qdb_cluster': [4395]} 2024-09-21T05:39:37.260 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:39:37.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feedde9b640 1 -- 172.21.15.22:0/2478521971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feec0004d40 msgr2=0x7feec0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feedde9b640 1 --2- 172.21.15.22:0/2478521971 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feec0004d40 0x7feec0005150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feed0009ed0 tx=0x7feec0005830 comp rx=0 tx=0).stop 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feed7fff640 1 -- 172.21.15.22:0/2478521971 reap_dead start 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feedde9b640 1 -- 172.21.15.22:0/2478521971 shutdown_connections 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feedde9b640 1 -- 172.21.15.22:0/2478521971 >> 172.21.15.22:0/2478521971 conn(0x7feed808bfc0 msgr2=0x7feed809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feedde9b640 1 -- 172.21.15.22:0/2478521971 shutdown_connections 2024-09-21T05:39:37.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.382+0000 7feedde9b640 1 -- 172.21.15.22:0/2478521971 wait complete. 2024-09-21T05:39:37.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.384+0000 7feedde9b640 1 Processor -- start 2024-09-21T05:39:37.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.384+0000 7feedde9b640 1 -- start start 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feedde9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0004d40 0x7feed80ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feedde9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 0x7feed81498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feedde9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed8149de0 0x7feed815e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feedde9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feed80b5a80 con 0x7feed815c790 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feedde9b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feed80b5c00 con 0x7feec0004d40 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feedde9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feed80d4640 con 0x7feed8149de0 2024-09-21T05:39:37.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 0x7feed81498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 0x7feed81498a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38752/0 (socket says 172.21.15.22:38752) 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 -- 172.21.15.22:0/329134641 learned_addr learned my addr 172.21.15.22:0/329134641 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed7fff640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed8149de0 0x7feed815e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed77fe640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0004d40 0x7feed80ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 -- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed8149de0 msgr2=0x7feed815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed8149de0 0x7feed815e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 -- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0004d40 msgr2=0x7feed80ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0004d40 0x7feed80ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 -- 172.21.15.22:0/329134641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed8148800 con 0x7feed815c790 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed77fe640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0004d40 0x7feed80ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feed6ffd640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 0x7feed81498a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feecc011ae0 tx=0x7feecc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.386+0000 7feebffff640 1 -- 172.21.15.22:0/329134641 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc00fcb0 con 0x7feed815c790 2024-09-21T05:39:37.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.387+0000 7feebffff640 1 -- 172.21.15.22:0/329134641 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feecc00fe50 con 0x7feed815c790 2024-09-21T05:39:37.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.387+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed8148ae0 con 0x7feed815c790 2024-09-21T05:39:37.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.387+0000 7feebffff640 1 -- 172.21.15.22:0/329134641 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc004280 con 0x7feed815c790 2024-09-21T05:39:37.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.387+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81490a0 con 0x7feed815c790 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.388+0000 7feebffff640 1 -- 172.21.15.22:0/329134641 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feecc0035f0 con 0x7feed815c790 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.389+0000 7feebffff640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feec4078bc0 0x7feec407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.389+0000 7feebffff640 1 -- 172.21.15.22:0/329134641 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feecc09b830 con 0x7feed815c790 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.389+0000 7feed77fe640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feec4078bc0 0x7feec407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.390+0000 7feed77fe640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feec4078bc0 0x7feec407b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feec8001eb0 tx=0x7feec80067f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.390+0000 7feee4e0f740 1 Processor -- start 2024-09-21T05:39:37.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.390+0000 7feee4e0f740 1 -- start start 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc1a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feee4e0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbbdb62820 con 0x55dbbd848270 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dbbdb51c50 con 0x55dbbd81f3b0 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dbbdb55a60 con 0x55dbbd827590 2024-09-21T05:39:37.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feed77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feed77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc19fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38762/0 (socket says 172.21.15.22:38762) 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.391+0000 7feed77fe640 1 -- 172.21.15.22:0/2306628408 learned_addr learned my addr 172.21.15.22:0/2306628408 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed6ffd640 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc1a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed7fff640 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed77fe640 1 -- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 msgr2=0x55dbbdc373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed77fe640 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed77fe640 1 -- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 msgr2=0x55dbbdc1a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed77fe640 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc1a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed77fe640 1 -- 172.21.15.22:0/2306628408 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbbd8b3850 con 0x55dbbd848270 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feed77fe640 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc19fa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feec8003af0 tx=0x7feec8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd4b7f72fa34075 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feebd7fa640 1 -- 172.21.15.22:0/2306628408 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8012880 con 0x55dbbd848270 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feebd7fa640 1 -- 172.21.15.22:0/2306628408 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feec8022030 con 0x55dbbd848270 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.392+0000 7feebd7fa640 1 -- 172.21.15.22:0/2306628408 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8012c20 con 0x55dbbd848270 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 -- 172.21.15.22:0/2306628408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 msgr2=0x55dbbdc19fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc19fa0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feec8003af0 tx=0x7feec8014000 comp rx=0 tx=0).stop 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 -- 172.21.15.22:0/2306628408 shutdown_connections 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc1a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2306628408 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc19fa0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 -- 172.21.15.22:0/2306628408 >> 172.21.15.22:0/2306628408 conn(0x55dbbd83c3c0 msgr2=0x55dbbdbf0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 -- 172.21.15.22:0/2306628408 shutdown_connections 2024-09-21T05:39:37.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.393+0000 7feee4e0f740 1 -- 172.21.15.22:0/2306628408 wait complete. 2024-09-21T05:39:37.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.394+0000 7feee4e0f740 1 -- start start 2024-09-21T05:39:37.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.394+0000 7feee4e0f740 1 Processor -- start 2024-09-21T05:39:37.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbbd8c2620 con 0x55dbbd848270 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dbbdaf75f0 con 0x55dbbd81f3b0 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dbbdc1aa20 con 0x55dbbd827590 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feee4e0f740 10 client.0 _ll_register_callbacks cb 0x55dbbdafb0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.395+0000 7feed7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc09760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38770/0 (socket says 172.21.15.22:38770) 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 -- 172.21.15.22:0/2457237962 learned_addr learned my addr 172.21.15.22:0/2457237962 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed6ffd640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed77fe640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 -- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 msgr2=0x55dbbdc08470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc08470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 -- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 msgr2=0x55dbbdc352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 -- 172.21.15.22:0/2457237962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbbd92a6b0 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed77fe640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feed7fff640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc09760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feed0036040 tx=0x7feed0003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02c27ebc534b43e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feeb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feeb67fc640 1 -- 172.21.15.22:0/2457237962 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0038040 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feeb67fc640 1 -- 172.21.15.22:0/2457237962 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feed0003f90 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec8022030 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feeb67fc640 1 -- 172.21.15.22:0/2457237962 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7feed0009ad0 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feee4e0f740 10 client.4722 fetch_fsmap learned FSMap version 8 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.396+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55dbbdc477b0 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feeb67fc640 1 -- 172.21.15.22:0/2457237962 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feec8022030 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feeb67fc640 1 -- 172.21.15.22:0/2457237962 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7feed0004280 con 0x55dbbd848270 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feee4e0f740 10 client.4722 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feee4e0f740 10 client.4722 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:39:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feee4e0f740 20 client.4722 populate_metadata read hostname 'smithi022' 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbd806e40 0x55dbbdc2bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feee4e0f740 4 client.4722 mds_command: new command op to 4395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 --> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbbdb64770 con 0x55dbbd806e40 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feed77fe640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbd806e40 0x55dbbdc2bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feeb67fc640 10 client.4722 ms_handle_connect on v2:172.21.15.22:6833/1656544793 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.397+0000 7feed77fe640 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbd806e40 0x55dbbdc2bb90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6da2da1806d9def server_cookie=9cf17cfaf0c4fe15 in_seq=0 out_seq=0 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feeb67fc640 1 -- 172.21.15.22:0/2457237962 <== mds.0 v2:172.21.15.22:6833/1656544793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dbbdb64770 con 0x55dbbd806e40 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feeb67fc640 10 client.4722 handle_command_reply: tid=0 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 client.4722 shutdown 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 msgr2=0x55dbbdc09760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc09760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feed0036040 tx=0x7feed0003de0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 shutdown_connections 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbd806e40 0x55dbbdc2bb90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc09760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc08470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2457237962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 >> 172.21.15.22:0/2457237962 conn(0x55dbbd83c3c0 msgr2=0x55dbbdbea9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feeb67fc640 0 client.4722 ms_handle_reset on v2:172.21.15.22:6833/1656544793 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 shutdown_connections 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 1 -- 172.21.15.22:0/2457237962 wait complete. 2024-09-21T05:39:37.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.398+0000 7feee4e0f740 20 client.4722 trim_cache size 0 max 16384 2024-09-21T05:39:37.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.417+0000 7feee4e0f740 1 Processor -- start 2024-09-21T05:39:37.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.417+0000 7feee4e0f740 1 -- start start 2024-09-21T05:39:37.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.418+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.418+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc2eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.418+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc30570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.418+0000 7feee4e0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feed8201730 con 0x55dbbd848270 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.418+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feed814b3c0 con 0x55dbbd81f3b0 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.418+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feed82010e0 con 0x55dbbd827590 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc30570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc30570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38780/0 (socket says 172.21.15.22:38780) 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 -- 172.21.15.22:0/1546318904 learned_addr learned my addr 172.21.15.22:0/1546318904 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed77fe640 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed6ffd640 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc2eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 -- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 msgr2=0x55dbbdc2eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc2eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 -- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 msgr2=0x55dbbdc46360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc46360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 -- 172.21.15.22:0/1546318904 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbbdc31250 con 0x55dbbd848270 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feed7fff640 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc30570 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feed0005c00 tx=0x7feed0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bd3ce1dede06f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feebcff9640 1 -- 172.21.15.22:0/1546318904 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0054070 con 0x55dbbd848270 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feebcff9640 1 -- 172.21.15.22:0/1546318904 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed0003f90 con 0x55dbbd848270 2024-09-21T05:39:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.419+0000 7feebcff9640 1 -- 172.21.15.22:0/1546318904 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed0038070 con 0x55dbbd848270 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 -- 172.21.15.22:0/1546318904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 msgr2=0x55dbbdc30570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc30570 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feed0005c00 tx=0x7feed0009480 comp rx=0 tx=0).stop 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 -- 172.21.15.22:0/1546318904 shutdown_connections 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd848270 0x55dbbdc30570 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc2eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 --2- 172.21.15.22:0/1546318904 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 -- 172.21.15.22:0/1546318904 >> 172.21.15.22:0/1546318904 conn(0x55dbbd83c3c0 msgr2=0x55dbbdbe5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 -- 172.21.15.22:0/1546318904 shutdown_connections 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.420+0000 7feee4e0f740 1 -- 172.21.15.22:0/1546318904 wait complete. 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.421+0000 7feee4e0f740 1 -- start start 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.421+0000 7feee4e0f740 1 Processor -- start 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc18560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd848270 0x55dbbdc18aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dbbdc08bf0 con 0x55dbbd81f3b0 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dbbdc47a90 con 0x55dbbd848270 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dbbdc2d890 con 0x55dbbd827590 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feee4e0f740 10 client.0 _ll_register_callbacks cb 0x7feec00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feed77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feed77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38794/0 (socket says 172.21.15.22:38794) 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.422+0000 7feed77fe640 1 -- 172.21.15.22:0/2399034374 learned_addr learned my addr 172.21.15.22:0/2399034374 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed7fff640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd848270 0x55dbbdc18aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed6ffd640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc18560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed77fe640 1 -- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 msgr2=0x55dbbdc18560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed77fe640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc18560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed77fe640 1 -- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd848270 msgr2=0x55dbbdc18aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:39:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed77fe640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd848270 0x55dbbdc18aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed77fe640 1 -- 172.21.15.22:0/2399034374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbbdc56850 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feed77fe640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc574a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55dbbdbf1e30 tx=0x7feec8001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93d1d0651878294 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feeb67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feeb67fc640 1 -- 172.21.15.22:0/2399034374 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80128e0 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feeb67fc640 1 -- 172.21.15.22:0/2399034374 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feec8012a80 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feed0003f90 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feeb67fc640 1 -- 172.21.15.22:0/2399034374 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7feec8042050 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feee4e0f740 10 client.4728 fetch_fsmap learned FSMap version 8 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.423+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55dbbdc579e0 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feeb67fc640 1 -- 172.21.15.22:0/2399034374 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feed0003f90 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feeb67fc640 1 -- 172.21.15.22:0/2399034374 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7feec8045020 con 0x55dbbd81f3b0 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 10 client.4728 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 10 client.4728 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 20 client.4728 populate_metadata read hostname 'smithi022' 2024-09-21T05:39:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbdc2b0f0 0x55dbbdbe9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 4 client.4728 mds_command: new command op to 4395 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 --> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] -- command(tid 0: {"prefix": "status"}) -- 0x55dbbdb60220 con 0x55dbbdc2b0f0 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feed6ffd640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbdc2b0f0 0x55dbbdbe9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feeb67fc640 10 client.4728 ms_handle_connect on v2:172.21.15.22:6833/1656544793 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feed6ffd640 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbdc2b0f0 0x55dbbdbe9490 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9b35fcd57e35a1c server_cookie=c931c4a320059efb in_seq=0 out_seq=0 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feeb67fc640 1 -- 172.21.15.22:0/2399034374 <== mds.0 v2:172.21.15.22:6833/1656544793 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x55dbbdb60220 con 0x55dbbdc2b0f0 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feeb67fc640 10 client.4728 handle_command_reply: tid=0 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 1 client.4728 shutdown 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 msgr2=0x55dbbdc574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.424+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc574a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55dbbdbf1e30 tx=0x7feec8001110 comp rx=0 tx=0).stop 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 shutdown_connections 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:6833/1656544793,v1:172.21.15.22:6835/1656544793] conn(0x55dbbdc2b0f0 0x55dbbdbe9490 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dbbd848270 0x55dbbdc18aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dbbd827590 0x55dbbdc18560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feeb67fc640 0 client.4728 ms_handle_reset on v2:172.21.15.22:6833/1656544793 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 --2- 172.21.15.22:0/2399034374 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dbbd81f3b0 0x55dbbdc574a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 >> 172.21.15.22:0/2399034374 conn(0x55dbbd83c3c0 msgr2=0x55dbbdc57ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 shutdown_connections 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 1 -- 172.21.15.22:0/2399034374 wait complete. 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feee4e0f740 20 client.4728 trim_cache size 0 max 16384 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4395, 2024-09-21T05:39:37.432 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 14.618050108, 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 8, 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 21, 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 21, 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 20.653538631 2024-09-21T05:39:37.433 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feec4078bc0 msgr2=0x7feec407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feec4078bc0 0x7feec407b0d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feec8001eb0 tx=0x7feec80067f0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 msgr2=0x7feed81498a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 0x7feed81498a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feecc011ae0 tx=0x7feecc009f10 comp rx=0 tx=0).stop 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 shutdown_connections 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feec4078bc0 0x7feec407b0d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feed8149de0 0x7feed815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feed815c790 0x7feed81498a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 --2- 172.21.15.22:0/329134641 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feec0004d40 0x7feed80ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:37.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 >> 172.21.15.22:0/329134641 conn(0x7feed808bfc0 msgr2=0x7feed806e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:37.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 shutdown_connections 2024-09-21T05:39:37.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:37.425+0000 7feedde9b640 1 -- 172.21.15.22:0/329134641 wait complete. 2024-09-21T05:39:37.438 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T05:39:37.465 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T05:39:37.465 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T05:39:37.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.466 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:39:37.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-09-21T05:39:37.549 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-09-21T05:39:37.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:37.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.551 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:39:37.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.608 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-09-21T05:39:37.675 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-09-21T05:39:37.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:37.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.677 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-09-21T05:39:37.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.697 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-09-21T05:39:37.769 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-09-21T05:39:37.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:37.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.771 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-09-21T05:39:37.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:37.830 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-09-21T05:39:37.898 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-09-21T05:39:37.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:39:37.900 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T05:39:37.900 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:39:37.900 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:39:37.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:39:38.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 -- 172.21.15.22:0/1779086509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00ca800 msgr2=0x7f96d00cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 --2- 172.21.15.22:0/1779086509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00ca800 0x7f96d00cba00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f96c0005fd0 tx=0x7f96c002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 -- 172.21.15.22:0/1779086509 shutdown_connections 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 --2- 172.21.15.22:0/1779086509 >> v2:172.21.15.134:3300/0 conn(0x7f96d00cad30 0x7f96d006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 --2- 172.21.15.22:0/1779086509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00ca800 0x7f96d00cba00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 -- 172.21.15.22:0/1779086509 >> 172.21.15.22:0/1779086509 conn(0x7f96d00b1cb0 msgr2=0x7f96d00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 -- 172.21.15.22:0/1779086509 shutdown_connections 2024-09-21T05:39:38.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.014+0000 7f96d5189640 1 -- 172.21.15.22:0/1779086509 wait complete. 2024-09-21T05:39:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.016+0000 7f96d5189640 1 Processor -- start 2024-09-21T05:39:38.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.016+0000 7f96d5189640 1 -- start start 2024-09-21T05:39:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00704e0 0x7f96d01525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 0x7f96d015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d00cad30 0x7f96d00c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f96d00ccf40 con 0x7f96d00ca800 2024-09-21T05:39:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f96d00747d0 con 0x7f96d00cad30 2024-09-21T05:39:38.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f96d00cd0c0 con 0x7f96d00704e0 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 0x7f96d015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 0x7f96d015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38808/0 (socket says 172.21.15.22:38808) 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 -- 172.21.15.22:0/2196688291 learned_addr learned my addr 172.21.15.22:0/2196688291 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96cf59e640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d00cad30 0x7f96d00c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ced9d640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00704e0 0x7f96d01525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 -- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00704e0 msgr2=0x7f96d01525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00704e0 0x7f96d01525c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 -- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d00cad30 msgr2=0x7f96d00c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d00cad30 0x7f96d00c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 -- 172.21.15.22:0/2196688291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d015c850 con 0x7f96d00ca800 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ced9d640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00704e0 0x7f96d01525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:38.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96ce59c640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 0x7f96d015bee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009f90 tx=0x7f96c400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c400cd60 con 0x7f96d00ca800 2024-09-21T05:39:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96c4004020 con 0x7f96d00ca800 2024-09-21T05:39:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d015cb30 con 0x7f96d00ca800 2024-09-21T05:39:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.018+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c4004450 con 0x7f96d00ca800 2024-09-21T05:39:38.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.019+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d00c7320 con 0x7f96d00ca800 2024-09-21T05:39:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.019+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d015c5c0 con 0x7f96d00ca800 2024-09-21T05:39:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.021+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f96c4009400 con 0x7f96d00ca800 2024-09-21T05:39:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.021+0000 7f96b77fe640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078b90 0x7f96bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.021+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f96c409d490 con 0x7f96d00ca800 2024-09-21T05:39:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.021+0000 7f96ced9d640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078b90 0x7f96bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:38.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.022+0000 7f96ced9d640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078b90 0x7f96bc07b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96c0034000 tx=0x7f96c0004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:38.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.026+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96c40658e0 con 0x7f96d00ca800 2024-09-21T05:39:38.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.252+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f96d015d050 con 0x7f96d00ca800 2024-09-21T05:39:38.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.252+0000 7f96b77fe640 1 -- 172.21.15.22:0/2196688291 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f96c40651e0 con 0x7f96d00ca800 2024-09-21T05:39:38.255 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078b90 msgr2=0x7f96bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078b90 0x7f96bc07b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96c0034000 tx=0x7f96c0004ab0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 msgr2=0x7f96d015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 0x7f96d015bee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009f90 tx=0x7f96c400ee50 comp rx=0 tx=0).stop 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 shutdown_connections 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f96d00cad30 0x7f96d00c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f96d00ca800 0x7f96d015bee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f96bc078b90 0x7f96bc07b0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 --2- 172.21.15.22:0/2196688291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f96d00704e0 0x7f96d01525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:38.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.254+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 >> 172.21.15.22:0/2196688291 conn(0x7f96d00b1cb0 msgr2=0x7f96d0073cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:38.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.255+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 shutdown_connections 2024-09-21T05:39:38.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38.255+0000 7f96d5189640 1 -- 172.21.15.22:0/2196688291 wait complete. 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:39:38.268 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:39:38.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:38.269 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:39:38.285 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42766sec preferred_lft 42766sec 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:38.286 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:39:38.287 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add name ceph-brx type bridge 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr flush dev ceph-brx 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set ceph-brx up 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T05:39:38.287 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:38.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:38.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:38.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:38.506 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:39:38.532 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:39:38.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:38.534 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:39:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:39:38.656 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:39:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:39:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:39:38.657 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:39:38.657 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:38.657 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:38.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:39:38.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:39:38.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:39:38.658 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:38.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:38.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:38.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:38.907 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:39:38.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:38.923 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-09-21T05:39:38.980 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:38.980 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:38.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:39:38.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T05:39:38.980 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:39.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:39.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:39.099 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T05:39:39.100 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:39.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:39.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:39.223 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:39.224 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:39.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:39:39.224 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T05:39:39.224 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:39.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:39.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:39.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:39.302 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:39:39.317 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:39:39.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:39.318 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:39:39.374 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:39:39.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:39.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:39:39.483 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:39:39.483 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:39:39.483 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:39:39.613 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:39:39.613 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:39:39.613 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:39:39.613 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:39.613 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:39:39.619 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:39:39.620 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:39:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:39:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:39:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:39:39.620 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:39:39.622 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:39:39.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:39.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:39:39.648 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:39.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:39.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:39.750 INFO:teuthology.orchestra.run.smithi022.stdout:client4734 2024-09-21T05:39:39.750 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:39.751 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:39.767 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:39.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:39.869 INFO:teuthology.orchestra.run.smithi022.stdout:client4734 2024-09-21T05:39:39.869 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:39.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:39.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:39.932 INFO:teuthology.orchestra.run.smithi022.stdout:client4734 2024-09-21T05:39:39.932 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/cdac58e0-7b6b-4531-9caa-dad309e0d36b.client4734/status') 2024-09-21T05:39:39.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:39.996 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.4734 (3)192.168.0.1:0/3217644448 2024-09-21T05:39:39.996 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2024-09-21T05:39:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2024-09-21T05:39:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2024-09-21T05:39:39.996 INFO:teuthology.orchestra.run.smithi022.stderr:69 bytes copied, 2.5948e-05 s, 2.7 MB/s 2024-09-21T05:39:39.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:40.003 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-21T05:39:40.003 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:39:40.004 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T05:39:40.004 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:39:40.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 -- 172.21.15.22:0/524307340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf8006e0e0 msgr2=0x7fcf8006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:40.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 --2- 172.21.15.22:0/524307340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf8006e0e0 0x7fcf8006e4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf8008d160 tx=0x7fcf7802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 -- 172.21.15.22:0/524307340 shutdown_connections 2024-09-21T05:39:40.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 --2- 172.21.15.22:0/524307340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf8006e0e0 0x7fcf8006e4f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 --2- 172.21.15.22:0/524307340 >> v2:172.21.15.134:3300/0 conn(0x7fcf800b9970 0x7fcf800b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 -- 172.21.15.22:0/524307340 >> 172.21.15.22:0/524307340 conn(0x7fcf8009d7f0 msgr2=0x7fcf800d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:40.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 -- 172.21.15.22:0/524307340 shutdown_connections 2024-09-21T05:39:40.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.118+0000 7fcf88438640 1 -- 172.21.15.22:0/524307340 wait complete. 2024-09-21T05:39:40.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.120+0000 7fcf88438640 1 Processor -- start 2024-09-21T05:39:40.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.120+0000 7fcf88438640 1 -- start start 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf88438640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8006e0e0 0x7fcf8015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf88438640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf800b5fa0 0x7fcf800ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf88438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 0x7fcf800ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf88438640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8006ea30 con 0x7fcf800b9970 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf88438640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcf800d5010 con 0x7fcf8006e0e0 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf88438640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8006ebb0 con 0x7fcf800b5fa0 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf869d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 0x7fcf800ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf869d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 0x7fcf800ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38824/0 (socket says 172.21.15.22:38824) 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.121+0000 7fcf869d5640 1 -- 172.21.15.22:0/3055626022 learned_addr learned my addr 172.21.15.22:0/3055626022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf861d4640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8006e0e0 0x7fcf8015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf859d3640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf800b5fa0 0x7fcf800ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:40.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf869d5640 1 -- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf800b5fa0 msgr2=0x7fcf800ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf869d5640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf800b5fa0 0x7fcf800ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf869d5640 1 -- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8006e0e0 msgr2=0x7fcf8015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf869d5640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8006e0e0 0x7fcf8015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf869d5640 1 -- 172.21.15.22:0/3055626022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf801e6850 con 0x7fcf800b9970 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf859d3640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf800b5fa0 0x7fcf800ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf869d5640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 0x7fcf800ca5b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009ca0 tx=0x7fcf78031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78031990 con 0x7fcf800b9970 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf78031c20 con 0x7fcf800b9970 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf801e6b30 con 0x7fcf800b9970 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780330d0 con 0x7fcf800b9970 2024-09-21T05:39:40.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.122+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf801e5bf0 con 0x7fcf800b9970 2024-09-21T05:39:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.123+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf801e6d70 con 0x7fcf800b9970 2024-09-21T05:39:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.124+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf78033270 con 0x7fcf800b9970 2024-09-21T05:39:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.125+0000 7fcf76ffd640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcf64078b90 0x7fcf6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.125+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcf780bc220 con 0x7fcf800b9970 2024-09-21T05:39:40.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.125+0000 7fcf861d4640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcf64078b90 0x7fcf6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:40.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.126+0000 7fcf861d4640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcf64078b90 0x7fcf6407b0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00bb30 tx=0x7fcf7c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:40.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.129+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcf7803f030 con 0x7fcf800b9970 2024-09-21T05:39:40.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.358+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcf8006e4f0 con 0x7fcf800b9970 2024-09-21T05:39:40.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.359+0000 7fcf76ffd640 1 -- 172.21.15.22:0/3055626022 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fcf78084660 con 0x7fcf800b9970 2024-09-21T05:39:40.361 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:39:40.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcf64078b90 msgr2=0x7fcf6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcf64078b90 0x7fcf6407b0a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00bb30 tx=0x7fcf7c00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 msgr2=0x7fcf800ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 0x7fcf800ca5b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009ca0 tx=0x7fcf78031760 comp rx=0 tx=0).stop 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 shutdown_connections 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcf800b9970 0x7fcf800ca5b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcf64078b90 0x7fcf6407b0a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcf800b5fa0 0x7fcf800ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 --2- 172.21.15.22:0/3055626022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcf8006e0e0 0x7fcf8015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 >> 172.21.15.22:0/3055626022 conn(0x7fcf8009d7f0 msgr2=0x7fcf801e7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 shutdown_connections 2024-09-21T05:39:40.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40.361+0000 7fcf88438640 1 -- 172.21.15.22:0/3055626022 wait complete. 2024-09-21T05:39:40.374 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:39:40.374 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:39:40.374 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:39:40.374 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T05:39:40.375 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T05:39:40.375 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:39:40.375 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:39:40.375 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:39:40.375 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T05:39:40.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.375 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:39:40.391 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:40.392 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:40.392 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:40.392 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42763sec preferred_lft 42763sec 2024-09-21T05:39:40.393 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:39:40.394 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:39:40.395 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link tentative 2024-09-21T05:39:40.395 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.395 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:39:40.457 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:39:40.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.458 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:39:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:39:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:39:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:39:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:39:40.515 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:39:40.515 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:40.515 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:40.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:39:40.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:39:40.516 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:39:40.516 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:40.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:40.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:40.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.644 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:39:40.659 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:39:40.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.660 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list-id 2024-09-21T05:39:40.716 INFO:teuthology.orchestra.run.smithi022.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T05:39:40.716 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:40.716 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:40.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:39:40.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T05:39:40.717 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:40.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:40.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:40.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.835 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T05:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:40.861 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 7e:90:d2:b8:c1:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::7c90:d2ff:feb8:c1dd/64 scope link 2024-09-21T05:39:40.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:40.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T05:39:40.929 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 7e:90:d2:b8:c1:dd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::7c90:d2ff:feb8:c1dd/64 scope link 2024-09-21T05:39:40.930 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:40.932 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T05:39:40.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T05:39:40.933 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:41.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:41.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:41.088 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:41.088 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:39:41.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.1 up 2024-09-21T05:39:41.088 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T05:39:41.088 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:39:41.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:41.167 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:41.167 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T05:39:41.182 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T05:39:41.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:41.183 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T05:39:41.235 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:39:41.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:41.236 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:39:41.332 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:39:41.332 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T05:39:41.332 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:39:41.343 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:39:41.343 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T05:39:41.343 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:39:41.343 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:41.343 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:41.352 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T05:39:41.352 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T05:39:41.352 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:39:41.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:41.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:41.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:39:41.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:39:41.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:39:41.353 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T05:39:41.354 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:39:41.354 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:39:41.354 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:39:41.354 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:39:41.355 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:39:41.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:41.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T05:39:41.382 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:41.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:41.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:41.479 INFO:teuthology.orchestra.run.smithi022.stdout:client4639 2024-09-21T05:39:41.479 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:41.479 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:41.496 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:41.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:41.592 INFO:teuthology.orchestra.run.smithi022.stdout:client4639 2024-09-21T05:39:41.593 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:41.649 INFO:teuthology.orchestra.run.smithi022.stdout:client4639 2024-09-21T05:39:41.649 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/cdac58e0-7b6b-4531-9caa-dad309e0d36b.client4639/status') 2024-09-21T05:39:41.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:41.709 INFO:teuthology.orchestra.run.smithi022.stdout:instance: client.4639 (3)172.21.15.22:0/3633220937 2024-09-21T05:39:41.709 INFO:teuthology.orchestra.run.smithi022.stdout:blocklisted: false 2024-09-21T05:39:41.709 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records in 2024-09-21T05:39:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:0+1 records out 2024-09-21T05:39:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:70 bytes copied, 2.7398e-05 s, 2.6 MB/s 2024-09-21T05:39:41.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:41.716 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T05:39:41.717 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:39:41.717 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T05:39:41.717 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:39:41.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 -- 172.21.15.22:0/1104921117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240d12c0 msgr2=0x7f2b240d16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 --2- 172.21.15.22:0/1104921117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240d12c0 0x7f2b240d16d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005fd0 tx=0x7f2b1c02a530 comp rx=0 tx=0).stop 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 -- 172.21.15.22:0/1104921117 shutdown_connections 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 --2- 172.21.15.22:0/1104921117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240d12c0 0x7f2b240d16d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 --2- 172.21.15.22:0/1104921117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240ccbd0 0x7f2b240d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 -- 172.21.15.22:0/1104921117 >> 172.21.15.22:0/1104921117 conn(0x7f2b24072ae0 msgr2=0x7f2b24072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 -- 172.21.15.22:0/1104921117 shutdown_connections 2024-09-21T05:39:41.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.831+0000 7f2b28a77640 1 -- 172.21.15.22:0/1104921117 wait complete. 2024-09-21T05:39:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.833+0000 7f2b28a77640 1 Processor -- start 2024-09-21T05:39:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.833+0000 7f2b28a77640 1 -- start start 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.834+0000 7f2b28a77640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240b51c0 0x7f2b240b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.834+0000 7f2b28a77640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 0x7f2b241f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.834+0000 7f2b28a77640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240d12c0 0x7f2b241f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.834+0000 7f2b28a77640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2b240cf220 con 0x7f2b240ccbd0 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.834+0000 7f2b28a77640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2b240cf0a0 con 0x7f2b240b51c0 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.834+0000 7f2b28a77640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2b240cf3a0 con 0x7f2b240d12c0 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 0x7f2b241f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 0x7f2b241f46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38844/0 (socket says 172.21.15.22:38844) 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 -- 172.21.15.22:0/117741781 learned_addr learned my addr 172.21.15.22:0/117741781 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b22d9d640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240d12c0 0x7f2b241f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b2259c640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240b51c0 0x7f2b240b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:41.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 -- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240d12c0 msgr2=0x7f2b241f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240d12c0 0x7f2b241f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 -- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240b51c0 msgr2=0x7f2b240b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240b51c0 0x7f2b240b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 -- 172.21.15.22:0/117741781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b240cbd90 con 0x7f2b240ccbd0 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b2259c640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240b51c0 0x7f2b240b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b22d9d640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240d12c0 0x7f2b241f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b21d9b640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 0x7f2b241f46d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00d6d0 tx=0x7f2b0c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c017070 con 0x7f2b240ccbd0 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b0c005310 con 0x7f2b240ccbd0 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b240c6ff0 con 0x7f2b240ccbd0 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c009820 con 0x7f2b240ccbd0 2024-09-21T05:39:41.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.835+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b240c75b0 con 0x7f2b240ccbd0 2024-09-21T05:39:41.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.837+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b240cbb00 con 0x7f2b240ccbd0 2024-09-21T05:39:41.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.837+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b0c0099c0 con 0x7f2b240ccbd0 2024-09-21T05:39:41.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.837+0000 7f2b1affd640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b04078b90 0x7f2b0407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:41.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.838+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2b0c09aeb0 con 0x7f2b240ccbd0 2024-09-21T05:39:41.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.838+0000 7f2b2259c640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b04078b90 0x7f2b0407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:41.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.838+0000 7f2b2259c640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b04078b90 0x7f2b0407b0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b14007e90 tx=0x7f2b140073a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:41.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:41.842+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b0c063270 con 0x7f2b240ccbd0 2024-09-21T05:39:42.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.069+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2b240cf890 con 0x7f2b240ccbd0 2024-09-21T05:39:42.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.070+0000 7f2b1affd640 1 -- 172.21.15.22:0/117741781 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f2b0c062b70 con 0x7f2b240ccbd0 2024-09-21T05:39:42.072 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b04078b90 msgr2=0x7f2b0407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b04078b90 0x7f2b0407b0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b14007e90 tx=0x7f2b140073a0 comp rx=0 tx=0).stop 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 msgr2=0x7f2b241f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 0x7f2b241f46d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00d6d0 tx=0x7f2b0c00db90 comp rx=0 tx=0).stop 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 shutdown_connections 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b04078b90 0x7f2b0407b0a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:42.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2b240d12c0 0x7f2b241f4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2b240ccbd0 0x7f2b241f46d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 --2- 172.21.15.22:0/117741781 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2b240b51c0 0x7f2b240b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 >> 172.21.15.22:0/117741781 conn(0x7f2b24072ae0 msgr2=0x7f2b240b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 shutdown_connections 2024-09-21T05:39:42.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:42.072+0000 7f2b28a77640 1 -- 172.21.15.22:0/117741781 wait complete. 2024-09-21T05:39:42.085 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:39:42.085 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:39:42.085 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-09-21T05:39:42.085 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T05:39:42.085 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T05:39:42.086 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:39:42.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:39:42.086 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:39:42.086 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T05:39:42.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:42.086 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-09-21T05:39:42.106 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:42.106 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:42.106 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:42.106 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:42.106 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:42.106 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:42.107 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:42.108 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:39:42.108 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:42.108 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T05:39:42.108 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42772sec preferred_lft 42772sec 2024-09-21T05:39:42.108 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::ec4:7aff:fe88:6de9/64 scope link noprefixroute 2024-09-21T05:39:42.108 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:42.109 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T05:39:42.109 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:42.109 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:42.109 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add name ceph-brx type bridge 2024-09-21T05:39:42.109 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr flush dev ceph-brx 2024-09-21T05:39:42.109 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set ceph-brx up 2024-09-21T05:39:42.110 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T05:39:42.110 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:42.203 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:42.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:42.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:42.328 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:39:42.354 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-09-21T05:39:42.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:42.354 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-09-21T05:39:42.424 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-09-21T05:39:42.424 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:39:42.424 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T05:39:42.424 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T05:39:42.424 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:39:42.425 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:42.425 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:42.425 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T05:39:42.425 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T05:39:42.425 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T05:39:42.425 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:42.511 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:42.749 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:42.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:42.757 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-09-21T05:39:42.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:42.773 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-09-21T05:39:42.829 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:42.829 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:42.829 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T05:39:42.829 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T05:39:42.829 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:42.918 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:42.947 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:42.955 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T05:39:42.955 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T05:39:42.956 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:43.005 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:43.085 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:43.091 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:43.091 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:43.091 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.0 up 2024-09-21T05:39:43.091 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T05:39:43.091 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:43.139 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:43.162 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:43.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:43.168 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T05:39:43.183 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T05:39:43.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:43.183 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T05:39:43.239 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:39:43.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:43.239 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:39:43.347 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:39:43.347 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T05:39:43.347 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-09-21T05:39:43.482 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:39:43.482 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T05:39:43.482 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-09-21T05:39:43.482 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:43.482 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:39:43.489 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:39:43.490 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T05:39:43.490 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:39:43.490 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:39:43.490 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:39:43.490 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:39:43.492 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:39:43.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:43.492 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T05:39:43.516 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:43.604 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:43.606 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:43.612 INFO:teuthology.orchestra.run.smithi134.stdout:client4752 2024-09-21T05:39:43.612 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:43.612 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:43.627 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:43.715 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:43.717 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:43.723 INFO:teuthology.orchestra.run.smithi134.stdout:client4752 2024-09-21T05:39:43.723 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:43.771 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:43.773 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:43.779 INFO:teuthology.orchestra.run.smithi134.stdout:client4752 2024-09-21T05:39:43.779 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/cdac58e0-7b6b-4531-9caa-dad309e0d36b.client4752/status') 2024-09-21T05:39:43.828 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:43.838 INFO:teuthology.orchestra.run.smithi134.stdout:instance: client.4752 (3)172.21.15.134:0/2191653253 2024-09-21T05:39:43.838 INFO:teuthology.orchestra.run.smithi134.stdout:blocklisted: false 2024-09-21T05:39:43.838 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records in 2024-09-21T05:39:43.838 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records out 2024-09-21T05:39:43.838 INFO:teuthology.orchestra.run.smithi134.stderr:71 bytes copied, 3.9818e-05 s, 1.8 MB/s 2024-09-21T05:39:43.839 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:43.845 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-21T05:39:43.845 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T05:39:43.845 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T05:39:43.845 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:39:43.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 -- 172.21.15.22:0/2131346653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c162410 msgr2=0x7f0c1c15ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:43.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2131346653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c162410 0x7f0c1c15ffe0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0c18009830 tx=0x7f0c1802a530 comp rx=0 tx=0).stop 2024-09-21T05:39:43.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 -- 172.21.15.22:0/2131346653 shutdown_connections 2024-09-21T05:39:43.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2131346653 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c162410 0x7f0c1c15ffe0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:43.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 -- 172.21.15.22:0/2131346653 >> 172.21.15.22:0/2131346653 conn(0x7f0c1c0710b0 msgr2=0x7f0c1c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:43.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 -- 172.21.15.22:0/2131346653 shutdown_connections 2024-09-21T05:39:43.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.975+0000 7f0c238cc640 1 -- 172.21.15.22:0/2131346653 wait complete. 2024-09-21T05:39:43.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.977+0000 7f0c238cc640 1 Processor -- start 2024-09-21T05:39:43.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.977+0000 7f0c238cc640 1 -- start start 2024-09-21T05:39:43.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c238cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c0cf180 0x7f0c1c1f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c238cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c162410 0x7f0c1c1f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c238cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 0x7f0c1c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c238cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c06f4c0 con 0x7f0c1c1ea610 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c238cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c06f340 con 0x7f0c1c0cf180 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c238cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c06f640 con 0x7f0c1c162410 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c21e69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 0x7f0c1c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c21e69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 0x7f0c1c0c9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:38854/0 (socket says 172.21.15.22:38854) 2024-09-21T05:39:43.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c21668640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c0cf180 0x7f0c1c1f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c21e69640 1 -- 172.21.15.22:0/2014683349 learned_addr learned my addr 172.21.15.22:0/2014683349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.978+0000 7f0c20e67640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c162410 0x7f0c1c1f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c21e69640 1 -- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c162410 msgr2=0x7f0c1c1f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c21e69640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c162410 0x7f0c1c1f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c21e69640 1 -- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c0cf180 msgr2=0x7f0c1c1f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c21e69640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c0cf180 0x7f0c1c1f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c21e69640 1 -- 172.21.15.22:0/2014683349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c1f6b30 con 0x7f0c1c1ea610 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c21e69640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 0x7f0c1c0c9730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400cae0 tx=0x7f0c1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14016020 con 0x7f0c1c1ea610 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c1400acf0 con 0x7f0c1c1ea610 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c1f5d10 con 0x7f0c1c1ea610 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c14012680 con 0x7f0c1c1ea610 2024-09-21T05:39:43.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.979+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c1f62d0 con 0x7f0c1c1ea610 2024-09-21T05:39:43.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.981+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c15ffe0 con 0x7f0c1c1ea610 2024-09-21T05:39:43.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.981+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c14019020 con 0x7f0c1c1ea610 2024-09-21T05:39:43.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.982+0000 7f0c11ffb640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0c00078bc0 0x7f0c0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:43.986 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.982+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1409b310 con 0x7f0c1c1ea610 2024-09-21T05:39:43.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.982+0000 7f0c21668640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0c00078bc0 0x7f0c0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:43.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.983+0000 7f0c21668640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0c00078bc0 0x7f0c0007b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0c18034000 tx=0x7f0c18004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:43.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:43.987+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c14020020 con 0x7f0c1c1ea610 2024-09-21T05:39:44.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.214+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c1c06def0 con 0x7f0c1c1ea610 2024-09-21T05:39:44.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.214+0000 7f0c11ffb640 1 -- 172.21.15.22:0/2014683349 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f0c14063060 con 0x7f0c1c1ea610 2024-09-21T05:39:44.217 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0c00078bc0 msgr2=0x7f0c0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0c00078bc0 0x7f0c0007b0d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0c18034000 tx=0x7f0c18004730 comp rx=0 tx=0).stop 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 msgr2=0x7f0c1c0c9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 0x7f0c1c0c9730 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c1400cae0 tx=0x7f0c1400a060 comp rx=0 tx=0).stop 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 shutdown_connections 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0c1c1ea610 0x7f0c1c0c9730 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0c00078bc0 0x7f0c0007b0d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0c1c162410 0x7f0c1c1f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:44.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 --2- 172.21.15.22:0/2014683349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0c1c0cf180 0x7f0c1c1f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:44.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 >> 172.21.15.22:0/2014683349 conn(0x7f0c1c0710b0 msgr2=0x7f0c1c1ead60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:44.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 shutdown_connections 2024-09-21T05:39:44.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:44.217+0000 7f0c238cc640 1 -- 172.21.15.22:0/2014683349 wait complete. 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi134.front.sepia.ceph.com 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:39:44.230 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T05:39:44.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.231 DEBUG:teuthology.orchestra.run.smithi134:> ip addr 2024-09-21T05:39:44.247 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:44.247 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:44.247 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:44.247 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.247 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e6 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f0 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:d9:38:e7 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: altname enp7s0f1 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e8 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether 0c:c4:7a:88:6d:e9 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: inet 172.21.15.134/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T05:39:44.248 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft 42770sec preferred_lft 42770sec 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::ec4:7aff:fe88:6de9/64 scope link noprefixroute 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c6:01:65:88:df:04 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::8cf5:2ff:febe:bba7/64 scope link 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c6:01:65:88:df:04 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c401:65ff:fe88:df04/64 scope link tentative 2024-09-21T05:39:44.249 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.250 DEBUG:teuthology.orchestra.run.smithi134:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:39:44.310 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-09-21T05:39:44.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.312 DEBUG:teuthology.orchestra.run.smithi134:> route 2024-09-21T05:39:44.368 INFO:teuthology.orchestra.run.smithi134.stdout:Kernel IP routing table 2024-09-21T05:39:44.368 INFO:teuthology.orchestra.run.smithi134.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:39:44.369 INFO:teuthology.orchestra.run.smithi134.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T05:39:44.369 INFO:teuthology.orchestra.run.smithi134.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T05:39:44.369 INFO:teuthology.orchestra.run.smithi134.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:39:44.369 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:44.369 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:44.369 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T05:39:44.369 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T05:39:44.369 DEBUG:teuthology.orchestra.run.smithi134:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T05:39:44.369 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:44.456 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:44.489 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:44.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.495 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-09-21T05:39:44.511 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T05:39:44.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.512 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list-id 2024-09-21T05:39:44.566 INFO:teuthology.orchestra.run.smithi134.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T05:39:44.567 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:44.567 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:44.567 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T05:39:44.567 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T05:39:44.567 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:44.654 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:44.677 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:44.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.683 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T05:39:44.707 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c2:72:5c:38:f1:c4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c072:5cff:fe38:f1c4/64 scope link tentative 2024-09-21T05:39:44.708 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:44.709 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T05:39:44.774 INFO:teuthology.orchestra.run.smithi134.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:39:44.774 INFO:teuthology.orchestra.run.smithi134.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:39:44.774 INFO:teuthology.orchestra.run.smithi134.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:39:44.774 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.774 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 ::1/128 scope host 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout: link/ether c2:72:5c:38:f1:c4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout: inet6 fe80::c072:5cff:fe38:f1c4/64 scope link tentative 2024-09-21T05:39:44.775 INFO:teuthology.orchestra.run.smithi134.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:39:44.776 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T05:39:44.776 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T05:39:44.777 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:44.864 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:44.923 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:44.930 DEBUG:teuthology.orchestra.run.smithi134:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:39:44.930 DEBUG:teuthology.orchestra.run.smithi134:> set -e 2024-09-21T05:39:44.930 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set brx.1 up 2024-09-21T05:39:44.930 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T05:39:44.930 DEBUG:teuthology.orchestra.run.smithi134:> ') 2024-09-21T05:39:44.978 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:45.000 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:45.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:45.007 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T05:39:45.022 INFO:teuthology.orchestra.run.smithi134.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T05:39:45.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:39:45.022 DEBUG:teuthology.orchestra.run.smithi134:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T05:39:45.077 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:39:45.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:45.078 DEBUG:teuthology.orchestra.run.smithi134:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:39:45.172 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:39:45.172 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T05:39:45.172 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-09-21T05:39:45.183 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:39:45.183 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T05:39:45.183 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-09-21T05:39:45.183 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:45.183 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:invalid new device string format 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:39:45.202 INFO:teuthology.orchestra.run.smithi134.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:39:45.203 INFO:teuthology.orchestra.run.smithi134.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T05:39:45.203 INFO:teuthology.orchestra.run.smithi134.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:39:45.203 INFO:teuthology.orchestra.run.smithi134.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:39:45.203 INFO:teuthology.orchestra.run.smithi134.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:39:45.203 INFO:teuthology.orchestra.run.smithi134.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:39:45.204 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:39:45.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:45.205 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T05:39:45.229 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:45.317 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:45.320 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:45.325 INFO:teuthology.orchestra.run.smithi134.stdout:client4651 2024-09-21T05:39:45.326 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:45.326 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:45.341 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:45.429 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:45.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:45.437 INFO:teuthology.orchestra.run.smithi134.stdout:client4651 2024-09-21T05:39:45.437 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:39:45.485 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:39:45.487 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:45.492 INFO:teuthology.orchestra.run.smithi134.stdout:client4651 2024-09-21T05:39:45.493 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/cdac58e0-7b6b-4531-9caa-dad309e0d36b.client4651/status') 2024-09-21T05:39:45.540 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:39:45.550 INFO:teuthology.orchestra.run.smithi134.stdout:instance: client.4651 (3)192.168.0.2:0/2641280795 2024-09-21T05:39:45.550 INFO:teuthology.orchestra.run.smithi134.stdout:blocklisted: false 2024-09-21T05:39:45.550 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records in 2024-09-21T05:39:45.550 INFO:teuthology.orchestra.run.smithi134.stderr:0+1 records out 2024-09-21T05:39:45.550 INFO:teuthology.orchestra.run.smithi134.stderr:69 bytes copied, 3.8348e-05 s, 1.8 MB/s 2024-09-21T05:39:45.551 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-21T05:39:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:39:45.557 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T05:39:46.105 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:39:46.106 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:46.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 -- 172.21.15.22:0/330909210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840ccbd0 msgr2=0x7f3f840d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/330909210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840ccbd0 0x7f3f840d0d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3f840cd8d0 tx=0x7f3f7402a5f0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 -- 172.21.15.22:0/330909210 shutdown_connections 2024-09-21T05:39:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/330909210 >> v2:172.21.15.134:3300/0 conn(0x7f3f840d12c0 0x7f3f840d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/330909210 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840ccbd0 0x7f3f840d0d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 -- 172.21.15.22:0/330909210 >> 172.21.15.22:0/330909210 conn(0x7f3f84072ae0 msgr2=0x7f3f84072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 -- 172.21.15.22:0/330909210 shutdown_connections 2024-09-21T05:39:46.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.236+0000 7f3f8b3da640 1 -- 172.21.15.22:0/330909210 wait complete. 2024-09-21T05:39:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.238+0000 7f3f8b3da640 1 Processor -- start 2024-09-21T05:39:46.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.238+0000 7f3f8b3da640 1 -- start start 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f8b3da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 0x7f3f840c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f8b3da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f840ccbd0 0x7f3f84152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f8b3da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f840d12c0 0x7f3f840a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f8b3da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f840cf220 con 0x7f3f840b51c0 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f8b3da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f840cf0a0 con 0x7f3f840d12c0 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f8b3da640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f840cf3a0 con 0x7f3f840ccbd0 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f89176640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 0x7f3f840c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f89176640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 0x7f3f840c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57448/0 (socket says 172.21.15.22:57448) 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.239+0000 7f3f89176640 1 -- 172.21.15.22:0/3186325846 learned_addr learned my addr 172.21.15.22:0/3186325846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89977640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f840d12c0 0x7f3f840a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f88975640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f840ccbd0 0x7f3f84152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89176640 1 -- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f840ccbd0 msgr2=0x7f3f84152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89176640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f840ccbd0 0x7f3f84152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89176640 1 -- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f840d12c0 msgr2=0x7f3f840a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89176640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f840d12c0 0x7f3f840a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89176640 1 -- 172.21.15.22:0/3186325846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f840c9600 con 0x7f3f840b51c0 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f89176640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 0x7f3f840c8a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c3a0 tx=0x7f3f8000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80017070 con 0x7f3f840b51c0 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f800053c0 con 0x7f3f840b51c0 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80003850 con 0x7f3f840b51c0 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f840c98e0 con 0x7f3f840b51c0 2024-09-21T05:39:46.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.240+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f840ca610 con 0x7f3f840b51c0 2024-09-21T05:39:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.241+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f840cf850 con 0x7f3f840b51c0 2024-09-21T05:39:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.242+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f8001a070 con 0x7f3f840b51c0 2024-09-21T05:39:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.243+0000 7f3f79ffb640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f68078b90 0x7f3f6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.243+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3f8009a100 con 0x7f3f840b51c0 2024-09-21T05:39:46.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.243+0000 7f3f88975640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f68078b90 0x7f3f6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.243+0000 7f3f88975640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f68078b90 0x7f3f6807b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f74033000 tx=0x7f3f74004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:46.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.247+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f80061e50 con 0x7f3f840b51c0 2024-09-21T05:39:46.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.436+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f840cc9b0 con 0x7f3f840b51c0 2024-09-21T05:39:46.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.437+0000 7f3f79ffb640 1 -- 172.21.15.22:0/3186325846 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7f3f80066880 con 0x7f3f840b51c0 2024-09-21T05:39:46.439 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:46.439 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f68078b90 msgr2=0x7f3f6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f68078b90 0x7f3f6807b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3f74033000 tx=0x7f3f74004730 comp rx=0 tx=0).stop 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 msgr2=0x7f3f840c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 0x7f3f840c8a20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000c3a0 tx=0x7f3f8000c860 comp rx=0 tx=0).stop 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 shutdown_connections 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f840d12c0 0x7f3f840a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f68078b90 0x7f3f6807b0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f840ccbd0 0x7f3f84152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 --2- 172.21.15.22:0/3186325846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f840b51c0 0x7f3f840c8a20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 >> 172.21.15.22:0/3186325846 conn(0x7f3f84072ae0 msgr2=0x7f3f840b6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:46.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 shutdown_connections 2024-09-21T05:39:46.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.439+0000 7f3f8b3da640 1 -- 172.21.15.22:0/3186325846 wait complete. 2024-09-21T05:39:46.452 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T05:39:20.094882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T05:39:21.155594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:39:46.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:39:46.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 -- 172.21.15.22:0/2382511709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 msgr2=0x7fd918004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 --2- 172.21.15.22:0/2382511709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd918004c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd91c0016c0 tx=0x7fd918005830 comp rx=0 tx=0).stop 2024-09-21T05:39:46.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 -- 172.21.15.22:0/2382511709 shutdown_connections 2024-09-21T05:39:46.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 --2- 172.21.15.22:0/2382511709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd918004c10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 -- 172.21.15.22:0/2382511709 >> 172.21.15.22:0/2382511709 conn(0x7fd92c09d880 msgr2=0x7fd92c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:46.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 -- 172.21.15.22:0/2382511709 shutdown_connections 2024-09-21T05:39:46.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.571+0000 7fd93381f640 1 -- 172.21.15.22:0/2382511709 wait complete. 2024-09-21T05:39:46.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.573+0000 7fd93381f640 1 Processor -- start 2024-09-21T05:39:46.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.573+0000 7fd93381f640 1 -- start start 2024-09-21T05:39:46.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd93381f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd92c0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd93381f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd92c1f5de0 0x7fd92c0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd93381f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92c1f61f0 0x7fd92c0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd93381f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c0d1410 con 0x7fd918004800 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd93381f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd92c0d1290 con 0x7fd92c1f61f0 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd93381f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd92c0d1590 con 0x7fd92c1f5de0 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd9315bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd92c0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd9315bb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd92c0ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57452/0 (socket says 172.21.15.22:57452) 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd9315bb640 1 -- 172.21.15.22:0/2520290 learned_addr learned my addr 172.21.15.22:0/2520290 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd930dba640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd92c1f5de0 0x7fd92c0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.574+0000 7fd931dbc640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92c1f61f0 0x7fd92c0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd9315bb640 1 -- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd92c1f5de0 msgr2=0x7fd92c0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd9315bb640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd92c1f5de0 0x7fd92c0cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd9315bb640 1 -- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92c1f61f0 msgr2=0x7fd92c0cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd9315bb640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92c1f61f0 0x7fd92c0cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd9315bb640 1 -- 172.21.15.22:0/2520290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd92c0cb8e0 con 0x7fd918004800 2024-09-21T05:39:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd9315bb640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd92c0ca750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd928011780 tx=0x7fd928011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92800fd50 con 0x7fd918004800 2024-09-21T05:39:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd928011e90 con 0x7fd918004800 2024-09-21T05:39:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd92c0cbbc0 con 0x7fd918004800 2024-09-21T05:39:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928004290 con 0x7fd918004800 2024-09-21T05:39:46.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.575+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd92c0cc180 con 0x7fd918004800 2024-09-21T05:39:46.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.576+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd92c0cf270 con 0x7fd918004800 2024-09-21T05:39:46.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.577+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9280035f0 con 0x7fd918004800 2024-09-21T05:39:46.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.578+0000 7fd921ffb640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd910078b90 0x7fd91007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.578+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd92809ba00 con 0x7fd918004800 2024-09-21T05:39:46.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.578+0000 7fd930dba640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd910078b90 0x7fd91007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.578+0000 7fd930dba640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd910078b90 0x7fd91007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd91c032000 tx=0x7fd91c02c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:46.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.582+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd9280637d0 con 0x7fd918004800 2024-09-21T05:39:46.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.768+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd92c15f460 con 0x7fd918004800 2024-09-21T05:39:46.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.769+0000 7fd921ffb640 1 -- 172.21.15.22:0/2520290 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd928068200 con 0x7fd918004800 2024-09-21T05:39:46.771 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:39:46.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd910078b90 msgr2=0x7fd91007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd910078b90 0x7fd91007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd91c032000 tx=0x7fd91c02c040 comp rx=0 tx=0).stop 2024-09-21T05:39:46.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 msgr2=0x7fd92c0ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd92c0ca750 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd928011780 tx=0x7fd928011c40 comp rx=0 tx=0).stop 2024-09-21T05:39:46.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 shutdown_connections 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd92c1f61f0 0x7fd92c0cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd910078b90 0x7fd91007b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd92c1f5de0 0x7fd92c0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 --2- 172.21.15.22:0/2520290 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd918004800 0x7fd92c0ca750 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 >> 172.21.15.22:0/2520290 conn(0x7fd92c09d880 msgr2=0x7fd92c0b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 shutdown_connections 2024-09-21T05:39:46.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.771+0000 7fd93381f640 1 -- 172.21.15.22:0/2520290 wait complete. 2024-09-21T05:39:46.784 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 -- 172.21.15.22:0/3776009055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45080ceb40 msgr2=0x7f45080cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 --2- 172.21.15.22:0/3776009055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45080ceb40 0x7f45080cef50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4500005fd0 tx=0x7f450002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 -- 172.21.15.22:0/3776009055 shutdown_connections 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 --2- 172.21.15.22:0/3776009055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45080ceb40 0x7f45080cef50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 -- 172.21.15.22:0/3776009055 >> 172.21.15.22:0/3776009055 conn(0x7f450808bfc0 msgr2=0x7f450809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 -- 172.21.15.22:0/3776009055 shutdown_connections 2024-09-21T05:39:46.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.904+0000 7f450e5df640 1 -- 172.21.15.22:0/3776009055 wait complete. 2024-09-21T05:39:46.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.906+0000 7f450e5df640 1 Processor -- start 2024-09-21T05:39:46.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.906+0000 7f450e5df640 1 -- start start 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.907+0000 7f450e5df640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45080b3540 0x7f450815da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.907+0000 7f450e5df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45080ceb40 0x7f4508160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.907+0000 7f450e5df640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 0x7f4508159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.907+0000 7f450e5df640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45080d1070 con 0x7f450815df90 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.907+0000 7f450e5df640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45080d0ef0 con 0x7f45080ceb40 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.907+0000 7f450e5df640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45080d11f0 con 0x7f45080b3540 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 0x7f4508159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 0x7f4508159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57478/0 (socket says 172.21.15.22:57478) 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 -- 172.21.15.22:0/2894425658 learned_addr learned my addr 172.21.15.22:0/2894425658 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f45077fe640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45080ceb40 0x7f4508160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f4507fff640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45080b3540 0x7f450815da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 -- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45080b3540 msgr2=0x7f450815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45080b3540 0x7f450815da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 -- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45080ceb40 msgr2=0x7f4508160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45080ceb40 0x7f4508160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 -- 172.21.15.22:0/2894425658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45080c9480 con 0x7f450815df90 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f45077fe640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45080ceb40 0x7f4508160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f4507fff640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45080b3540 0x7f450815da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450cb7c640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 0x7f4508159c00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4500009cd0 tx=0x7f4500003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500031b90 con 0x7f450815df90 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4500031e20 con 0x7f450815df90 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45080c9760 con 0x7f450815df90 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000332f0 con 0x7f450815df90 2024-09-21T05:39:46.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.908+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45080c9c00 con 0x7f450815df90 2024-09-21T05:39:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.910+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45080c9980 con 0x7f450815df90 2024-09-21T05:39:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.910+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4500031660 con 0x7f450815df90 2024-09-21T05:39:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.911+0000 7f4504ff9640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44f4078b90 0x7f44f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.911+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f45000bc2c0 con 0x7f450815df90 2024-09-21T05:39:46.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.911+0000 7f4507fff640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44f4078b90 0x7f44f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:46.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.912+0000 7f4507fff640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44f4078b90 0x7f44f407b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f44f80046b0 tx=0x7f44f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:46.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:46.915+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f450003f030 con 0x7f450815df90 2024-09-21T05:39:47.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.102+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f450806e170 con 0x7f450815df90 2024-09-21T05:39:47.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.102+0000 7f4504ff9640 1 -- 172.21.15.22:0/2894425658 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f4500084820 con 0x7f450815df90 2024-09-21T05:39:47.104 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 32 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44f4078b90 msgr2=0x7f44f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44f4078b90 0x7f44f407b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f44f80046b0 tx=0x7f44f8009040 comp rx=0 tx=0).stop 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 msgr2=0x7f4508159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 0x7f4508159c00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4500009cd0 tx=0x7f4500003ba0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 shutdown_connections 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f450815df90 0x7f4508159c00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45080ceb40 0x7f4508160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44f4078b90 0x7f44f407b0a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 --2- 172.21.15.22:0/2894425658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45080b3540 0x7f450815da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.104+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 >> 172.21.15.22:0/2894425658 conn(0x7f450808bfc0 msgr2=0x7f450806f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:47.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.105+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 shutdown_connections 2024-09-21T05:39:47.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.105+0000 7f450e5df640 1 -- 172.21.15.22:0/2894425658 wait complete. 2024-09-21T05:39:47.118 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 -- 172.21.15.22:0/958717175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d2430 msgr2=0x7f90f80cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 --2- 172.21.15.22:0/958717175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d2430 0x7f90f80cf040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f90f80b4650 tx=0x7f90e802a560 comp rx=0 tx=0).stop 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 -- 172.21.15.22:0/958717175 shutdown_connections 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 --2- 172.21.15.22:0/958717175 >> v2:172.21.15.134:3300/0 conn(0x7f90f80d0c70 0x7f90f80d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 --2- 172.21.15.22:0/958717175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f80b6020 0x7f90f80b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 --2- 172.21.15.22:0/958717175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d2430 0x7f90f80cf040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 -- 172.21.15.22:0/958717175 >> 172.21.15.22:0/958717175 conn(0x7f90f809d880 msgr2=0x7f90f808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:47.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 -- 172.21.15.22:0/958717175 shutdown_connections 2024-09-21T05:39:47.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.236+0000 7f90feb28640 1 -- 172.21.15.22:0/958717175 wait complete. 2024-09-21T05:39:47.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.237+0000 7f90feb28640 1 Processor -- start 2024-09-21T05:39:47.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.237+0000 7f90feb28640 1 -- start start 2024-09-21T05:39:47.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90feb28640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f80b6020 0x7f90f80b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90feb28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 0x7f90f815b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90feb28640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f80d2430 0x7f90f80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90feb28640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f90f81624a0 con 0x7f90f80d0c70 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90feb28640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f90f80b6970 con 0x7f90f80d2430 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90feb28640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f90f80b3900 con 0x7f90f80b6020 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 0x7f90f815b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 0x7f90f815b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57506/0 (socket says 172.21.15.22:57506) 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 -- 172.21.15.22:0/3383805498 learned_addr learned my addr 172.21.15.22:0/3383805498 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90fc8c4640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f80b6020 0x7f90f80b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90fd0c5640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f80d2430 0x7f90f80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 -- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f80b6020 msgr2=0x7f90f80b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f80b6020 0x7f90f80b3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 -- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f80d2430 msgr2=0x7f90f80ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f80d2430 0x7f90f80ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.239+0000 7f90f7fff640 1 -- 172.21.15.22:0/3383805498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f815afd0 con 0x7f90f80d0c70 2024-09-21T05:39:47.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.240+0000 7f90f7fff640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 0x7f90f815b700 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00d740 tx=0x7f90ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:47.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.240+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec017070 con 0x7f90f80d0c70 2024-09-21T05:39:47.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.240+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90ec009d20 con 0x7f90f80d0c70 2024-09-21T05:39:47.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.240+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f815e0b0 con 0x7f90f80d0c70 2024-09-21T05:39:47.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.240+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0106c0 con 0x7f90f80d0c70 2024-09-21T05:39:47.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.240+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f815daf0 con 0x7f90f80d0c70 2024-09-21T05:39:47.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.241+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f80d1080 con 0x7f90f80d0c70 2024-09-21T05:39:47.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.242+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f90ec01a070 con 0x7f90f80d0c70 2024-09-21T05:39:47.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.242+0000 7f90f57fa640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f90e4078bc0 0x7f90e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.242+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f90ec09add0 con 0x7f90f80d0c70 2024-09-21T05:39:47.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.242+0000 7f90fc8c4640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f90e4078bc0 0x7f90e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.243+0000 7f90fc8c4640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f90e4078bc0 0x7f90e407b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e8033000 tx=0x7f90e8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:47.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.247+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f90ec062c50 con 0x7f90f80d0c70 2024-09-21T05:39:47.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.432+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90f80b5a20 con 0x7f90f80d0c70 2024-09-21T05:39:47.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.433+0000 7f90f57fa640 1 -- 172.21.15.22:0/3383805498 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f90ec067680 con 0x7f90f80d0c70 2024-09-21T05:39:47.435 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 64 2024-09-21T05:39:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f90e4078bc0 msgr2=0x7f90e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f90e4078bc0 0x7f90e407b0d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90e8033000 tx=0x7f90e8004730 comp rx=0 tx=0).stop 2024-09-21T05:39:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 msgr2=0x7f90f815b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 0x7f90f815b700 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00d740 tx=0x7f90ec00dc00 comp rx=0 tx=0).stop 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 shutdown_connections 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f90f80d2430 0x7f90f80ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f90f80d0c70 0x7f90f815b700 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f90e4078bc0 0x7f90e407b0d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 --2- 172.21.15.22:0/3383805498 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f90f80b6020 0x7f90f80b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 >> 172.21.15.22:0/3383805498 conn(0x7f90f809d880 msgr2=0x7f90f80b9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 shutdown_connections 2024-09-21T05:39:47.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.435+0000 7f90feb28640 1 -- 172.21.15.22:0/3383805498 wait complete. 2024-09-21T05:39:47.448 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-09-21T05:39:47.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db39c6640 1 -- 172.21.15.22:0/576516173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d94003a30 msgr2=0x7f0d94003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db39c6640 1 --2- 172.21.15.22:0/576516173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d94003a30 0x7f0d94003ea0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c630 tx=0x7f0d94005830 comp rx=0 tx=0).stop 2024-09-21T05:39:47.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db1f63640 1 -- 172.21.15.22:0/576516173 reap_dead start 2024-09-21T05:39:47.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db39c6640 1 -- 172.21.15.22:0/576516173 shutdown_connections 2024-09-21T05:39:47.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db39c6640 1 -- 172.21.15.22:0/576516173 >> 172.21.15.22:0/576516173 conn(0x7f0dac09d8f0 msgr2=0x7f0dac0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:47.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db39c6640 1 -- 172.21.15.22:0/576516173 shutdown_connections 2024-09-21T05:39:47.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.564+0000 7f0db39c6640 1 -- 172.21.15.22:0/576516173 wait complete. 2024-09-21T05:39:47.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.566+0000 7f0db39c6640 1 Processor -- start 2024-09-21T05:39:47.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.566+0000 7f0db39c6640 1 -- start start 2024-09-21T05:39:47.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db39c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d94003a30 0x7f0dac0c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db39c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dac154af0 0x7f0dac1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db39c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 0x7f0dac15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db39c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac0cfdb0 con 0x7f0dac15d180 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db39c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0dac0cfc30 con 0x7f0dac154af0 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db39c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0dac0cff30 con 0x7f0d94003a30 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 0x7f0dac15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 0x7f0dac15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57516/0 (socket says 172.21.15.22:57516) 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 -- 172.21.15.22:0/250569649 learned_addr learned my addr 172.21.15.22:0/250569649 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db0f61640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dac154af0 0x7f0dac1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1762640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d94003a30 0x7f0dac0c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 -- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d94003a30 msgr2=0x7f0dac0c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d94003a30 0x7f0dac0c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 -- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dac154af0 msgr2=0x7f0dac1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dac154af0 0x7f0dac1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.567+0000 7f0db1f63640 1 -- 172.21.15.22:0/250569649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dac15dad0 con 0x7f0dac15d180 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.568+0000 7f0db1f63640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 0x7f0dac15d590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0d3e00 tx=0x7f0da400fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.568+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4011910 con 0x7f0dac15d180 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.568+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0da4011bc0 con 0x7f0dac15d180 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.568+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dac0c9ce0 con 0x7f0dac15d180 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.568+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4010430 con 0x7f0dac15d180 2024-09-21T05:39:47.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.568+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dac0b4180 con 0x7f0dac15d180 2024-09-21T05:39:47.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.569+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac0cf310 con 0x7f0dac15d180 2024-09-21T05:39:47.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.570+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0da4019020 con 0x7f0dac15d180 2024-09-21T05:39:47.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.570+0000 7f0da9ffb640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d98078bc0 0x7f0d9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:47.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.570+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0da409b3d0 con 0x7f0dac15d180 2024-09-21T05:39:47.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.571+0000 7f0db1762640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d98078bc0 0x7f0d9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:47.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.571+0000 7f0db1762640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d98078bc0 0x7f0d9807b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c033000 tx=0x7f0d9c001d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:47.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.574+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0dac15dad0 con 0x7f0dac15d180 2024-09-21T05:39:47.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.836+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f0dac0b3fb0 con 0x7f0dac15d180 2024-09-21T05:39:47.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.883+0000 7f0da9ffb640 1 -- 172.21.15.22:0/250569649 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v43) ==== 145+0+0 (secure 0 0 0) 0x7f0da40632c0 con 0x7f0dac15d180 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d98078bc0 msgr2=0x7f0d9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d98078bc0 0x7f0d9807b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c033000 tx=0x7f0d9c001d70 comp rx=0 tx=0).stop 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 msgr2=0x7f0dac15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 0x7f0dac15d590 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0d3e00 tx=0x7f0da400fa20 comp rx=0 tx=0).stop 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 shutdown_connections 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dac15d180 0x7f0dac15d590 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dac154af0 0x7f0dac1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d98078bc0 0x7f0d9807b0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 --2- 172.21.15.22:0/250569649 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d94003a30 0x7f0dac0c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:47.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 >> 172.21.15.22:0/250569649 conn(0x7f0dac09d8f0 msgr2=0x7f0dac0b9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:47.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 shutdown_connections 2024-09-21T05:39:47.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:47.886+0000 7f0db39c6640 1 -- 172.21.15.22:0/250569649 wait complete. 2024-09-21T05:39:47.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:39:48.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 -- 172.21.15.22:0/4012551875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd006e0e0 msgr2=0x7febd006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 --2- 172.21.15.22:0/4012551875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd006e0e0 0x7febd006e4f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7febd008d160 tx=0x7febc002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 -- 172.21.15.22:0/4012551875 shutdown_connections 2024-09-21T05:39:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 --2- 172.21.15.22:0/4012551875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd006e0e0 0x7febd006e4f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 --2- 172.21.15.22:0/4012551875 >> v2:172.21.15.134:3300/0 conn(0x7febd00b9970 0x7febd00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 -- 172.21.15.22:0/4012551875 >> 172.21.15.22:0/4012551875 conn(0x7febd009d7f0 msgr2=0x7febd00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 -- 172.21.15.22:0/4012551875 shutdown_connections 2024-09-21T05:39:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.013+0000 7febd48ec640 1 -- 172.21.15.22:0/4012551875 wait complete. 2024-09-21T05:39:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.015+0000 7febd48ec640 1 Processor -- start 2024-09-21T05:39:48.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.015+0000 7febd48ec640 1 -- start start 2024-09-21T05:39:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.016+0000 7febd48ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006e0e0 0x7febd009e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febd48ec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febd00b5fa0 0x7febd0152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febd48ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 0x7febd00b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febd48ec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7febd006ea30 con 0x7febd00b9970 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febd48ec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febd00d5010 con 0x7febd006e0e0 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febd48ec640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7febd006ebb0 con 0x7febd00b5fa0 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 0x7febd00b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 0x7febd00b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57524/0 (socket says 172.21.15.22:57524) 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 -- 172.21.15.22:0/2815863038 learned_addr learned my addr 172.21.15.22:0/2815863038 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febce59c640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006e0e0 0x7febd009e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febcdd9b640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febd00b5fa0 0x7febd0152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 -- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febd00b5fa0 msgr2=0x7febd0152a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febd00b5fa0 0x7febd0152a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 -- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006e0e0 msgr2=0x7febd009e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006e0e0 0x7febd009e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 -- 172.21.15.22:0/2815863038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd015e260 con 0x7febd00b9970 2024-09-21T05:39:48.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febce59c640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006e0e0 0x7febd009e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febced9d640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 0x7febd00b4d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7febc002aea0 tx=0x7febc0003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc0031990 con 0x7febd00b9970 2024-09-21T05:39:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febc0031c20 con 0x7febd00b9970 2024-09-21T05:39:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd00cd490 con 0x7febd00b9970 2024-09-21T05:39:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.017+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc00330d0 con 0x7febd00b9970 2024-09-21T05:39:48.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.018+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd00cdda0 con 0x7febd00b9970 2024-09-21T05:39:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.019+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd00cd200 con 0x7febd00b9970 2024-09-21T05:39:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.019+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febc0033290 con 0x7febd00b9970 2024-09-21T05:39:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.020+0000 7febc6ffd640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febb0078b90 0x7febb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.020+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7febc00bc400 con 0x7febd00b9970 2024-09-21T05:39:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.020+0000 7febce59c640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febb0078b90 0x7febb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.021+0000 7febce59c640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febb0078b90 0x7febb007b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7febc8005fd0 tx=0x7febc8005ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:48.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.024+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7febc003f030 con 0x7febd00b9970 2024-09-21T05:39:48.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.211+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7febd006e4f0 con 0x7febd00b9970 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.212+0000 7febc6ffd640 1 -- 172.21.15.22:0/2815863038 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7febc00847b0 con 0x7febd00b9970 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:39:48.214 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:39:48.215 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:39:48.216 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febb0078b90 msgr2=0x7febb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febb0078b90 0x7febb007b0a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7febc8005fd0 tx=0x7febc8005ee0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 msgr2=0x7febd00b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 0x7febd00b4d30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7febc002aea0 tx=0x7febc0003990 comp rx=0 tx=0).stop 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 shutdown_connections 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febd00b9970 0x7febd00b4d30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febb0078b90 0x7febb007b0a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febd00b5fa0 0x7febd0152a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 --2- 172.21.15.22:0/2815863038 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febd006e0e0 0x7febd009e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 >> 172.21.15.22:0/2815863038 conn(0x7febd009d7f0 msgr2=0x7febd00cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 shutdown_connections 2024-09-21T05:39:48.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.214+0000 7febd48ec640 1 -- 172.21.15.22:0/2815863038 wait complete. 2024-09-21T05:39:48.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:39:48.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 -- 172.21.15.22:0/4225132438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 msgr2=0x7fec0000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 --2- 172.21.15.22:0/4225132438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec0000b9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7febfc009b00 tx=0x7fec00005830 comp rx=0 tx=0).stop 2024-09-21T05:39:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 -- 172.21.15.22:0/4225132438 shutdown_connections 2024-09-21T05:39:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 --2- 172.21.15.22:0/4225132438 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec0000b9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 -- 172.21.15.22:0/4225132438 >> 172.21.15.22:0/4225132438 conn(0x7fec1408bfc0 msgr2=0x7fec1409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 -- 172.21.15.22:0/4225132438 shutdown_connections 2024-09-21T05:39:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.345+0000 7fec1b238640 1 -- 172.21.15.22:0/4225132438 wait complete. 2024-09-21T05:39:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.347+0000 7fec1b238640 1 Processor -- start 2024-09-21T05:39:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.347+0000 7fec1b238640 1 -- start start 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.348+0000 7fec1b238640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec141602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.348+0000 7fec1b238640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec141f8ca0 0x7fec141f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.348+0000 7fec1b238640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec141f2520 0x7fec141f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.348+0000 7fec1b238640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fec140d0f90 con 0x7fec00005250 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.348+0000 7fec1b238640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fec140d0e10 con 0x7fec141f2520 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.348+0000 7fec1b238640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fec140d1110 con 0x7fec141f8ca0 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec141602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec141602f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57538/0 (socket says 172.21.15.22:57538) 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 -- 172.21.15.22:0/3860308922 learned_addr learned my addr 172.21.15.22:0/3860308922 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec197d5640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec141f2520 0x7fec141f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec13fff640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec141f8ca0 0x7fec141f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 -- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec141f8ca0 msgr2=0x7fec141f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec141f8ca0 0x7fec141f90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 -- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec141f2520 msgr2=0x7fec141f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec141f2520 0x7fec141f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 -- 172.21.15.22:0/3860308922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec14160e60 con 0x7fec00005250 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec18fd4640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec141602f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00e890 tx=0x7fec0c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c015070 con 0x7fec00005250 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec0c00cd40 con 0x7fec00005250 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec14161140 con 0x7fec00005250 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0116c0 con 0x7fec00005250 2024-09-21T05:39:48.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.349+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec14161700 con 0x7fec00005250 2024-09-21T05:39:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.350+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1406f910 con 0x7fec00005250 2024-09-21T05:39:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.351+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec0c018070 con 0x7fec00005250 2024-09-21T05:39:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.352+0000 7fec117fa640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febf4078b90 0x7febf407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.352+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec0c09b3b0 con 0x7fec00005250 2024-09-21T05:39:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.352+0000 7fec13fff640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febf4078b90 0x7febf407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.352+0000 7fec13fff640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febf4078b90 0x7febf407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec140a2430 tx=0x7fec04006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.356+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec0c063910 con 0x7fec00005250 2024-09-21T05:39:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.543+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fec1415ea30 con 0x7fec00005250 2024-09-21T05:39:48.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.547+0000 7fec117fa640 1 -- 172.21.15.22:0/3860308922 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fec0c063210 con 0x7fec00005250 2024-09-21T05:39:48.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febf4078b90 msgr2=0x7febf407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febf4078b90 0x7febf407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec140a2430 tx=0x7fec04006b70 comp rx=0 tx=0).stop 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 msgr2=0x7fec141602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec141602f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00e890 tx=0x7fec0c00ed50 comp rx=0 tx=0).stop 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 shutdown_connections 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fec141f2520 0x7fec141f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7febf4078b90 0x7febf407b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fec141f8ca0 0x7fec141f90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 --2- 172.21.15.22:0/3860308922 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fec00005250 0x7fec141602f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 >> 172.21.15.22:0/3860308922 conn(0x7fec1408bfc0 msgr2=0x7fec14068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:48.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 shutdown_connections 2024-09-21T05:39:48.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.549+0000 7fec1b238640 1 -- 172.21.15.22:0/3860308922 wait complete. 2024-09-21T05:39:48.563 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:39:48.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:39:48.605 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:39:48.645 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:39:48.645 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:48.645 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:48.662 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:48.662 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:48.718 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:48.718 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:48.774 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:39:48.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 -- 172.21.15.22:0/2904058366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f28004800 msgr2=0x7f8f28004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2904058366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f28004800 0x7f8f28004c10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009b20 tx=0x7f8f28005830 comp rx=0 tx=0).stop 2024-09-21T05:39:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 -- 172.21.15.22:0/2904058366 shutdown_connections 2024-09-21T05:39:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2904058366 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f28004800 0x7f8f28004c10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 -- 172.21.15.22:0/2904058366 >> 172.21.15.22:0/2904058366 conn(0x7f8f400b1cb0 msgr2=0x7f8f400b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 -- 172.21.15.22:0/2904058366 shutdown_connections 2024-09-21T05:39:48.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.931+0000 7f8f482e5640 1 -- 172.21.15.22:0/2904058366 wait complete. 2024-09-21T05:39:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.933+0000 7f8f482e5640 1 Processor -- start 2024-09-21T05:39:48.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.933+0000 7f8f482e5640 1 -- start start 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f482e5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f28004800 0x7f8f401f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f482e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 0x7f8f40157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f482e5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f400cb9d0 0x7f8f401f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f482e5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40074a80 con 0x7f8f4009aa30 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f482e5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f40074900 con 0x7f8f400cb9d0 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f482e5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8f40074c00 con 0x7f8f28004800 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f45880640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 0x7f8f40157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f45880640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 0x7f8f40157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57554/0 (socket says 172.21.15.22:57554) 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.934+0000 7f8f45880640 1 -- 172.21.15.22:0/2153825565 learned_addr learned my addr 172.21.15.22:0/2153825565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f46882640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f400cb9d0 0x7f8f401f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f46081640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f28004800 0x7f8f401f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f45880640 1 -- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f28004800 msgr2=0x7f8f401f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f45880640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f28004800 0x7f8f401f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f45880640 1 -- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f400cb9d0 msgr2=0x7f8f401f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f45880640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f400cb9d0 0x7f8f401f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f45880640 1 -- 172.21.15.22:0/2153825565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f40157b50 con 0x7f8f4009aa30 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f45880640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 0x7f8f40157610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8f34036040 tx=0x7f8f34033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34038040 con 0x7f8f4009aa30 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f34004020 con 0x7f8f4009aa30 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4015f090 con 0x7f8f4009aa30 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34004450 con 0x7f8f4009aa30 2024-09-21T05:39:48.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.935+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401f12b0 con 0x7f8f4009aa30 2024-09-21T05:39:48.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.936+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f401f1ff0 con 0x7f8f4009aa30 2024-09-21T05:39:48.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.937+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f34045020 con 0x7f8f4009aa30 2024-09-21T05:39:48.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.938+0000 7f8f3effd640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f2c078bc0 0x7f8f2c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:48.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.938+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8f340bb5d0 con 0x7f8f4009aa30 2024-09-21T05:39:48.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.938+0000 7f8f46081640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f2c078bc0 0x7f8f2c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:48.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.939+0000 7f8f46081640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f2c078bc0 0x7f8f2c07b0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f38001050 tx=0x7f8f380067a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:48.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:48.942+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f401f1ff0 con 0x7f8f4009aa30 2024-09-21T05:39:49.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.140+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8f401f10a0 con 0x7f8f4009aa30 2024-09-21T05:39:49.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.140+0000 7f8f3effd640 1 -- 172.21.15.22:0/2153825565 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f8f40157b50 con 0x7f8f4009aa30 2024-09-21T05:39:49.143 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f2c078bc0 msgr2=0x7f8f2c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f2c078bc0 0x7f8f2c07b0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8f38001050 tx=0x7f8f380067a0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 msgr2=0x7f8f40157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 0x7f8f40157610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8f34036040 tx=0x7f8f34033650 comp rx=0 tx=0).stop 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 shutdown_connections 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f400cb9d0 0x7f8f401f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f4009aa30 0x7f8f40157610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f2c078bc0 0x7f8f2c07b0d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 --2- 172.21.15.22:0/2153825565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f28004800 0x7f8f401f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 >> 172.21.15.22:0/2153825565 conn(0x7f8f400b1cb0 msgr2=0x7f8f4015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.142+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 shutdown_connections 2024-09-21T05:39:49.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.143+0000 7f8f482e5640 1 -- 172.21.15.22:0/2153825565 wait complete. 2024-09-21T05:39:49.156 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:39:49.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:49.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T05:39:49.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:39:49.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:49.183 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:39:49.248 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:49.248 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:49.267 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:49.267 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:49.327 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:49.327 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:49.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:39:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dc762640 1 -- 172.21.15.22:0/591085141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15bc003fd0 msgr2=0x7f15bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dc762640 1 --2- 172.21.15.22:0/591085141 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15bc003fd0 0x7f15bc004800 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009b20 tx=0x7f15bc005830 comp rx=0 tx=0).stop 2024-09-21T05:39:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dacff640 1 -- 172.21.15.22:0/591085141 reap_dead start 2024-09-21T05:39:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dc762640 1 -- 172.21.15.22:0/591085141 shutdown_connections 2024-09-21T05:39:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dc762640 1 -- 172.21.15.22:0/591085141 >> 172.21.15.22:0/591085141 conn(0x7f15d40d15f0 msgr2=0x7f15d4073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dc762640 1 -- 172.21.15.22:0/591085141 shutdown_connections 2024-09-21T05:39:49.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.503+0000 7f15dc762640 1 -- 172.21.15.22:0/591085141 wait complete. 2024-09-21T05:39:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.505+0000 7f15dc762640 1 Processor -- start 2024-09-21T05:39:49.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.505+0000 7f15dc762640 1 -- start start 2024-09-21T05:39:49.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dc762640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15bc003fd0 0x7f15d4157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dc762640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 0x7f15d4140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dc762640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d4141800 0x7f15d4140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dc762640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4074720 con 0x7f15d413ca60 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dc762640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15d40745a0 con 0x7f15bc003fd0 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dc762640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15d40cfe60 con 0x7f15d4141800 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 0x7f15d4140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:49.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 0x7f15d4140440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57574/0 (socket says 172.21.15.22:57574) 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 -- 172.21.15.22:0/3981323932 learned_addr learned my addr 172.21.15.22:0/3981323932 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dacff640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d4141800 0x7f15d4140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15da4fe640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15bc003fd0 0x7f15d4157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 -- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d4141800 msgr2=0x7f15d4140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d4141800 0x7f15d4140980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 -- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15bc003fd0 msgr2=0x7f15d4157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15bc003fd0 0x7f15d4157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15d9cfd640 1 -- 172.21.15.22:0/3981323932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d4157b50 con 0x7f15d413ca60 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15dacff640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d4141800 0x7f15d4140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.506+0000 7f15da4fe640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15bc003fd0 0x7f15d4157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:49.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.507+0000 7f15d9cfd640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 0x7f15d4140440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009c40 tx=0x7f15c8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.507+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8038040 con 0x7f15d413ca60 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.507+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15c8004020 con 0x7f15d413ca60 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.507+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d41f2920 con 0x7f15d413ca60 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.507+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8004450 con 0x7f15d413ca60 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.507+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41f2ee0 con 0x7f15d413ca60 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.508+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d41f2690 con 0x7f15d413ca60 2024-09-21T05:39:49.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.509+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15c8045020 con 0x7f15d413ca60 2024-09-21T05:39:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.509+0000 7f15d2ffd640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15c0078b90 0x7f15c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.509+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f15c80bb6e0 con 0x7f15d413ca60 2024-09-21T05:39:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.509+0000 7f15da4fe640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15c0078b90 0x7f15c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:49.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.510+0000 7f15da4fe640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15c0078b90 0x7f15c007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f15c40020d0 tx=0x7f15c4002060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:49.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.514+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15c8083b30 con 0x7f15d413ca60 2024-09-21T05:39:49.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.713+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f15d4073e50 con 0x7f15d413ca60 2024-09-21T05:39:49.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.714+0000 7f15d2ffd640 1 -- 172.21.15.22:0/3981323932 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f15c8083430 con 0x7f15d413ca60 2024-09-21T05:39:49.716 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:39:49.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15c0078b90 msgr2=0x7f15c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15c0078b90 0x7f15c007b0a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f15c40020d0 tx=0x7f15c4002060 comp rx=0 tx=0).stop 2024-09-21T05:39:49.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 msgr2=0x7f15d4140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:49.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 0x7f15d4140440 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009c40 tx=0x7f15c8033670 comp rx=0 tx=0).stop 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 shutdown_connections 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15c0078b90 0x7f15c007b0a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15d4141800 0x7f15d4140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15d413ca60 0x7f15d4140440 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 --2- 172.21.15.22:0/3981323932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15bc003fd0 0x7f15d4157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 >> 172.21.15.22:0/3981323932 conn(0x7f15d40d15f0 msgr2=0x7f15d4068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 shutdown_connections 2024-09-21T05:39:49.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:49.716+0000 7f15dc762640 1 -- 172.21.15.22:0/3981323932 wait complete. 2024-09-21T05:39:49.730 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T05:39:49.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:49.730 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T05:39:49.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:39:49.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:49.762 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T05:39:49.833 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:49.833 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:49.850 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:49.850 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:49.906 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:39:49.906 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:49.963 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:39:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/294038717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf80b6100 msgr2=0x7f9bf80b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/294038717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf80b6100 0x7f9bf80b1320 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bec005fd0 tx=0x7f9bec02a5d0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/294038717 shutdown_connections 2024-09-21T05:39:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/294038717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf80b6100 0x7f9bf80b1320 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/294038717 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bf80b6680 0x7f9bf80b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/294038717 >> 172.21.15.22:0/294038717 conn(0x7f9bf80d12a0 msgr2=0x7f9bf80d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:50.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/294038717 shutdown_connections 2024-09-21T05:39:50.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.118+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/294038717 wait complete. 2024-09-21T05:39:50.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.120+0000 7f9bfd3c3640 1 Processor -- start 2024-09-21T05:39:50.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.120+0000 7f9bfd3c3640 1 -- start start 2024-09-21T05:39:50.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bfd3c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bf80b6100 0x7f9bf815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bfd3c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf80b6680 0x7f9bf815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bfd3c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 0x7f9bf8164c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bfd3c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf8165a90 con 0x7f9bf8161f50 2024-09-21T05:39:50.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bfd3c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf80b20d0 con 0x7f9bf80b6680 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bfd3c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf8165c10 con 0x7f9bf80b6100 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 0x7f9bf8164c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 0x7f9bf8164c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57604/0 (socket says 172.21.15.22:57604) 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 -- 172.21.15.22:0/3594307823 learned_addr learned my addr 172.21.15.22:0/3594307823 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf6ffd640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bf80b6100 0x7f9bf815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf67fc640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf80b6680 0x7f9bf815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 -- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bf80b6100 msgr2=0x7f9bf815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bf80b6100 0x7f9bf815f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 -- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf80b6680 msgr2=0x7f9bf815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf80b6680 0x7f9bf815b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 -- 172.21.15.22:0/3594307823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf815f540 con 0x7f9bf8161f50 2024-09-21T05:39:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.122+0000 7f9bf77fe640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 0x7f9bf8164c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c760 tx=0x7f9bf000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.123+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0003b70 con 0x7f9bf8161f50 2024-09-21T05:39:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.123+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bf0003d10 con 0x7f9bf8161f50 2024-09-21T05:39:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.123+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0005330 con 0x7f9bf8161f50 2024-09-21T05:39:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.123+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf80b44b0 con 0x7f9bf8161f50 2024-09-21T05:39:50.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.123+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf80b4a70 con 0x7f9bf8161f50 2024-09-21T05:39:50.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.124+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf80b0de0 con 0x7f9bf8161f50 2024-09-21T05:39:50.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.124+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9bf00054d0 con 0x7f9bf8161f50 2024-09-21T05:39:50.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.125+0000 7f9bdf7fe640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9be4078b90 0x7f9be407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.125+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9bf009a8d0 con 0x7f9bf8161f50 2024-09-21T05:39:50.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.125+0000 7f9bf6ffd640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9be4078b90 0x7f9be407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.126+0000 7f9bf6ffd640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9be4078b90 0x7f9be407b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80ceda0 tx=0x7f9be8005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:50.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.130+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9bf0062760 con 0x7f9bf8161f50 2024-09-21T05:39:50.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.329+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9bf809f3a0 con 0x7f9bf8161f50 2024-09-21T05:39:50.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.329+0000 7f9bdf7fe640 1 -- 172.21.15.22:0/3594307823 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f9bf0067190 con 0x7f9bf8161f50 2024-09-21T05:39:50.331 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9be4078b90 msgr2=0x7f9be407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9be4078b90 0x7f9be407b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9bf80ceda0 tx=0x7f9be8005f90 comp rx=0 tx=0).stop 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 msgr2=0x7f9bf8164c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 0x7f9bf8164c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000c760 tx=0x7f9bf000cb60 comp rx=0 tx=0).stop 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 shutdown_connections 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9bf8161f50 0x7f9bf8164c50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9bf80b6680 0x7f9bf815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9be4078b90 0x7f9be407b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 --2- 172.21.15.22:0/3594307823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9bf80b6100 0x7f9bf815f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 >> 172.21.15.22:0/3594307823 conn(0x7f9bf80d12a0 msgr2=0x7f9bf80cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:50.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 shutdown_connections 2024-09-21T05:39:50.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.331+0000 7f9bfd3c3640 1 -- 172.21.15.22:0/3594307823 wait complete. 2024-09-21T05:39:50.344 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T05:39:50.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:50.345 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T05:39:50.371 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:39:50.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:50.371 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T05:39:50.428 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:50.428 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:50.447 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:50.447 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:50.506 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:39:50.506 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:39:50.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:39:50.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 -- 172.21.15.22:0/2926365451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2140d1d40 msgr2=0x7fe2140b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 --2- 172.21.15.22:0/2926365451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2140d1d40 0x7fe2140b5210 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe204009830 tx=0x7fe20402a5c0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 -- 172.21.15.22:0/2926365451 shutdown_connections 2024-09-21T05:39:50.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 --2- 172.21.15.22:0/2926365451 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2140d1d40 0x7fe2140b5210 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 -- 172.21.15.22:0/2926365451 >> 172.21.15.22:0/2926365451 conn(0x7fe2140b3f10 msgr2=0x7fe2140b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 -- 172.21.15.22:0/2926365451 shutdown_connections 2024-09-21T05:39:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.680+0000 7fe219be6640 1 -- 172.21.15.22:0/2926365451 wait complete. 2024-09-21T05:39:50.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.681+0000 7fe219be6640 1 Processor -- start 2024-09-21T05:39:50.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.682+0000 7fe219be6640 1 -- start start 2024-09-21T05:39:50.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe219be6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2140ccba0 0x7fe2140cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe219be6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2140d1d40 0x7fe214151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe219be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 0x7fe21415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe219be6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2140d1440 con 0x7fe21415c6f0 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe219be6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2140d12c0 con 0x7fe2140d1d40 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe219be6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe2140d15c0 con 0x7fe2140ccba0 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe213fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 0x7fe21415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe213fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 0x7fe21415a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57622/0 (socket says 172.21.15.22:57622) 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe213fff640 1 -- 172.21.15.22:0/4192012807 learned_addr learned my addr 172.21.15.22:0/4192012807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe212ffd640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2140d1d40 0x7fe214151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe2137fe640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2140ccba0 0x7fe2140cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.683+0000 7fe213fff640 1 -- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2140ccba0 msgr2=0x7fe2140cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe213fff640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2140ccba0 0x7fe2140cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe213fff640 1 -- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2140d1d40 msgr2=0x7fe214151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe213fff640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2140d1d40 0x7fe214151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe213fff640 1 -- 172.21.15.22:0/4192012807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2140cbd70 con 0x7fe21415c6f0 2024-09-21T05:39:50.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe213fff640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 0x7fe21415a170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c730 tx=0x7fe20c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c003b70 con 0x7fe21415c6f0 2024-09-21T05:39:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe20c003d10 con 0x7fe21415c6f0 2024-09-21T05:39:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2140cc040 con 0x7fe21415c6f0 2024-09-21T05:39:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00aab0 con 0x7fe21415c6f0 2024-09-21T05:39:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.684+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2141f3a80 con 0x7fe21415c6f0 2024-09-21T05:39:50.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.685+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2140b5210 con 0x7fe21415c6f0 2024-09-21T05:39:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.686+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe20c00ac50 con 0x7fe21415c6f0 2024-09-21T05:39:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.686+0000 7fe1fbfff640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe200078b90 0x7fe20007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.686+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe20c09aa40 con 0x7fe21415c6f0 2024-09-21T05:39:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.686+0000 7fe2137fe640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe200078b90 0x7fe20007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:50.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.687+0000 7fe2137fe640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe200078b90 0x7fe20007b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe204034000 tx=0x7fe204004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:50.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.691+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe20c0627b0 con 0x7fe21415c6f0 2024-09-21T05:39:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.890+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2140b3b20 con 0x7fe21415c6f0 2024-09-21T05:39:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.890+0000 7fe1fbfff640 1 -- 172.21.15.22:0/4192012807 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fe20c0671e0 con 0x7fe21415c6f0 2024-09-21T05:39:50.892 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe200078b90 msgr2=0x7fe20007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe200078b90 0x7fe20007b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe204034000 tx=0x7fe204004730 comp rx=0 tx=0).stop 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 msgr2=0x7fe21415a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 0x7fe21415a170 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c730 tx=0x7fe20c00cbf0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 shutdown_connections 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe21415c6f0 0x7fe21415a170 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2140d1d40 0x7fe214151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe200078b90 0x7fe20007b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 --2- 172.21.15.22:0/4192012807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2140ccba0 0x7fe2140cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.892+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 >> 172.21.15.22:0/4192012807 conn(0x7fe2140b3f10 msgr2=0x7fe21415ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.893+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 shutdown_connections 2024-09-21T05:39:50.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:50.893+0000 7fe219be6640 1 -- 172.21.15.22:0/4192012807 wait complete. 2024-09-21T05:39:50.906 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T05:39:50.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:50.907 DEBUG:teuthology.orchestra.run.smithi134:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T05:39:50.930 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:39:50.933 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:39:50.939 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:39:50.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:39:50.939 DEBUG:teuthology.orchestra.run.smithi134:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T05:39:50.998 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:39:50.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:51.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 -- 172.21.15.22:0/2811435762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 msgr2=0x7fcda00d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/2811435762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 0x7fcda00d3330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009860 tx=0x7fcd9402a4b0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 -- 172.21.15.22:0/2811435762 shutdown_connections 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/2811435762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda00d3870 0x7fcda00ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/2811435762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 0x7fcda00d3330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 -- 172.21.15.22:0/2811435762 >> 172.21.15.22:0/2811435762 conn(0x7fcda008bfc0 msgr2=0x7fcda009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 -- 172.21.15.22:0/2811435762 shutdown_connections 2024-09-21T05:39:51.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.113+0000 7fcda4cd3640 1 -- 172.21.15.22:0/2811435762 wait complete. 2024-09-21T05:39:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.115+0000 7fcda4cd3640 1 Processor -- start 2024-09-21T05:39:51.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.115+0000 7fcda4cd3640 1 -- start start 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcda4cd3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 0x7fcda00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcda4cd3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda00d3870 0x7fcda01f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcda4cd3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 0x7fcda01505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcda4cd3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcda00d0e60 con 0x7fcda00b57f0 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcda4cd3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcda00d0ce0 con 0x7fcda00b3540 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcda4cd3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcda00d0fe0 con 0x7fcda00d3870 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcd9ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 0x7fcda01505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcd9ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 0x7fcda01505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57632/0 (socket says 172.21.15.22:57632) 2024-09-21T05:39:51.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.116+0000 7fcd9ed9d640 1 -- 172.21.15.22:0/1571541810 learned_addr learned my addr 172.21.15.22:0/1571541810 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9dd9b640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda00d3870 0x7fcda01f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9e59c640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 0x7fcda00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9ed9d640 1 -- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda00d3870 msgr2=0x7fcda01f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9ed9d640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda00d3870 0x7fcda01f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9ed9d640 1 -- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 msgr2=0x7fcda00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9ed9d640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 0x7fcda00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9ed9d640 1 -- 172.21.15.22:0/1571541810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda01f5c70 con 0x7fcda00b57f0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd9ed9d640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 0x7fcda01505e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800ebf0 tx=0x7fcd9800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98003b70 con 0x7fcda00b57f0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda00ccd10 con 0x7fcda00b57f0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd98003d10 con 0x7fcda00b57f0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98004280 con 0x7fcda00b57f0 2024-09-21T05:39:51.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.117+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda00cd2d0 con 0x7fcda00b57f0 2024-09-21T05:39:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.118+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda00d1500 con 0x7fcda00b57f0 2024-09-21T05:39:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.119+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd98004420 con 0x7fcda00b57f0 2024-09-21T05:39:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.120+0000 7fcd8effd640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd80078bc0 0x7fcd8007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.120+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcd9809b920 con 0x7fcda00b57f0 2024-09-21T05:39:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.120+0000 7fcd9e59c640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd80078bc0 0x7fcd8007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.121+0000 7fcd9e59c640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd80078bc0 0x7fcd8007b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd90007fd0 tx=0x7fcd90001390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:51.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.124+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd98063690 con 0x7fcda00b57f0 2024-09-21T05:39:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.349+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcda01f5a00 con 0x7fcda00b57f0 2024-09-21T05:39:51.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.349+0000 7fcd8effd640 1 -- 172.21.15.22:0/1571541810 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fcd980680c0 con 0x7fcda00b57f0 2024-09-21T05:39:51.352 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:51.352 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:51.352 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd80078bc0 msgr2=0x7fcd8007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd80078bc0 0x7fcd8007b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd90007fd0 tx=0x7fcd90001390 comp rx=0 tx=0).stop 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 msgr2=0x7fcda01505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 0x7fcda01505e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800ebf0 tx=0x7fcd9800c6a0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 shutdown_connections 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcda00b57f0 0x7fcda01505e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd80078bc0 0x7fcd8007b0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcda00d3870 0x7fcda01f4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 --2- 172.21.15.22:0/1571541810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcda00b3540 0x7fcda00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 >> 172.21.15.22:0/1571541810 conn(0x7fcda008bfc0 msgr2=0x7fcda01619c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 shutdown_connections 2024-09-21T05:39:51.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.351+0000 7fcda4cd3640 1 -- 172.21.15.22:0/1571541810 wait complete. 2024-09-21T05:39:51.365 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:51.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 -- 172.21.15.22:0/3841879544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534074420 msgr2=0x7f453406fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 --2- 172.21.15.22:0/3841879544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534074420 0x7f453406fe30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4524009830 tx=0x7f452402a510 comp rx=0 tx=0).stop 2024-09-21T05:39:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 -- 172.21.15.22:0/3841879544 shutdown_connections 2024-09-21T05:39:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 --2- 172.21.15.22:0/3841879544 >> v2:172.21.15.134:3300/0 conn(0x7f4534155ce0 0x7f45340b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 --2- 172.21.15.22:0/3841879544 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534074420 0x7f453406fe30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 -- 172.21.15.22:0/3841879544 >> 172.21.15.22:0/3841879544 conn(0x7f453406def0 msgr2=0x7f453406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:51.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 -- 172.21.15.22:0/3841879544 shutdown_connections 2024-09-21T05:39:51.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.480+0000 7f453a82e640 1 -- 172.21.15.22:0/3841879544 wait complete. 2024-09-21T05:39:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.481+0000 7f453a82e640 1 Processor -- start 2024-09-21T05:39:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.482+0000 7f453a82e640 1 -- start start 2024-09-21T05:39:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f453a82e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4534074420 0x7f45341f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f453a82e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534155ce0 0x7f4534143360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f453a82e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 0x7f4534159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f453a82e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45340b5060 con 0x7f45341438a0 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f453a82e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45340b4ee0 con 0x7f4534074420 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f453a82e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45340b51e0 con 0x7f4534155ce0 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f4538dcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 0x7f4534159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f4538dcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 0x7f4534159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57638/0 (socket says 172.21.15.22:57638) 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f4538dcb640 1 -- 172.21.15.22:0/739910139 learned_addr learned my addr 172.21.15.22:0/739910139 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:51.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f4533fff640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4534074420 0x7f45341f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f45337fe640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534155ce0 0x7f4534143360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f4538dcb640 1 -- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534155ce0 msgr2=0x7f4534143360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.483+0000 7f4538dcb640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534155ce0 0x7f4534143360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4538dcb640 1 -- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4534074420 msgr2=0x7f45341f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4538dcb640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4534074420 0x7f45341f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4538dcb640 1 -- 172.21.15.22:0/739910139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45341f6500 con 0x7f45341438a0 2024-09-21T05:39:51.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4538dcb640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 0x7f4534159ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b670 tx=0x7f452c00bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c003970 con 0x7f45341438a0 2024-09-21T05:39:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f452c003b10 con 0x7f45341438a0 2024-09-21T05:39:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c00a7f0 con 0x7f45341438a0 2024-09-21T05:39:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45341f25f0 con 0x7f45341438a0 2024-09-21T05:39:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.484+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45341f2bb0 con 0x7f45341438a0 2024-09-21T05:39:51.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.485+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45340b5700 con 0x7f45341438a0 2024-09-21T05:39:51.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.486+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f452c01e020 con 0x7f45341438a0 2024-09-21T05:39:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.486+0000 7f4530ff9640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4520078b90 0x7f452007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.486+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f452c09cf50 con 0x7f45341438a0 2024-09-21T05:39:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.487+0000 7f4533fff640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4520078b90 0x7f452007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.487+0000 7f4533fff640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4520078b90 0x7f452007b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4524034000 tx=0x7f4524004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.491+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45341f6500 con 0x7f45341438a0 2024-09-21T05:39:51.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.713+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45340cb970 con 0x7f45341438a0 2024-09-21T05:39:51.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.714+0000 7f4530ff9640 1 -- 172.21.15.22:0/739910139 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f452c064d30 con 0x7f45341438a0 2024-09-21T05:39:51.716 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:51.716 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:51.716 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:51.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4520078b90 msgr2=0x7f452007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4520078b90 0x7f452007b0a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4524034000 tx=0x7f4524004730 comp rx=0 tx=0).stop 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 msgr2=0x7f4534159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 0x7f4534159ff0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b670 tx=0x7f452c00bb30 comp rx=0 tx=0).stop 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 shutdown_connections 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45341438a0 0x7f4534159ff0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4520078b90 0x7f452007b0a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4534155ce0 0x7f4534143360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 --2- 172.21.15.22:0/739910139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4534074420 0x7f45341f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 >> 172.21.15.22:0/739910139 conn(0x7f453406def0 msgr2=0x7f453415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 shutdown_connections 2024-09-21T05:39:51.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.716+0000 7f453a82e640 1 -- 172.21.15.22:0/739910139 wait complete. 2024-09-21T05:39:51.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/2958667135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 msgr2=0x7fdfd00cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/2958667135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00cec10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0005fd0 tx=0x7fdfc002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/2958667135 shutdown_connections 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/2958667135 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00cec10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/2958667135 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd01600b0 0x7fdfd00ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/2958667135 >> 172.21.15.22:0/2958667135 conn(0x7fdfd006d4f0 msgr2=0x7fdfd0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:51.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.843+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/2958667135 shutdown_connections 2024-09-21T05:39:51.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.844+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/2958667135 wait complete. 2024-09-21T05:39:51.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.845+0000 7fdfd7e7e640 1 Processor -- start 2024-09-21T05:39:51.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.845+0000 7fdfd7e7e640 1 -- start start 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd00cf150 0x7fdfd01f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfd01600b0 0x7fdfd01f9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd00b4460 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd00d1be0 con 0x7fdfd01600b0 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd00b45e0 con 0x7fdfd00cf150 2024-09-21T05:39:51.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57646/0 (socket says 172.21.15.22:57646) 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 -- 172.21.15.22:0/1690304956 learned_addr learned my addr 172.21.15.22:0/1690304956 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd641b640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfd01600b0 0x7fdfd01f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5419640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd00cf150 0x7fdfd01f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 -- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd00cf150 msgr2=0x7fdfd01f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd00cf150 0x7fdfd01f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 -- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfd01600b0 msgr2=0x7fdfd01f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfd01600b0 0x7fdfd01f9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 -- 172.21.15.22:0/1690304956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd00cb990 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5419640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd00cf150 0x7fdfd01f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd5c1a640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00c9790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00e790 tx=0x7fdfcc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00eea0 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfcc00cd50 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd00cbc70 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.847+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc010690 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.848+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd00cc230 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.849+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd00b4b20 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.849+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfcc010830 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.850+0000 7fdfc67fc640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfb4078b90 0x7fdfb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:51.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.850+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdfcc09b5c0 con 0x7fdfd00b7cc0 2024-09-21T05:39:51.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.850+0000 7fdfd5419640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfb4078b90 0x7fdfb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:51.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.851+0000 7fdfd5419640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfb4078b90 0x7fdfb407b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0033000 tx=0x7fdfc0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:51.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:51.854+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfcc063440 con 0x7fdfd00b7cc0 2024-09-21T05:39:52.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.041+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfd006def0 con 0x7fdfd00b7cc0 2024-09-21T05:39:52.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.043+0000 7fdfc67fc640 1 -- 172.21.15.22:0/1690304956 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7fdfcc067e70 con 0x7fdfd00b7cc0 2024-09-21T05:39:52.045 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:52.045 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfb4078b90 msgr2=0x7fdfb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfb4078b90 0x7fdfb407b0a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0033000 tx=0x7fdfc0004730 comp rx=0 tx=0).stop 2024-09-21T05:39:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 msgr2=0x7fdfd00c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00c9790 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00e790 tx=0x7fdfcc00ec50 comp rx=0 tx=0).stop 2024-09-21T05:39:52.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 shutdown_connections 2024-09-21T05:39:52.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfd01600b0 0x7fdfd01f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfb4078b90 0x7fdfb407b0a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfd00cf150 0x7fdfd01f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 --2- 172.21.15.22:0/1690304956 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfd00b7cc0 0x7fdfd00c9790 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 >> 172.21.15.22:0/1690304956 conn(0x7fdfd006d4f0 msgr2=0x7fdfd015d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 shutdown_connections 2024-09-21T05:39:52.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.045+0000 7fdfd7e7e640 1 -- 172.21.15.22:0/1690304956 wait complete. 2024-09-21T05:39:52.058 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:39:52.058 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 -- 172.21.15.22:0/594937823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 msgr2=0x7fd5440b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 --2- 172.21.15.22:0/594937823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440b4040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009830 tx=0x7fd53c02a510 comp rx=0 tx=0).stop 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 -- 172.21.15.22:0/594937823 shutdown_connections 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 --2- 172.21.15.22:0/594937823 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440b4040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 --2- 172.21.15.22:0/594937823 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 0x7fd5440d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 -- 172.21.15.22:0/594937823 >> 172.21.15.22:0/594937823 conn(0x7fd54408bfc0 msgr2=0x7fd54409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 -- 172.21.15.22:0/594937823 shutdown_connections 2024-09-21T05:39:52.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.172+0000 7fd54c09b640 1 -- 172.21.15.22:0/594937823 wait complete. 2024-09-21T05:39:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.174+0000 7fd54c09b640 1 Processor -- start 2024-09-21T05:39:52.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.174+0000 7fd54c09b640 1 -- start start 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.175+0000 7fd54c09b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 0x7fd5441f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.175+0000 7fd54c09b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5440d0ce0 0x7fd5440cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.175+0000 7fd54c09b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.175+0000 7fd54c09b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd5440b4b00 con 0x7fd5440d4a50 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.175+0000 7fd54c09b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd5440b4980 con 0x7fd5440d0ce0 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.175+0000 7fd54c09b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd5440b4c80 con 0x7fd5440b9be0 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57670/0 (socket says 172.21.15.22:57670) 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 -- 172.21.15.22:0/1360385265 learned_addr learned my addr 172.21.15.22:0/1360385265 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd549636640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5440d0ce0 0x7fd5440cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd549e37640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 0x7fd5441f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 -- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 msgr2=0x7fd5441f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 0x7fd5441f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 -- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5440d0ce0 msgr2=0x7fd5440cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5440d0ce0 0x7fd5440cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 -- 172.21.15.22:0/1360385265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5441f6270 con 0x7fd5440d4a50 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd549e37640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 0x7fd5441f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54a638640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440cd7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009950 tx=0x7fd53c033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c031d60 con 0x7fd5440d4a50 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd53c005020 con 0x7fd5440d4a50 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5440cc7d0 con 0x7fd5440d4a50 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c005450 con 0x7fd5440d4a50 2024-09-21T05:39:52.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.176+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd544151cd0 con 0x7fd5440d4a50 2024-09-21T05:39:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.177+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5441f5fe0 con 0x7fd5440d4a50 2024-09-21T05:39:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.178+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd53c0047a0 con 0x7fd5440d4a50 2024-09-21T05:39:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.179+0000 7fd53a7fc640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd528078b90 0x7fd52807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.179+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd53c0bbf20 con 0x7fd5440d4a50 2024-09-21T05:39:52.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.179+0000 7fd549e37640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd528078b90 0x7fd52807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.179+0000 7fd549e37640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd528078b90 0x7fd52807b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd540007fd0 tx=0x7fd540009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:52.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.183+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd53c03f030 con 0x7fd5440d4a50 2024-09-21T05:39:52.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.406+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5440b51a0 con 0x7fd5440d4a50 2024-09-21T05:39:52.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.406+0000 7fd53a7fc640 1 -- 172.21.15.22:0/1360385265 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fd53c083d00 con 0x7fd5440d4a50 2024-09-21T05:39:52.409 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:52.409 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:52.409 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd528078b90 msgr2=0x7fd52807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd528078b90 0x7fd52807b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd540007fd0 tx=0x7fd540009040 comp rx=0 tx=0).stop 2024-09-21T05:39:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 msgr2=0x7fd5440cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440cd7f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009950 tx=0x7fd53c033ce0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 shutdown_connections 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd5440d4a50 0x7fd5440cd7f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd5440d0ce0 0x7fd5440cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd528078b90 0x7fd52807b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 --2- 172.21.15.22:0/1360385265 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd5440b9be0 0x7fd5441f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 >> 172.21.15.22:0/1360385265 conn(0x7fd54408bfc0 msgr2=0x7fd5440b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 shutdown_connections 2024-09-21T05:39:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.409+0000 7fd54c09b640 1 -- 172.21.15.22:0/1360385265 wait complete. 2024-09-21T05:39:52.422 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:39:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 -- 172.21.15.22:0/3184910481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b806e470 msgr2=0x7f56b8070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 --2- 172.21.15.22:0/3184910481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b806e470 0x7f56b8070ee0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009830 tx=0x7f56b402a490 comp rx=0 tx=0).stop 2024-09-21T05:39:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 -- 172.21.15.22:0/3184910481 shutdown_connections 2024-09-21T05:39:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 --2- 172.21.15.22:0/3184910481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b806cde0 0x7f56b806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 --2- 172.21.15.22:0/3184910481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b806e470 0x7f56b8070ee0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 -- 172.21.15.22:0/3184910481 >> 172.21.15.22:0/3184910481 conn(0x7f56b80cd530 msgr2=0x7f56b80cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:52.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 -- 172.21.15.22:0/3184910481 shutdown_connections 2024-09-21T05:39:52.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.537+0000 7f56bfa30640 1 -- 172.21.15.22:0/3184910481 wait complete. 2024-09-21T05:39:52.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.538+0000 7f56bfa30640 1 Processor -- start 2024-09-21T05:39:52.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.539+0000 7f56bfa30640 1 -- start start 2024-09-21T05:39:52.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bfa30640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b806cde0 0x7f56b81f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bfa30640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b806e470 0x7f56b8142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bfa30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 0x7f56b8145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bfa30640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f56b80cbb70 con 0x7f56b815cd50 2024-09-21T05:39:52.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bfa30640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56b80cb9f0 con 0x7f56b806cde0 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bfa30640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56b815d870 con 0x7f56b806e470 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 0x7f56b8145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 0x7f56b8145650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57682/0 (socket says 172.21.15.22:57682) 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 -- 172.21.15.22:0/243740847 learned_addr learned my addr 172.21.15.22:0/243740847 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bd7cc640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b806cde0 0x7f56b81f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bcfcb640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b806e470 0x7f56b8142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 -- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b806e470 msgr2=0x7f56b8142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b806e470 0x7f56b8142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 -- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b806cde0 msgr2=0x7f56b81f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b806cde0 0x7f56b81f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.540+0000 7f56bdfcd640 1 -- 172.21.15.22:0/243740847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b8143330 con 0x7f56b815cd50 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.541+0000 7f56bdfcd640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 0x7f56b8145650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56b000cb10 tx=0x7f56b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.541+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0016020 con 0x7f56b815cd50 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.541+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56b000ad60 con 0x7f56b815cd50 2024-09-21T05:39:52.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.541+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0012680 con 0x7f56b815cd50 2024-09-21T05:39:52.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.541+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b8146550 con 0x7f56b815cd50 2024-09-21T05:39:52.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.541+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b81f6e20 con 0x7f56b815cd50 2024-09-21T05:39:52.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.542+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b806d1f0 con 0x7f56b815cd50 2024-09-21T05:39:52.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.543+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f56b0019020 con 0x7f56b815cd50 2024-09-21T05:39:52.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.543+0000 7f56adffb640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f569c078bc0 0x7f569c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.543+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f56b009b060 con 0x7f56b815cd50 2024-09-21T05:39:52.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.543+0000 7f56bd7cc640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f569c078bc0 0x7f569c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.544+0000 7f56bd7cc640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f569c078bc0 0x7f569c07b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034000 tx=0x7f56b4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:52.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.548+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56b0020030 con 0x7f56b815cd50 2024-09-21T05:39:52.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.771+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56b8146000 con 0x7f56b815cd50 2024-09-21T05:39:52.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.771+0000 7f56adffb640 1 -- 172.21.15.22:0/243740847 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f56b00634a0 con 0x7f56b815cd50 2024-09-21T05:39:52.774 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:52.774 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":8,"btime":"2024-09-21T05:39:25:833071+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4307,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3812791221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3812791221},{"type":"v1","addr":"172.21.15.134:6837","nonce":3812791221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2683532310","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2683532310},{"type":"v1","addr":"172.21.15.134:6836","nonce":2683532310}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4392,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3088275486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3088275486},{"type":"v1","addr":"172.21.15.22:6834","nonce":3088275486}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:39:22.195858+0000","modified":"2024-09-21T05:39:25.213840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4395":{"gid":4395,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/1656544793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1656544793},{"type":"v1","addr":"172.21.15.22:6835","nonce":1656544793}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4395,"qdb_cluster":[4395]},"id":1}]} 2024-09-21T05:39:52.774 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 8 2024-09-21T05:39:52.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f569c078bc0 msgr2=0x7f569c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f569c078bc0 0x7f569c07b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f56b4034000 tx=0x7f56b4004730 comp rx=0 tx=0).stop 2024-09-21T05:39:52.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 msgr2=0x7f56b8145650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 0x7f56b8145650 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56b000cb10 tx=0x7f56b000a060 comp rx=0 tx=0).stop 2024-09-21T05:39:52.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 shutdown_connections 2024-09-21T05:39:52.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56b815cd50 0x7f56b8145650 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f569c078bc0 0x7f569c07b0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f56b806e470 0x7f56b8142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 --2- 172.21.15.22:0/243740847 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56b806cde0 0x7f56b81f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 >> 172.21.15.22:0/243740847 conn(0x7f56b80cd530 msgr2=0x7f56b815d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:52.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 shutdown_connections 2024-09-21T05:39:52.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.774+0000 7f56bfa30640 1 -- 172.21.15.22:0/243740847 wait complete. 2024-09-21T05:39:52.788 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 -- 172.21.15.22:0/3563066025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a04004800 msgr2=0x7f8a04004c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/3563066025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a04004800 0x7f8a04004c30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8a08006f70 tx=0x7f8a04005830 comp rx=0 tx=0).stop 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 -- 172.21.15.22:0/3563066025 shutdown_connections 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/3563066025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a04004800 0x7f8a04004c30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 -- 172.21.15.22:0/3563066025 >> 172.21.15.22:0/3563066025 conn(0x7f8a1806ea80 msgr2=0x7f8a180b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 -- 172.21.15.22:0/3563066025 shutdown_connections 2024-09-21T05:39:52.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.903+0000 7f8a1f32b640 1 -- 172.21.15.22:0/3563066025 wait complete. 2024-09-21T05:39:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.905+0000 7f8a1f32b640 1 Processor -- start 2024-09-21T05:39:52.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.905+0000 7f8a1f32b640 1 -- start start 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.906+0000 7f8a1f32b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a04004800 0x7f8a1809bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.906+0000 7f8a1f32b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 0x7f8a1806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.906+0000 7f8a1f32b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a181481e0 0x7f8a1806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1f32b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a180b22b0 con 0x7f8a18147dd0 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1f32b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a180b2130 con 0x7f8a04004800 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1f32b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8a180aeb00 con 0x7f8a181481e0 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 0x7f8a1806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 0x7f8a1806cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57700/0 (socket says 172.21.15.22:57700) 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 -- 172.21.15.22:0/2767396433 learned_addr learned my addr 172.21.15.22:0/2767396433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1d0c7640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a04004800 0x7f8a1809bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1d8c8640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a181481e0 0x7f8a1806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 -- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a181481e0 msgr2=0x7f8a1806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a181481e0 0x7f8a1806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 -- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a04004800 msgr2=0x7f8a1809bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a04004800 0x7f8a1809bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 -- 172.21.15.22:0/2767396433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1809c080 con 0x7f8a18147dd0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1c8c6640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 0x7f8a1806cde0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a08031040 tx=0x7f8a08001c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08032030 con 0x7f8a18147dd0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a080035f0 con 0x7f8a18147dd0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a181528e0 con 0x7f8a18147dd0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.907+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08003a00 con 0x7f8a18147dd0 2024-09-21T05:39:52.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.908+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a18152ea0 con 0x7f8a18147dd0 2024-09-21T05:39:52.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.909+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1809c250 con 0x7f8a18147dd0 2024-09-21T05:39:52.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.909+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a08035020 con 0x7f8a18147dd0 2024-09-21T05:39:52.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.910+0000 7f8a0dffb640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f89fc078b90 0x7f89fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:52.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.910+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8a080bad30 con 0x7f8a18147dd0 2024-09-21T05:39:52.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.910+0000 7f8a1d0c7640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f89fc078b90 0x7f89fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:52.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.911+0000 7f8a1d0c7640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f89fc078b90 0x7f89fc07b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a14006540 tx=0x7f8a14006b10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:52.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:52.915+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a08047030 con 0x7f8a18147dd0 2024-09-21T05:39:53.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.103+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a180af060 con 0x7f8a18147dd0 2024-09-21T05:39:53.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.104+0000 7f8a0dffb640 1 -- 172.21.15.22:0/2767396433 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16653 (secure 0 0 0) 0x7f8a08083170 con 0x7f8a18147dd0 2024-09-21T05:39:53.106 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:53.106 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:39:23.821370+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:39:20.094882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T05:39:21.155594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:39:53.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f89fc078b90 msgr2=0x7f89fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f89fc078b90 0x7f89fc07b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a14006540 tx=0x7f8a14006b10 comp rx=0 tx=0).stop 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 msgr2=0x7f8a1806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 0x7f8a1806cde0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8a08031040 tx=0x7f8a08001c30 comp rx=0 tx=0).stop 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 shutdown_connections 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f89fc078b90 0x7f89fc07b0a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a181481e0 0x7f8a1806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a18147dd0 0x7f8a1806cde0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 --2- 172.21.15.22:0/2767396433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a04004800 0x7f8a1809bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 >> 172.21.15.22:0/2767396433 conn(0x7f8a1806ea80 msgr2=0x7f8a180b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 shutdown_connections 2024-09-21T05:39:53.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.106+0000 7f8a1f32b640 1 -- 172.21.15.22:0/2767396433 wait complete. 2024-09-21T05:39:53.120 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:39:53.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1660638718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 msgr2=0x7f40c00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1660638718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00ccbc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40c00ce1b0 tx=0x7f40b002a530 comp rx=0 tx=0).stop 2024-09-21T05:39:53.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1660638718 shutdown_connections 2024-09-21T05:39:53.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1660638718 >> v2:172.21.15.134:3300/0 conn(0x7f40c00b7cc0 0x7f40c00cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1660638718 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00ccbc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1660638718 >> 172.21.15.22:0/1660638718 conn(0x7f40c006d4f0 msgr2=0x7f40c0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:53.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1660638718 shutdown_connections 2024-09-21T05:39:53.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.235+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1660638718 wait complete. 2024-09-21T05:39:53.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.236+0000 7f40c5e0e640 1 Processor -- start 2024-09-21T05:39:53.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.237+0000 7f40c5e0e640 1 -- start start 2024-09-21T05:39:53.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40c5e0e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c00b7cc0 0x7f40c00cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40c5e0e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c00cf150 0x7f40c00b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40c5e0e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40c5e0e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f40c00b4460 con 0x7f40c01600b0 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40c5e0e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f40c00d1ba0 con 0x7f40c00cf150 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40c5e0e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f40c00b45e0 con 0x7f40c00b7cc0 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57714/0 (socket says 172.21.15.22:57714) 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 -- 172.21.15.22:0/1782226628 learned_addr learned my addr 172.21.15.22:0/1782226628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40beffd640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c00cf150 0x7f40c00b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bf7fe640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c00b7cc0 0x7f40c00cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 -- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c00b7cc0 msgr2=0x7f40c00cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c00b7cc0 0x7f40c00cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 -- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c00cf150 msgr2=0x7f40c00b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.238+0000 7f40bffff640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c00cf150 0x7f40c00b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40bffff640 1 -- 172.21.15.22:0/1782226628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c01f60a0 con 0x7f40c01600b0 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40bffff640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00b7220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c730 tx=0x7f40b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8017070 con 0x7f40c01600b0 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40b8003c60 con 0x7f40c01600b0 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b80052d0 con 0x7f40c01600b0 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c01f6380 con 0x7f40c01600b0 2024-09-21T05:39:53.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.239+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c01f6940 con 0x7f40c01600b0 2024-09-21T05:39:53.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.240+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40c00cec10 con 0x7f40c01600b0 2024-09-21T05:39:53.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.241+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f40b801a070 con 0x7f40c01600b0 2024-09-21T05:39:53.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.241+0000 7f40a7fff640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f40ac078bc0 0x7f40ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.241+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f40b809a570 con 0x7f40c01600b0 2024-09-21T05:39:53.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.242+0000 7f40bf7fe640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f40ac078bc0 0x7f40ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:53.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.242+0000 7f40bf7fe640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f40ac078bc0 0x7f40ac07b0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40b0034000 tx=0x7f40b0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:53.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.246+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40b80629b0 con 0x7f40c01600b0 2024-09-21T05:39:53.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.473+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f40c00ccee0 con 0x7f40c01600b0 2024-09-21T05:39:53.476 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:39:53.474+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:39:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.475+0000 7f40a7fff640 1 -- 172.21.15.22:0/1782226628 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f40b80622b0 con 0x7f40c01600b0 2024-09-21T05:39:53.477 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f40ac078bc0 msgr2=0x7f40ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f40ac078bc0 0x7f40ac07b0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40b0034000 tx=0x7f40b0004730 comp rx=0 tx=0).stop 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 msgr2=0x7f40c00b7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00b7220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c730 tx=0x7f40b800cbf0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 shutdown_connections 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f40c01600b0 0x7f40c00b7220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f40c00cf150 0x7f40c00b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f40ac078bc0 0x7f40ac07b0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 --2- 172.21.15.22:0/1782226628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f40c00b7cc0 0x7f40c00cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 >> 172.21.15.22:0/1782226628 conn(0x7f40c006d4f0 msgr2=0x7f40c00b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 shutdown_connections 2024-09-21T05:39:53.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.479+0000 7f40c5e0e640 1 -- 172.21.15.22:0/1782226628 wait complete. 2024-09-21T05:39:53.484 INFO:tasks.ceph.mds.a.smithi022.stderr: -6850> 2024-09-21T05:39:28.787+0000 7f46c25e3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:39:53.493 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:39:53.543 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:39:53.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6209d8b640 1 -- 172.21.15.22:0/888269897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61ec004800 msgr2=0x7f61ec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6209d8b640 1 --2- 172.21.15.22:0/888269897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f61ec004800 0x7f61ec004c10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009b20 tx=0x7f61ec005830 comp rx=0 tx=0).stop 2024-09-21T05:39:53.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6203fff640 1 -- 172.21.15.22:0/888269897 reap_dead start 2024-09-21T05:39:53.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6209d8b640 1 -- 172.21.15.22:0/888269897 shutdown_connections 2024-09-21T05:39:53.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6209d8b640 1 -- 172.21.15.22:0/888269897 >> 172.21.15.22:0/888269897 conn(0x7f6204072ae0 msgr2=0x7f6204072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:53.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6209d8b640 1 -- 172.21.15.22:0/888269897 shutdown_connections 2024-09-21T05:39:53.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.613+0000 7f6209d8b640 1 -- 172.21.15.22:0/888269897 wait complete. 2024-09-21T05:39:53.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.615+0000 7f6209d8b640 1 Processor -- start 2024-09-21T05:39:53.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.615+0000 7f6209d8b640 1 -- start start 2024-09-21T05:39:53.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6209d8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61ec004800 0x7f62041f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6209d8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 0x7f62041f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6209d8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040ab990 0x7f62040abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6209d8b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62040d16d0 con 0x7f62041f6af0 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6209d8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f620415ca00 con 0x7f62040ab990 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6209d8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f62040d1850 con 0x7f61ec004800 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6202ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 0x7f62041f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6202ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 0x7f62041f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:57752/0 (socket says 172.21.15.22:57752) 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.616+0000 7f6202ffd640 1 -- 172.21.15.22:0/2593351364 learned_addr learned my addr 172.21.15.22:0/2593351364 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6202ffd640 1 -- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61ec004800 msgr2=0x7f62041f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6202ffd640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61ec004800 0x7f62041f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6202ffd640 1 -- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040ab990 msgr2=0x7f62040abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6202ffd640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040ab990 0x7f62040abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6202ffd640 1 -- 172.21.15.22:0/2593351364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62041f7440 con 0x7f62041f6af0 2024-09-21T05:39:53.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6202ffd640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 0x7f62041f6f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009a70 tx=0x7f61f8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8038040 con 0x7f62041f6af0 2024-09-21T05:39:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61f80337e0 con 0x7f62041f6af0 2024-09-21T05:39:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8033c30 con 0x7f62041f6af0 2024-09-21T05:39:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62040ac2e0 con 0x7f62041f6af0 2024-09-21T05:39:53.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.617+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62040aca20 con 0x7f62041f6af0 2024-09-21T05:39:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.618+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62040ac4b0 con 0x7f62041f6af0 2024-09-21T05:39:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.619+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61f8043020 con 0x7f62041f6af0 2024-09-21T05:39:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.619+0000 7f61ebfff640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078bc0 0x7f61f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.620+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f61f80bb190 con 0x7f62041f6af0 2024-09-21T05:39:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.620+0000 7f62037fe640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078bc0 0x7f61f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:53.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.620+0000 7f62037fe640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078bc0 0x7f61f007b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005fd0 tx=0x7f61f400ddc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:53.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.624+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62041f7440 con 0x7f62041f6af0 2024-09-21T05:39:53.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:53.851+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f62040d0d80 con 0x7f62041f6af0 2024-09-21T05:39:54.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.480+0000 7f61ebfff640 1 -- 172.21.15.22:0/2593351364 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f61f80835f0 con 0x7f62041f6af0 2024-09-21T05:39:54.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078bc0 msgr2=0x7f61f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078bc0 0x7f61f007b0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005fd0 tx=0x7f61f400ddc0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 msgr2=0x7f62041f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 0x7f62041f6f00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009a70 tx=0x7f61f8033670 comp rx=0 tx=0).stop 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 shutdown_connections 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040ab990 0x7f62040abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62041f6af0 0x7f62041f6f00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078bc0 0x7f61f007b0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 --2- 172.21.15.22:0/2593351364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f61ec004800 0x7f62041f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 >> 172.21.15.22:0/2593351364 conn(0x7f6204072ae0 msgr2=0x7f620415ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 shutdown_connections 2024-09-21T05:39:54.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.483+0000 7f6209d8b640 1 -- 172.21.15.22:0/2593351364 wait complete. 2024-09-21T05:39:54.500 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:39:54.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 -- 172.21.15.22:0/1863392136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde8003a50 msgr2=0x7fcde8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:54.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 --2- 172.21.15.22:0/1863392136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde8003a50 0x7fcde8003ec0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcdec006fc0 tx=0x7fcde8005830 comp rx=0 tx=0).stop 2024-09-21T05:39:54.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 -- 172.21.15.22:0/1863392136 shutdown_connections 2024-09-21T05:39:54.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 --2- 172.21.15.22:0/1863392136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcde8003a50 0x7fcde8003ec0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 -- 172.21.15.22:0/1863392136 >> 172.21.15.22:0/1863392136 conn(0x7fcdfc0cee20 msgr2=0x7fcdfc0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:54.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 -- 172.21.15.22:0/1863392136 shutdown_connections 2024-09-21T05:39:54.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.626+0000 7fce01830640 1 -- 172.21.15.22:0/1863392136 wait complete. 2024-09-21T05:39:54.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.628+0000 7fce01830640 1 Processor -- start 2024-09-21T05:39:54.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.628+0000 7fce01830640 1 -- start start 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.629+0000 7fce01830640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde8003a50 0x7fcdfc1de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.629+0000 7fce01830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 0x7fcdfc1f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.629+0000 7fce01830640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdfc1cf720 0x7fcdfc1f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.629+0000 7fce01830640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc0cd630 con 0x7fcdfc1cf310 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.629+0000 7fce01830640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc0cd4b0 con 0x7fcde8003a50 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.629+0000 7fce01830640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcdfc0cd7b0 con 0x7fcdfc1cf720 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 0x7fcdfc1f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 0x7fcdfc1f6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36762/0 (socket says 172.21.15.22:36762) 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 -- 172.21.15.22:0/4265573902 learned_addr learned my addr 172.21.15.22:0/4265573902 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdfaffd640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde8003a50 0x7fcdfc1de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdfb7fe640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdfc1cf720 0x7fcdfc1f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:54.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 -- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdfc1cf720 msgr2=0x7fcdfc1f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdfc1cf720 0x7fcdfc1f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 -- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde8003a50 msgr2=0x7fcdfc1de0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde8003a50 0x7fcdfc1de0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 -- 172.21.15.22:0/4265573902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc1dea40 con 0x7fcdfc1cf310 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf27fc640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 0x7fcdfc1f6ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcde4011780 tx=0x7fcde4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde400fd50 con 0x7fcdfc1cf310 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde4011e90 con 0x7fcdfc1cf310 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc1ded00 con 0x7fcdfc1cf310 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004290 con 0x7fcdfc1cf310 2024-09-21T05:39:54.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.630+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc0c7150 con 0x7fcdfc1cf310 2024-09-21T05:39:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.632+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdfc06a750 con 0x7fcdfc1cf310 2024-09-21T05:39:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.632+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcde40035f0 con 0x7fcdfc1cf310 2024-09-21T05:39:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.633+0000 7fcdf8ff9640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcddc078b90 0x7fcddc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.633+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7fcde409b9b0 con 0x7fcdfc1cf310 2024-09-21T05:39:54.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.633+0000 7fcdfaffd640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcddc078b90 0x7fcddc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:54.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.633+0000 7fcdfaffd640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcddc078b90 0x7fcddc07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdec033000 tx=0x7fcdfc09bcb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:54.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.637+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcde40636a0 con 0x7fcdfc1cf310 2024-09-21T05:39:54.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:54.825+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcdfc0cdca0 con 0x7fcdfc1cf310 2024-09-21T05:39:55.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.482+0000 7fcdf8ff9640 1 -- 172.21.15.22:0/4265573902 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7fcde40680d0 con 0x7fcdfc1cf310 2024-09-21T05:39:55.484 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:39:55.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcddc078b90 msgr2=0x7fcddc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:55.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcddc078b90 0x7fcddc07b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdec033000 tx=0x7fcdfc09bcb0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 msgr2=0x7fcdfc1f6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:55.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 0x7fcdfc1f6ef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcde4011780 tx=0x7fcde4011c40 comp rx=0 tx=0).stop 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 shutdown_connections 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcddc078b90 0x7fcddc07b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdfc1cf720 0x7fcdfc1f7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdfc1cf310 0x7fcdfc1f6ef0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 --2- 172.21.15.22:0/4265573902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcde8003a50 0x7fcdfc1de0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 >> 172.21.15.22:0/4265573902 conn(0x7fcdfc0cee20 msgr2=0x7fcdfc1e5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 shutdown_connections 2024-09-21T05:39:55.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.486+0000 7fce01830640 1 -- 172.21.15.22:0/4265573902 wait complete. 2024-09-21T05:39:55.503 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:39:55.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 -- 172.21.15.22:0/2555784929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a00d1ce0 msgr2=0x7fe4a0160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:55.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 --2- 172.21.15.22:0/2555784929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a00d1ce0 0x7fe4a0160a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe488005fd0 tx=0x7fe48802a5d0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 -- 172.21.15.22:0/2555784929 shutdown_connections 2024-09-21T05:39:55.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 --2- 172.21.15.22:0/2555784929 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a0071090 0x7fe4a00714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 --2- 172.21.15.22:0/2555784929 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a00d1ce0 0x7fe4a0160a00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 -- 172.21.15.22:0/2555784929 >> 172.21.15.22:0/2555784929 conn(0x7fe4a00ced50 msgr2=0x7fe4a00cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:55.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 -- 172.21.15.22:0/2555784929 shutdown_connections 2024-09-21T05:39:55.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.629+0000 7fe4a5893640 1 -- 172.21.15.22:0/2555784929 wait complete. 2024-09-21T05:39:55.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.631+0000 7fe4a5893640 1 Processor -- start 2024-09-21T05:39:55.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.631+0000 7fe4a5893640 1 -- start start 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a0071090 0x7fe4a0149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a00d1ce0 0x7fe4a00b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 0x7fe4a00b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00d01a0 con 0x7fe4a015a310 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00d0020 con 0x7fe4a00d1ce0 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00d0320 con 0x7fe4a0071090 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 0x7fe4a00b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 0x7fe4a00b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36772/0 (socket says 172.21.15.22:36772) 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 -- 172.21.15.22:0/867959836 learned_addr learned my addr 172.21.15.22:0/867959836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:55.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49effd640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a0071090 0x7fe4a0149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49e7fc640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a00d1ce0 0x7fe4a00b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 -- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a0071090 msgr2=0x7fe4a0149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a0071090 0x7fe4a0149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 -- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a00d1ce0 msgr2=0x7fe4a00b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a00d1ce0 0x7fe4a00b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 -- 172.21.15.22:0/867959836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a0159970 con 0x7fe4a015a310 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe49f7fe640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 0x7fe4a00b3980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe49800d660 tx=0x7fe49800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498017070 con 0x7fe4a015a310 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a00b23a0 con 0x7fe4a015a310 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.633+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe498005310 con 0x7fe4a015a310 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.634+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498009820 con 0x7fe4a015a310 2024-09-21T05:39:55.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.634+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a00b2960 con 0x7fe4a015a310 2024-09-21T05:39:55.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.635+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a00b2110 con 0x7fe4a015a310 2024-09-21T05:39:55.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.635+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4980099c0 con 0x7fe4a015a310 2024-09-21T05:39:55.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.636+0000 7fe497fff640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe480078bc0 0x7fe48007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:55.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.636+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7fe49809ac60 con 0x7fe4a015a310 2024-09-21T05:39:55.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.636+0000 7fe49effd640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe480078bc0 0x7fe48007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:55.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.637+0000 7fe49effd640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe480078bc0 0x7fe48007b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00ce780 tx=0x7fe490009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:55.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.641+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe498062b30 con 0x7fe4a015a310 2024-09-21T05:39:55.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:55.828+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe4a00c8dc0 con 0x7fe4a015a310 2024-09-21T05:39:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.484+0000 7fe497fff640 1 -- 172.21.15.22:0/867959836 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7fe498067560 con 0x7fe4a015a310 2024-09-21T05:39:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:39:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.490+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe480078bc0 msgr2=0x7fe48007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.490+0000 7fe4a5893640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe480078bc0 0x7fe48007b0d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00ce780 tx=0x7fe490009040 comp rx=0 tx=0).stop 2024-09-21T05:39:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.490+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 msgr2=0x7fe4a00b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:56.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.490+0000 7fe4a5893640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 0x7fe4a00b3980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe49800d660 tx=0x7fe49800db20 comp rx=0 tx=0).stop 2024-09-21T05:39:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 shutdown_connections 2024-09-21T05:39:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a015a310 0x7fe4a00b3980 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a00d1ce0 0x7fe4a00b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe480078bc0 0x7fe48007b0d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 --2- 172.21.15.22:0/867959836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a0071090 0x7fe4a0149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 >> 172.21.15.22:0/867959836 conn(0x7fe4a00ced50 msgr2=0x7fe4a015cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.491+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 shutdown_connections 2024-09-21T05:39:56.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.492+0000 7fe4a5893640 1 -- 172.21.15.22:0/867959836 wait complete. 2024-09-21T05:39:56.510 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:39:56.510 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:39:56.511 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:39:56.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:39:56.511 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:39:56.511 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:39:56.511 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:39:56.511 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:39:56.511 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:39:56.511 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:39:56.512 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:39:56.512 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:39:56.512 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:39:56.512 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:39:56.512 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:39:56.512 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:39:56.523 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:39:56.523 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:39:56.525 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:39:56.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:39:56.526 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:39:56.527 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:39:56.527 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:39:56.527 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:39:56.567 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:39:56.568 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:56.569 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:39:56.569 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:56.570 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:39:56.570 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:56.571 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:39:56.571 DEBUG:teuthology.parallel:result is None 2024-09-21T05:39:56.571 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:39:56.663 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:39:56.663 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:39:56.663 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:39:56.664 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.698+0000 7f528e6a4640 1 -- 172.21.15.22:0/4089045269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 msgr2=0x7f52880ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.698+0000 7f528e6a4640 1 --2- 172.21.15.22:0/4089045269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ceb40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5280009be0 tx=0x7f528002a840 comp rx=0 tx=0).stop 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.698+0000 7f528e6a4640 1 -- 172.21.15.22:0/4089045269 shutdown_connections 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.698+0000 7f528e6a4640 1 --2- 172.21.15.22:0/4089045269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ceb40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.698+0000 7f528e6a4640 1 --2- 172.21.15.22:0/4089045269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 0x7f52880d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.699+0000 7f528e6a4640 1 -- 172.21.15.22:0/4089045269 >> 172.21.15.22:0/4089045269 conn(0x7f528808bfc0 msgr2=0x7f528809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:56.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.699+0000 7f528e6a4640 1 -- 172.21.15.22:0/4089045269 shutdown_connections 2024-09-21T05:39:56.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.699+0000 7f528e6a4640 1 -- 172.21.15.22:0/4089045269 wait complete. 2024-09-21T05:39:56.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.700+0000 7f528e6a4640 1 Processor -- start 2024-09-21T05:39:56.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.701+0000 7f528e6a4640 1 -- start start 2024-09-21T05:39:56.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528e6a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 0x7f528815de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:56.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528e6a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:56.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528e6a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 0x7f52880ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:56.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528e6a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52880d0ce0 con 0x7f528815e6e0 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528e6a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52880cf4e0 con 0x7f52880d3870 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528e6a4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f52880d0e60 con 0x7f52880b3540 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 0x7f52880ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 0x7f52880ca5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36812/0 (socket says 172.21.15.22:36812) 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 -- 172.21.15.22:0/215890307 learned_addr learned my addr 172.21.15.22:0/215890307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f52877fe640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f5287fff640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 0x7f528815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 -- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 msgr2=0x7f528815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 0x7f528815de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 -- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 msgr2=0x7f52880ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f528cc41640 1 -- 172.21.15.22:0/215890307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52881f5560 con 0x7f528815e6e0 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f52877fe640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.702+0000 7f5287fff640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 0x7f528815de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:39:56.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.703+0000 7f528cc41640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 0x7f52880ca5d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5280005fd0 tx=0x7f5280033f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.703+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280042020 con 0x7f528815e6e0 2024-09-21T05:39:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.703+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5280004040 con 0x7f528815e6e0 2024-09-21T05:39:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.703+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280037040 con 0x7f528815e6e0 2024-09-21T05:39:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.703+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528815eef0 con 0x7f528815e6e0 2024-09-21T05:39:56.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.703+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52880cc950 con 0x7f528815e6e0 2024-09-21T05:39:56.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.704+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528806e170 con 0x7f528815e6e0 2024-09-21T05:39:56.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.705+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5280045020 con 0x7f528815e6e0 2024-09-21T05:39:56.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.705+0000 7f5284ff9640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5274078b90 0x7f527407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:56.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.705+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f52800bba80 con 0x7f528815e6e0 2024-09-21T05:39:56.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.706+0000 7f5287fff640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5274078b90 0x7f527407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:56.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.706+0000 7f5287fff640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5274078b90 0x7f527407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52780046b0 tx=0x7f5278009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.710+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f528004c030 con 0x7f528815e6e0 2024-09-21T05:39:56.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:56.908+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f52880cafe0 con 0x7f528815e6e0 2024-09-21T05:39:57.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.503+0000 7f5284ff9640 1 -- 172.21.15.22:0/215890307 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f528002f070 con 0x7f528815e6e0 2024-09-21T05:39:57.505 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.506+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5274078b90 msgr2=0x7f527407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.506+0000 7f528e6a4640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5274078b90 0x7f527407b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52780046b0 tx=0x7f5278009040 comp rx=0 tx=0).stop 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.506+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 msgr2=0x7f52880ca5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.506+0000 7f528e6a4640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 0x7f52880ca5d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5280005fd0 tx=0x7f5280033f00 comp rx=0 tx=0).stop 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.506+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 shutdown_connections 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.506+0000 7f528e6a4640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f528815e6e0 0x7f52880ca5d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.507+0000 7f528e6a4640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52880d3870 0x7f52880ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.507+0000 7f528e6a4640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5274078b90 0x7f527407b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.507+0000 7f528e6a4640 1 --2- 172.21.15.22:0/215890307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52880b3540 0x7f528815de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.507+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 >> 172.21.15.22:0/215890307 conn(0x7f528808bfc0 msgr2=0x7f52880b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.507+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 shutdown_connections 2024-09-21T05:39:57.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.507+0000 7f528e6a4640 1 -- 172.21.15.22:0/215890307 wait complete. 2024-09-21T05:39:57.524 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:39:57.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f1581c4d640 1 -- 172.21.15.22:0/3487657693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1564003fd0 msgr2=0x7f1564004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f1581c4d640 1 --2- 172.21.15.22:0/3487657693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1564003fd0 0x7f1564004800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1570009b50 tx=0x7f1564005830 comp rx=0 tx=0).stop 2024-09-21T05:39:57.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f157bfff640 1 -- 172.21.15.22:0/3487657693 reap_dead start 2024-09-21T05:39:57.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f1581c4d640 1 -- 172.21.15.22:0/3487657693 shutdown_connections 2024-09-21T05:39:57.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f1581c4d640 1 -- 172.21.15.22:0/3487657693 >> 172.21.15.22:0/3487657693 conn(0x7f157c06d7d0 msgr2=0x7f157c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:57.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f1581c4d640 1 -- 172.21.15.22:0/3487657693 shutdown_connections 2024-09-21T05:39:57.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.641+0000 7f1581c4d640 1 -- 172.21.15.22:0/3487657693 wait complete. 2024-09-21T05:39:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.643+0000 7f1581c4d640 1 Processor -- start 2024-09-21T05:39:57.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.643+0000 7f1581c4d640 1 -- start start 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.644+0000 7f1581c4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1564003fd0 0x7f157c09cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.644+0000 7f1581c4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f157c154570 0x7f157c15b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.644+0000 7f1581c4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 0x7f157c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.644+0000 7f1581c4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f157c0b39c0 con 0x7f157c154980 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.644+0000 7f1581c4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f157c073bc0 con 0x7f157c154570 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.644+0000 7f1581c4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f157c0b3b40 con 0x7f1564003fd0 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 0x7f157c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 0x7f157c15b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36834/0 (socket says 172.21.15.22:36834) 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 -- 172.21.15.22:0/1319915601 learned_addr learned my addr 172.21.15.22:0/1319915601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:57.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157affd640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f157c154570 0x7f157c15b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157b7fe640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1564003fd0 0x7f157c09cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 -- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1564003fd0 msgr2=0x7f157c09cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1564003fd0 0x7f157c09cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 -- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f157c154570 msgr2=0x7f157c15b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f157c154570 0x7f157c15b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 -- 172.21.15.22:0/1319915601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c09d6a0 con 0x7f157c154980 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f157bfff640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 0x7f157c15b8a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f157c0ce0e0 tx=0x7f1574006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157400fa80 con 0x7f157c154980 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f157400fc40 con 0x7f157c154980 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c1f6c10 con 0x7f157c154980 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574011430 con 0x7f157c154980 2024-09-21T05:39:57.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.645+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1f71d0 con 0x7f157c154980 2024-09-21T05:39:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.646+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157c070180 con 0x7f157c154980 2024-09-21T05:39:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.647+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15740115d0 con 0x7f157c154980 2024-09-21T05:39:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.648+0000 7f1563fff640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1568078b90 0x7f156807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.648+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f157409b1c0 con 0x7f157c154980 2024-09-21T05:39:57.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.648+0000 7f157b7fe640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1568078b90 0x7f156807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:57.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.649+0000 7f157b7fe640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1568078b90 0x7f156807b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f156c009bb0 tx=0x7f156c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:57.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.653+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1574063950 con 0x7f157c154980 2024-09-21T05:39:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.915+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f157c0b13a0 con 0x7f157c154980 2024-09-21T05:39:57.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.915+0000 7f1563fff640 1 -- 172.21.15.22:0/1319915601 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f1574063250 con 0x7f157c154980 2024-09-21T05:39:57.917 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:39:57.917 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:39:57.918 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.919 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.920 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.921 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.922 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:39:57.923 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.924 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:39:57.925 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:39:57.926 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.927 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.928 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mds", 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDxW+5m/hmOFRAAC9IDLWbjTp3bfPYX31uShg==", 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T05:39:57.929 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDxW+5m+yKOFRAARU+U+C+6x/RP646ynBcD/w==", 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-osd", 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDxW+5m7SqOFRAAyLCaebj5jpmj5+6N5Phhjg==", 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T05:39:57.930 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDxW+5mSTKOFRAAaVxQrdYsKMS1efxoeDKMLA==", 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDxW+5mdTqOFRAA9gOZKlGgdx7QeIVXrO3PDA==", 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T05:39:57.931 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDxW+5mqEKOFRAAukj+pSQPINQI7o6mLt29JQ==", 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:39:57.932 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:39:57.933 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:39:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1568078b90 msgr2=0x7f156807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1568078b90 0x7f156807b0a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f156c009bb0 tx=0x7f156c005f90 comp rx=0 tx=0).stop 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 msgr2=0x7f157c15b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 0x7f157c15b8a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f157c0ce0e0 tx=0x7f1574006660 comp rx=0 tx=0).stop 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 shutdown_connections 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f157c154980 0x7f157c15b8a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f157c154570 0x7f157c15b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1568078b90 0x7f156807b0a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 --2- 172.21.15.22:0/1319915601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1564003fd0 0x7f157c09cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.917+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 >> 172.21.15.22:0/1319915601 conn(0x7f157c06d7d0 msgr2=0x7f157c0aea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.918+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 shutdown_connections 2024-09-21T05:39:57.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:57.918+0000 7f1581c4d640 1 -- 172.21.15.22:0/1319915601 wait complete. 2024-09-21T05:39:57.936 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T05:39:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 -- 172.21.15.22:0/3430760540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c0b6680 msgr2=0x7f754c0b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 --2- 172.21.15.22:0/3430760540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c0b6680 0x7f754c0b0de0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7544009820 tx=0x7f754402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 -- 172.21.15.22:0/3430760540 shutdown_connections 2024-09-21T05:39:58.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 --2- 172.21.15.22:0/3430760540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f754c162f60 0x7f754c165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 --2- 172.21.15.22:0/3430760540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c0b6680 0x7f754c0b0de0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 -- 172.21.15.22:0/3430760540 >> 172.21.15.22:0/3430760540 conn(0x7f754c0d12a0 msgr2=0x7f754c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:58.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 -- 172.21.15.22:0/3430760540 shutdown_connections 2024-09-21T05:39:58.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.051+0000 7f7550f2b640 1 -- 172.21.15.22:0/3430760540 wait complete. 2024-09-21T05:39:58.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.053+0000 7f7550f2b640 1 Processor -- start 2024-09-21T05:39:58.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.053+0000 7f7550f2b640 1 -- start start 2024-09-21T05:39:58.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7550f2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 0x7f754c0c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7550f2b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f754c0b6680 0x7f754c1f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7550f2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c162f60 0x7f754c1f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7550f2b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f754c165a40 con 0x7f754c0b6100 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7550f2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1658c0 con 0x7f754c162f60 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7550f2b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f754c165bc0 con 0x7f754c0b6680 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f754a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 0x7f754c0c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f754a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 0x7f754c0c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36846/0 (socket says 172.21.15.22:36846) 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f754a59c640 1 -- 172.21.15.22:0/286565196 learned_addr learned my addr 172.21.15.22:0/286565196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f7549d9b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f754c0b6680 0x7f754c1f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.054+0000 7f754ad9d640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c162f60 0x7f754c1f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f754a59c640 1 -- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f754c0b6680 msgr2=0x7f754c1f8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f754a59c640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f754c0b6680 0x7f754c1f8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f754a59c640 1 -- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c162f60 msgr2=0x7f754c1f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f754a59c640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c162f60 0x7f754c1f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f754a59c640 1 -- 172.21.15.22:0/286565196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c0b02a0 con 0x7f754c0b6100 2024-09-21T05:39:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f754a59c640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 0x7f754c0c71b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f754402aa90 tx=0x7f7544003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754403f070 con 0x7f754c0b6100 2024-09-21T05:39:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75440049b0 con 0x7f754c0b6100 2024-09-21T05:39:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c0cd420 con 0x7f754c0b6100 2024-09-21T05:39:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544004e00 con 0x7f754c0b6100 2024-09-21T05:39:58.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.055+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c163370 con 0x7f754c0b6100 2024-09-21T05:39:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.056+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c0b0de0 con 0x7f754c0b6100 2024-09-21T05:39:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.057+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7544042070 con 0x7f754c0b6100 2024-09-21T05:39:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.057+0000 7f7542ffd640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f752c078b90 0x7f752c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.058+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f75440bb6a0 con 0x7f754c0b6100 2024-09-21T05:39:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.058+0000 7f7549d9b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f752c078b90 0x7f752c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.058+0000 7f7549d9b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f752c078b90 0x7f752c07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f754c0cdf00 tx=0x7f7534001db0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:58.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.062+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7544083870 con 0x7f754c0b6100 2024-09-21T05:39:58.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.324+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f754c0ce5e0 con 0x7f754c0b6100 2024-09-21T05:39:58.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.327+0000 7f7542ffd640 1 -- 172.21.15.22:0/286565196 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f75440882a0 con 0x7f754c0b6100 2024-09-21T05:39:58.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f752c078b90 msgr2=0x7f752c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f752c078b90 0x7f752c07b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f754c0cdf00 tx=0x7f7534001db0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 msgr2=0x7f754c0c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 0x7f754c0c71b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f754402aa90 tx=0x7f7544003c10 comp rx=0 tx=0).stop 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 shutdown_connections 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f754c162f60 0x7f754c1f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f752c078b90 0x7f752c07b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f754c0b6680 0x7f754c1f8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 --2- 172.21.15.22:0/286565196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f754c0b6100 0x7f754c0c71b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 >> 172.21.15.22:0/286565196 conn(0x7f754c0d12a0 msgr2=0x7f754c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.329+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 shutdown_connections 2024-09-21T05:39:58.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.330+0000 7f7550f2b640 1 -- 172.21.15.22:0/286565196 wait complete. 2024-09-21T05:39:58.343 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T05:39:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 -- 172.21.15.22:0/3831216742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda601600b0 msgr2=0x7fda600ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 --2- 172.21.15.22:0/3831216742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda601600b0 0x7fda600ccbc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fda50005fd0 tx=0x7fda5002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 -- 172.21.15.22:0/3831216742 shutdown_connections 2024-09-21T05:39:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 --2- 172.21.15.22:0/3831216742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda600b7cc0 0x7fda600cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 --2- 172.21.15.22:0/3831216742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda601600b0 0x7fda600ccbc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 -- 172.21.15.22:0/3831216742 >> 172.21.15.22:0/3831216742 conn(0x7fda6006d4f0 msgr2=0x7fda60071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 -- 172.21.15.22:0/3831216742 shutdown_connections 2024-09-21T05:39:58.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.458+0000 7fda65463640 1 -- 172.21.15.22:0/3831216742 wait complete. 2024-09-21T05:39:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.460+0000 7fda65463640 1 Processor -- start 2024-09-21T05:39:58.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.460+0000 7fda65463640 1 -- start start 2024-09-21T05:39:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda65463640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 0x7fda601f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda65463640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600cf150 0x7fda600ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda65463640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda601600b0 0x7fda601f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda65463640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fda600b4460 con 0x7fda600b7cc0 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda65463640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fda600d1be0 con 0x7fda600cf150 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda65463640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fda600b45e0 con 0x7fda601600b0 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda5effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 0x7fda601f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda5effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 0x7fda601f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36862/0 (socket says 172.21.15.22:36862) 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda5effd640 1 -- 172.21.15.22:0/867230469 learned_addr learned my addr 172.21.15.22:0/867230469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda5e7fc640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600cf150 0x7fda600ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.461+0000 7fda5f7fe640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda601600b0 0x7fda601f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda5effd640 1 -- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda601600b0 msgr2=0x7fda601f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda5effd640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda601600b0 0x7fda601f74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda5effd640 1 -- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600cf150 msgr2=0x7fda600ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda5effd640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600cf150 0x7fda600ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda5effd640 1 -- 172.21.15.22:0/867230469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda600cb660 con 0x7fda600b7cc0 2024-09-21T05:39:58.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda5effd640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 0x7fda601f36f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda50006010 tx=0x7fda500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5003e030 con 0x7fda600b7cc0 2024-09-21T05:39:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda50003ca0 con 0x7fda600b7cc0 2024-09-21T05:39:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50035040 con 0x7fda600b7cc0 2024-09-21T05:39:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda600cb940 con 0x7fda600b7cc0 2024-09-21T05:39:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.462+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda600cbf00 con 0x7fda600b7cc0 2024-09-21T05:39:58.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.463+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda600b4b20 con 0x7fda600b7cc0 2024-09-21T05:39:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.464+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fda50041030 con 0x7fda600b7cc0 2024-09-21T05:39:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.465+0000 7fda477fe640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fda4c078b90 0x7fda4c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.465+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fda500baf00 con 0x7fda600b7cc0 2024-09-21T05:39:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.465+0000 7fda5e7fc640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fda4c078b90 0x7fda4c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.466+0000 7fda5e7fc640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fda4c078b90 0x7fda4c07b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda601f7430 tx=0x7fda5400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:58.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.469+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda500830d0 con 0x7fda600b7cc0 2024-09-21T05:39:58.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.732+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fda600b3a70 con 0x7fda600b7cc0 2024-09-21T05:39:58.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.735+0000 7fda477fe640 1 -- 172.21.15.22:0/867230469 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fda50087b00 con 0x7fda600b7cc0 2024-09-21T05:39:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fda4c078b90 msgr2=0x7fda4c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fda4c078b90 0x7fda4c07b0a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fda601f7430 tx=0x7fda5400b2e0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 msgr2=0x7fda601f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 0x7fda601f36f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fda50006010 tx=0x7fda500038a0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 shutdown_connections 2024-09-21T05:39:58.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fda4c078b90 0x7fda4c07b0a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fda601600b0 0x7fda601f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fda600cf150 0x7fda600ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 --2- 172.21.15.22:0/867230469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fda600b7cc0 0x7fda601f36f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 >> 172.21.15.22:0/867230469 conn(0x7fda6006d4f0 msgr2=0x7fda600b7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 shutdown_connections 2024-09-21T05:39:58.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.737+0000 7fda65463640 1 -- 172.21.15.22:0/867230469 wait complete. 2024-09-21T05:39:58.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T05:39:58.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 -- 172.21.15.22:0/1867541340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cffd0 msgr2=0x7f15380d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 --2- 172.21.15.22:0/1867541340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cffd0 0x7f15380d4940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1530005fd0 tx=0x7f153002aa10 comp rx=0 tx=0).stop 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 -- 172.21.15.22:0/1867541340 shutdown_connections 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 --2- 172.21.15.22:0/1867541340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cffd0 0x7f15380d4940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 --2- 172.21.15.22:0/1867541340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15380b3910 0x7f15380b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 -- 172.21.15.22:0/1867541340 >> 172.21.15.22:0/1867541340 conn(0x7f153808bfc0 msgr2=0x7f153809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 -- 172.21.15.22:0/1867541340 shutdown_connections 2024-09-21T05:39:58.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.865+0000 7f153faf2640 1 -- 172.21.15.22:0/1867541340 wait complete. 2024-09-21T05:39:58.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.867+0000 7f153faf2640 1 Processor -- start 2024-09-21T05:39:58.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.867+0000 7f153faf2640 1 -- start start 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.868+0000 7f153faf2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15380b3910 0x7f15380d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.868+0000 7f153faf2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 0x7f15380d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.868+0000 7f153faf2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15380cffd0 0x7f15381f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.868+0000 7f153faf2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f15380d5000 con 0x7f15380cfbc0 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.868+0000 7f153faf2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15380d4e80 con 0x7f15380cffd0 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.868+0000 7f153faf2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15380b3340 con 0x7f15380b3910 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 0x7f15380d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 0x7f15380d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36884/0 (socket says 172.21.15.22:36884) 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 -- 172.21.15.22:0/1313369973 learned_addr learned my addr 172.21.15.22:0/1313369973 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:58.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153e08f640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15380cffd0 0x7f15381f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d88e640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15380b3910 0x7f15380d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 -- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15380b3910 msgr2=0x7f15380d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15380b3910 0x7f15380d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 -- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15380cffd0 msgr2=0x7f15381f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15380cffd0 0x7f15381f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 -- 172.21.15.22:0/1313369973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15380c9470 con 0x7f15380cfbc0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153d08d640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 0x7f15380d15f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f152800c400 tx=0x7f152800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528017070 con 0x7f15380cfbc0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15280053c0 con 0x7f15380cfbc0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15380c9750 con 0x7f15380cfbc0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528003830 con 0x7f15380cfbc0 2024-09-21T05:39:58.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.869+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15380c9d10 con 0x7f15380cfbc0 2024-09-21T05:39:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.870+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380b5bf0 con 0x7f15380cfbc0 2024-09-21T05:39:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.871+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f152801a070 con 0x7f15380cfbc0 2024-09-21T05:39:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.872+0000 7f152e7fc640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f151c078b90 0x7f151c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.872+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f1528099d00 con 0x7f15380cfbc0 2024-09-21T05:39:58.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.872+0000 7f153d88e640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f151c078b90 0x7f151c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:58.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.873+0000 7f153d88e640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f151c078b90 0x7f151c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1534009c00 tx=0x7f1534005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:58.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:58.876+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1528061e40 con 0x7f15380cfbc0 2024-09-21T05:39:59.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.138+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f15380b4230 con 0x7f15380cfbc0 2024-09-21T05:39:59.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.141+0000 7f152e7fc640 1 -- 172.21.15.22:0/1313369973 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f1528066870 con 0x7f15380cfbc0 2024-09-21T05:39:59.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f151c078b90 msgr2=0x7f151c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f151c078b90 0x7f151c07b0a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1534009c00 tx=0x7f1534005f30 comp rx=0 tx=0).stop 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 msgr2=0x7f15380d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 0x7f15380d15f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f152800c400 tx=0x7f152800c8c0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 shutdown_connections 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15380cffd0 0x7f15381f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15380cfbc0 0x7f15380d15f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f151c078b90 0x7f151c07b0a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 --2- 172.21.15.22:0/1313369973 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15380b3910 0x7f15380d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 >> 172.21.15.22:0/1313369973 conn(0x7f153808bfc0 msgr2=0x7f15380b9420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 shutdown_connections 2024-09-21T05:39:59.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.143+0000 7f153faf2640 1 -- 172.21.15.22:0/1313369973 wait complete. 2024-09-21T05:39:59.156 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.272+0000 7f62b764d640 1 -- 172.21.15.22:0/2862158040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f629c003c10 msgr2=0x7f629c004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.272+0000 7f62b764d640 1 --2- 172.21.15.22:0/2862158040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f629c003c10 0x7f629c004060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f62ac006f70 tx=0x7f629c005830 comp rx=0 tx=0).stop 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.272+0000 7f62b5bea640 1 -- 172.21.15.22:0/2862158040 reap_dead start 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.272+0000 7f62b764d640 1 -- 172.21.15.22:0/2862158040 shutdown_connections 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.272+0000 7f62b764d640 1 -- 172.21.15.22:0/2862158040 >> 172.21.15.22:0/2862158040 conn(0x7f62b009d7f0 msgr2=0x7f62b00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.272+0000 7f62b764d640 1 -- 172.21.15.22:0/2862158040 shutdown_connections 2024-09-21T05:39:59.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.273+0000 7f62b764d640 1 -- 172.21.15.22:0/2862158040 wait complete. 2024-09-21T05:39:59.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.274+0000 7f62b764d640 1 Processor -- start 2024-09-21T05:39:59.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.274+0000 7f62b764d640 1 -- start start 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f629c003c10 0x7f62b01f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b01e7e20 0x7f62b0160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 0x7f62b00ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d5110 con 0x7f62b00ca120 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d4f90 con 0x7f62b01e7e20 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f62b006f1c0 con 0x7f629c003c10 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 0x7f62b00ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 0x7f62b00ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36900/0 (socket says 172.21.15.22:36900) 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 -- 172.21.15.22:0/1547238001 learned_addr learned my addr 172.21.15.22:0/1547238001 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:59.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b4be8640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b01e7e20 0x7f62b0160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b53e9640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f629c003c10 0x7f62b01f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 -- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f629c003c10 msgr2=0x7f62b01f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f629c003c10 0x7f62b01f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 -- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b01e7e20 msgr2=0x7f62b0160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b01e7e20 0x7f62b0160c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 -- 172.21.15.22:0/1547238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b00caa70 con 0x7f62b00ca120 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b4be8640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b01e7e20 0x7f62b0160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b53e9640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f629c003c10 0x7f62b01f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b5bea640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 0x7f62b00ca530 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f62b00cb520 tx=0x7f62a8001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8003b80 con 0x7f62b00ca120 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62a8003e30 con 0x7f62b00ca120 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b00c83c0 con 0x7f62b00ca120 2024-09-21T05:39:59.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.276+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8004290 con 0x7f62b00ca120 2024-09-21T05:39:59.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.277+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b00c8980 con 0x7f62b00ca120 2024-09-21T05:39:59.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.277+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b00b5fa0 con 0x7f62b00ca120 2024-09-21T05:39:59.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.278+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62a8019020 con 0x7f62b00ca120 2024-09-21T05:39:59.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.279+0000 7f62a5ffb640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6294078b90 0x7f629407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.279+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f62a809ad00 con 0x7f62b00ca120 2024-09-21T05:39:59.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.279+0000 7f62b53e9640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6294078b90 0x7f629407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.279+0000 7f62b53e9640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6294078b90 0x7f629407b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62ac033000 tx=0x7f62ac02d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:59.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.283+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62b00caa70 con 0x7f62b00ca120 2024-09-21T05:39:59.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.547+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f62b00c8110 con 0x7f62b00ca120 2024-09-21T05:39:59.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.550+0000 7f62a5ffb640 1 -- 172.21.15.22:0/1547238001 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f62a8063490 con 0x7f62b00ca120 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.552+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6294078b90 msgr2=0x7f629407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.552+0000 7f62b764d640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6294078b90 0x7f629407b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f62ac033000 tx=0x7f62ac02d000 comp rx=0 tx=0).stop 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.552+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 msgr2=0x7f62b00ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.552+0000 7f62b764d640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 0x7f62b00ca530 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f62b00cb520 tx=0x7f62a8001b80 comp rx=0 tx=0).stop 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 shutdown_connections 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62b00ca120 0x7f62b00ca530 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62b01e7e20 0x7f62b0160c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6294078b90 0x7f629407b0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 --2- 172.21.15.22:0/1547238001 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f629c003c10 0x7f62b01f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 >> 172.21.15.22:0/1547238001 conn(0x7f62b009d7f0 msgr2=0x7f62b00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 shutdown_connections 2024-09-21T05:39:59.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.553+0000 7f62b764d640 1 -- 172.21.15.22:0/1547238001 wait complete. 2024-09-21T05:39:59.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4be569640 1 -- 172.21.15.22:0/1428841252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a0004800 msgr2=0x7fa4a0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4be569640 1 --2- 172.21.15.22:0/1428841252 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4a0004800 0x7fa4a0004c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80b7120 tx=0x7fa4a0005830 comp rx=0 tx=0).stop 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4bcb06640 1 -- 172.21.15.22:0/1428841252 reap_dead start 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4be569640 1 -- 172.21.15.22:0/1428841252 shutdown_connections 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4be569640 1 -- 172.21.15.22:0/1428841252 >> 172.21.15.22:0/1428841252 conn(0x7fa4b808bfc0 msgr2=0x7fa4b809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4be569640 1 -- 172.21.15.22:0/1428841252 shutdown_connections 2024-09-21T05:39:59.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.682+0000 7fa4be569640 1 -- 172.21.15.22:0/1428841252 wait complete. 2024-09-21T05:39:59.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.684+0000 7fa4be569640 1 Processor -- start 2024-09-21T05:39:59.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.684+0000 7fa4be569640 1 -- start start 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4be569640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0004800 0x7fa4b80cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4be569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 0x7fa4b80a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4be569640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4b81f5aa0 0x7fa4b80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4be569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80cfd70 con 0x7fa4b80cbe30 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4be569640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b80cfbf0 con 0x7fa4a0004800 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4be569640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b80a17e0 con 0x7fa4b81f5aa0 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 0x7fa4b80a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 0x7fa4b80a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36910/0 (socket says 172.21.15.22:36910) 2024-09-21T05:39:59.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 -- 172.21.15.22:0/3300846763 learned_addr learned my addr 172.21.15.22:0/3300846763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4bcb06640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4b81f5aa0 0x7fa4b80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b7fff640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0004800 0x7fa4b80cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 -- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4b81f5aa0 msgr2=0x7fa4b80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4b81f5aa0 0x7fa4b80c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 -- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0004800 msgr2=0x7fa4b80cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0004800 0x7fa4b80cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 -- 172.21.15.22:0/3300846763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b809e4e0 con 0x7fa4b80cbe30 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.686+0000 7fa4b77fe640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 0x7fa4b80a0c40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac036040 tx=0x7fa4ac033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.687+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac043070 con 0x7fa4b80cbe30 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.687+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4ac004120 con 0x7fa4b80cbe30 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.687+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b80ce440 con 0x7fa4b80cbe30 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.687+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac038040 con 0x7fa4b80cbe30 2024-09-21T05:39:59.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.687+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81f69b0 con 0x7fa4b80cbe30 2024-09-21T05:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.688+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b81630e0 con 0x7fa4b80cbe30 2024-09-21T05:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.688+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa4ac046070 con 0x7fa4b80cbe30 2024-09-21T05:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.689+0000 7fa4b4ff9640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa4a4078b90 0x7fa4a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.689+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fa4ac0baed0 con 0x7fa4b80cbe30 2024-09-21T05:39:59.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.689+0000 7fa4b7fff640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa4a4078b90 0x7fa4a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:39:59.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.690+0000 7fa4b7fff640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa4a4078b90 0x7fa4a407b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006fd0 tx=0x7fa4a8006b60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:39:59.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.693+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4ac0830a0 con 0x7fa4b80cbe30 2024-09-21T05:39:59.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.952+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fa4b80d0440 con 0x7fa4b80cbe30 2024-09-21T05:39:59.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.954+0000 7fa4b4ff9640 1 -- 172.21.15.22:0/3300846763 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fa4ac087ad0 con 0x7fa4b80cbe30 2024-09-21T05:39:59.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.956+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa4a4078b90 msgr2=0x7fa4a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.956+0000 7fa4be569640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa4a4078b90 0x7fa4a407b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006fd0 tx=0x7fa4a8006b60 comp rx=0 tx=0).stop 2024-09-21T05:39:59.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.956+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 msgr2=0x7fa4b80a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:39:59.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.956+0000 7fa4be569640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 0x7fa4b80a0c40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac036040 tx=0x7fa4ac033670 comp rx=0 tx=0).stop 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 shutdown_connections 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa4a4078b90 0x7fa4a407b0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4b81f5aa0 0x7fa4b80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4b80cbe30 0x7fa4b80a0c40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 --2- 172.21.15.22:0/3300846763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4a0004800 0x7fa4b80cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 >> 172.21.15.22:0/3300846763 conn(0x7fa4b808bfc0 msgr2=0x7fa4b806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 shutdown_connections 2024-09-21T05:39:59.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:39:59.957+0000 7fa4be569640 1 -- 172.21.15.22:0/3300846763 wait complete. 2024-09-21T05:39:59.970 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f35a9b640 1 -- 172.21.15.22:0/1964740094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f18003a00 msgr2=0x7f0f18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/1964740094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f18003a00 0x7f0f18003e70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000bae0 tx=0x7f0f18005830 comp rx=0 tx=0).stop 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f2ffff640 1 -- 172.21.15.22:0/1964740094 reap_dead start 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f35a9b640 1 -- 172.21.15.22:0/1964740094 shutdown_connections 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f35a9b640 1 -- 172.21.15.22:0/1964740094 >> 172.21.15.22:0/1964740094 conn(0x7f0f3009d7f0 msgr2=0x7f0f300d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f35a9b640 1 -- 172.21.15.22:0/1964740094 shutdown_connections 2024-09-21T05:40:00.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.086+0000 7f0f35a9b640 1 -- 172.21.15.22:0/1964740094 wait complete. 2024-09-21T05:40:00.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.088+0000 7f0f35a9b640 1 Processor -- start 2024-09-21T05:40:00.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.088+0000 7f0f35a9b640 1 -- start start 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f18003a00 0x7f0f301f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3009eb50 0x7f0f301f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 0x7f0f300a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300d4f00 con 0x7f0f30160750 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0f300d4d80 con 0x7f0f18003a00 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0f300d5080 con 0x7f0f3009eb50 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 0x7f0f300a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 0x7f0f300a17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36938/0 (socket says 172.21.15.22:36938) 2024-09-21T05:40:00.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 -- 172.21.15.22:0/2690509390 learned_addr learned my addr 172.21.15.22:0/2690509390 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2f7fe640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f18003a00 0x7f0f301f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2effd640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3009eb50 0x7f0f301f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 -- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3009eb50 msgr2=0x7f0f301f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3009eb50 0x7f0f301f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 -- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f18003a00 msgr2=0x7f0f301f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f18003a00 0x7f0f301f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 -- 172.21.15.22:0/2690509390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f300a0750 con 0x7f0f30160750 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2f7fe640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f18003a00 0x7f0f301f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2effd640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3009eb50 0x7f0f301f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f2ffff640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 0x7f0f300a17f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0f18004780 tx=0x7f0f28006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f280107d0 con 0x7f0f30160750 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f28016070 con 0x7f0f30160750 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f300a0a30 con 0x7f0f30160750 2024-09-21T05:40:00.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.090+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f28010d00 con 0x7f0f30160750 2024-09-21T05:40:00.095 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.091+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f300a0ff0 con 0x7f0f30160750 2024-09-21T05:40:00.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.092+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f300b5fa0 con 0x7f0f30160750 2024-09-21T05:40:00.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.092+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f2801b460 con 0x7f0f30160750 2024-09-21T05:40:00.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.093+0000 7f0f17fff640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.093+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f0f2809b360 con 0x7f0f30160750 2024-09-21T05:40:00.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.093+0000 7f0f2f7fe640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.094+0000 7f0f2f7fe640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f20036000 tx=0x7f0f200046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:00.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.098+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f300b68f0 con 0x7f0f30160750 2024-09-21T05:40:00.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.360+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f0f300d3fb0 con 0x7f0f30160750 2024-09-21T05:40:00.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.362+0000 7f0f17fff640 1 -- 172.21.15.22:0/2690509390 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f0f2806c020 con 0x7f0f30160750 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.364+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0f1c078bc0 msgr2=0x7f0f1c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.364+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0f20036000 tx=0x7f0f200046c0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.364+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 msgr2=0x7f0f300a17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.364+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 0x7f0f300a17f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0f18004780 tx=0x7f0f28006420 comp rx=0 tx=0).stop 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 shutdown_connections 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0f30160750 0x7f0f300a17f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0f1c078bc0 0x7f0f1c07b0d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0f3009eb50 0x7f0f301f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 --2- 172.21.15.22:0/2690509390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0f18003a00 0x7f0f301f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 >> 172.21.15.22:0/2690509390 conn(0x7f0f3009d7f0 msgr2=0x7f0f300b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:00.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 shutdown_connections 2024-09-21T05:40:00.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.365+0000 7f0f35a9b640 1 -- 172.21.15.22:0/2690509390 wait complete. 2024-09-21T05:40:00.378 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:40:00.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 -- 172.21.15.22:0/2688603128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0070030 msgr2=0x7fdde0074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 --2- 172.21.15.22:0/2688603128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0070030 0x7fdde0074640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009830 tx=0x7fddc802a510 comp rx=0 tx=0).stop 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 -- 172.21.15.22:0/2688603128 shutdown_connections 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 --2- 172.21.15.22:0/2688603128 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 0x7fdde0074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 --2- 172.21.15.22:0/2688603128 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0070030 0x7fdde0074640 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 -- 172.21.15.22:0/2688603128 >> 172.21.15.22:0/2688603128 conn(0x7fdde00b1cb0 msgr2=0x7fdde00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 -- 172.21.15.22:0/2688603128 shutdown_connections 2024-09-21T05:40:00.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.493+0000 7fdde5120640 1 -- 172.21.15.22:0/2688603128 wait complete. 2024-09-21T05:40:00.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.495+0000 7fdde5120640 1 Processor -- start 2024-09-21T05:40:00.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.495+0000 7fdde5120640 1 -- start start 2024-09-21T05:40:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde0070030 0x7fdde0154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 0x7fdde01f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 0x7fdde01f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00b3a40 con 0x7fdde0073170 2024-09-21T05:40:00.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdde00b38c0 con 0x7fdde0070030 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdde00b3bc0 con 0x7fdde0070440 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 0x7fdde01f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 0x7fdde01f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36958/0 (socket says 172.21.15.22:36958) 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 -- 172.21.15.22:0/972669072 learned_addr learned my addr 172.21.15.22:0/972669072 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fddde59c640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 0x7fdde01f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fddded9d640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde0070030 0x7fdde0154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 -- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 msgr2=0x7fdde01f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 0x7fdde01f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 -- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde0070030 msgr2=0x7fdde0154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde0070030 0x7fdde0154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 -- 172.21.15.22:0/972669072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde0158910 con 0x7fdde0073170 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fddde59c640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 0x7fdde01f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:00.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdddf59e640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 0x7fdde01f70f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c940 tx=0x7fddd800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8004460 con 0x7fdde0073170 2024-09-21T05:40:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddd80077a0 con 0x7fdde0073170 2024-09-21T05:40:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.497+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde0158bf0 con 0x7fdde0073170 2024-09-21T05:40:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.498+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd8007c10 con 0x7fdde0073170 2024-09-21T05:40:00.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.498+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde01591b0 con 0x7fdde0073170 2024-09-21T05:40:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.499+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde00b40e0 con 0x7fdde0073170 2024-09-21T05:40:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.499+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fddd80035f0 con 0x7fdde0073170 2024-09-21T05:40:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.500+0000 7fddd77fe640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddc0078b90 0x7fddc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.500+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fddd809b220 con 0x7fdde0073170 2024-09-21T05:40:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.500+0000 7fddded9d640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddc0078b90 0x7fddc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.501+0000 7fddded9d640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddc0078b90 0x7fddc007b0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddd0002020 tx=0x7fddd0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:00.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.504+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fddd8013030 con 0x7fdde0073170 2024-09-21T05:40:00.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.695+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdde00c3860 con 0x7fdde0073170 2024-09-21T05:40:00.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.696+0000 7fddd77fe640 1 -- 172.21.15.22:0/972669072 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7fddd800de20 con 0x7fdde0073170 2024-09-21T05:40:00.698 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:00.698 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:40:00.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:00.699 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:40:00.699 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:40:00.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:00.699 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddc0078b90 msgr2=0x7fddc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddc0078b90 0x7fddc007b0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddd0002020 tx=0x7fddd0009040 comp rx=0 tx=0).stop 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 msgr2=0x7fdde01f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 0x7fdde01f70f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddd800c940 tx=0x7fddd800ce00 comp rx=0 tx=0).stop 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 shutdown_connections 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde0073170 0x7fdde01f70f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddc0078b90 0x7fddc007b0a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdde0070440 0x7fdde01f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 --2- 172.21.15.22:0/972669072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdde0070030 0x7fdde0154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 >> 172.21.15.22:0/972669072 conn(0x7fdde00b1cb0 msgr2=0x7fdde00ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:00.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.698+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 shutdown_connections 2024-09-21T05:40:00.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.699+0000 7fdde5120640 1 -- 172.21.15.22:0/972669072 wait complete. 2024-09-21T05:40:00.712 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:40:00.712 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 -- 172.21.15.22:0/769801558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 msgr2=0x7f66840d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 --2- 172.21.15.22:0/769801558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f66840d28b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6680005fd0 tx=0x7f668002a670 comp rx=0 tx=0).stop 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 -- 172.21.15.22:0/769801558 shutdown_connections 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 --2- 172.21.15.22:0/769801558 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f66840d28b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 -- 172.21.15.22:0/769801558 >> 172.21.15.22:0/769801558 conn(0x7f66840d45c0 msgr2=0x7f66840d49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 -- 172.21.15.22:0/769801558 shutdown_connections 2024-09-21T05:40:00.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.831+0000 7f668bf89640 1 -- 172.21.15.22:0/769801558 wait complete. 2024-09-21T05:40:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.833+0000 7f668bf89640 1 Processor -- start 2024-09-21T05:40:00.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.833+0000 7f668bf89640 1 -- start start 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.834+0000 7f668bf89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66840cf280 0x7f66840ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.834+0000 7f668bf89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f6684160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.834+0000 7f668bf89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f668415eb70 0x7f668406e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.834+0000 7f668bf89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f66840b2ca0 con 0x7f66840d24a0 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.834+0000 7f668bf89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f66840d2df0 con 0x7f668415eb70 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.834+0000 7f668bf89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f66840b2e20 con 0x7f66840cf280 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f6684160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f6684160a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36974/0 (socket says 172.21.15.22:36974) 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 -- 172.21.15.22:0/984364974 learned_addr learned my addr 172.21.15.22:0/984364974 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:00.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f668a526640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f668415eb70 0x7f668406e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 -- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66840cf280 msgr2=0x7f66840ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689d25640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66840cf280 0x7f66840ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66840cf280 0x7f66840ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 -- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f668415eb70 msgr2=0x7f668406e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f668415eb70 0x7f668406e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 -- 172.21.15.22:0/984364974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668415f980 con 0x7f66840d24a0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f668a526640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f668415eb70 0x7f668406e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689d25640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66840cf280 0x7f66840ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f6689524640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f6684160a50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6680034040 tx=0x7f6680003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668003e030 con 0x7f66840d24a0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6680003e00 con 0x7f66840d24a0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668415fc60 con 0x7f66840d24a0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680035040 con 0x7f66840d24a0 2024-09-21T05:40:00.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.835+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668414b9f0 con 0x7f66840d24a0 2024-09-21T05:40:00.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.837+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668415f6f0 con 0x7f66840d24a0 2024-09-21T05:40:00.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.837+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6680041030 con 0x7f66840d24a0 2024-09-21T05:40:00.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.838+0000 7f667a7fc640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:00.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.838+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f66800bb300 con 0x7f66840d24a0 2024-09-21T05:40:00.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.838+0000 7f6689d25640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:00.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.839+0000 7f6689d25640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6674007fd0 tx=0x7f6674009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:00.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:00.842+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6680083a90 con 0x7f66840d24a0 2024-09-21T05:40:01.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.029+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6684076820 con 0x7f66840d24a0 2024-09-21T05:40:01.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.557+0000 7f667a7fc640 1 -- 172.21.15.22:0/984364974 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f6680083390 con 0x7f66840d24a0 2024-09-21T05:40:01.559 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 msgr2=0x7f667007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6674007fd0 tx=0x7f6674009040 comp rx=0 tx=0).stop 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 msgr2=0x7f6684160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f6684160a50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6680034040 tx=0x7f6680003910 comp rx=0 tx=0).stop 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 shutdown_connections 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f668415eb70 0x7f668406e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f66840d24a0 0x7f6684160a50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 --2- 172.21.15.22:0/984364974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f66840cf280 0x7f66840ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 >> 172.21.15.22:0/984364974 conn(0x7f66840d45c0 msgr2=0x7f6684157330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 shutdown_connections 2024-09-21T05:40:01.562 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.559+0000 7f668bf89640 1 -- 172.21.15.22:0/984364974 wait complete. 2024-09-21T05:40:01.577 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:40:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f630355e640 1 -- 172.21.15.22:0/781345753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62e8004d90 msgr2=0x7f62e8005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f630355e640 1 --2- 172.21.15.22:0/781345753 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62e8004d90 0x7f62e8005240 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f62f4001f00 tx=0x7f62e8005830 comp rx=0 tx=0).stop 2024-09-21T05:40:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f6301afb640 1 -- 172.21.15.22:0/781345753 reap_dead start 2024-09-21T05:40:01.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f630355e640 1 -- 172.21.15.22:0/781345753 shutdown_connections 2024-09-21T05:40:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f630355e640 1 -- 172.21.15.22:0/781345753 >> 172.21.15.22:0/781345753 conn(0x7f62fc06d7d0 msgr2=0x7f62fc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f630355e640 1 -- 172.21.15.22:0/781345753 shutdown_connections 2024-09-21T05:40:01.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.703+0000 7f630355e640 1 -- 172.21.15.22:0/781345753 wait complete. 2024-09-21T05:40:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.705+0000 7f630355e640 1 Processor -- start 2024-09-21T05:40:01.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.705+0000 7f630355e640 1 -- start start 2024-09-21T05:40:01.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f630355e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e8004d90 0x7f62fc1e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f630355e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 0x7f62fc141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f630355e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62fc06f7a0 0x7f62fc06fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f630355e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc0b44a0 con 0x7f62fc09b6e0 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f630355e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62fc0b4320 con 0x7f62e8004d90 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f630355e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f62fc0b4620 con 0x7f62fc06f7a0 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f6300af9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 0x7f62fc141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f6300af9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 0x7f62fc141a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36986/0 (socket says 172.21.15.22:36986) 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f6300af9640 1 -- 172.21.15.22:0/3559029224 learned_addr learned my addr 172.21.15.22:0/3559029224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f63012fa640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e8004d90 0x7f62fc1e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.706+0000 7f6301afb640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62fc06f7a0 0x7f62fc06fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f6300af9640 1 -- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62fc06f7a0 msgr2=0x7f62fc06fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f6300af9640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62fc06f7a0 0x7f62fc06fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:01.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f6300af9640 1 -- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e8004d90 msgr2=0x7f62fc1e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f6300af9640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e8004d90 0x7f62fc1e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f6300af9640 1 -- 172.21.15.22:0/3559029224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62fc0700f0 con 0x7f62fc09b6e0 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f6300af9640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 0x7f62fc141a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62f8011ae0 tx=0x7f62f8009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f801e070 con 0x7f62fc09b6e0 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62f8004460 con 0x7f62fc09b6e0 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f800fc40 con 0x7f62fc09b6e0 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62fc1e5830 con 0x7f62fc09b6e0 2024-09-21T05:40:01.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.707+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62fc1e5df0 con 0x7f62fc09b6e0 2024-09-21T05:40:01.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.708+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62fc1e55a0 con 0x7f62fc09b6e0 2024-09-21T05:40:01.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.709+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62f8021070 con 0x7f62fc09b6e0 2024-09-21T05:40:01.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.710+0000 7f62edffb640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f62e0078b90 0x7f62e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:01.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.710+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7f62f809be40 con 0x7f62fc09b6e0 2024-09-21T05:40:01.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.710+0000 7f63012fa640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f62e0078b90 0x7f62e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:01.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.711+0000 7f63012fa640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f62e0078b90 0x7f62e007b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62f0001b40 tx=0x7f62f0001ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:01.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.714+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62fc0700f0 con 0x7f62fc09b6e0 2024-09-21T05:40:01.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:01.904+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f62fc15ea20 con 0x7f62fc09b6e0 2024-09-21T05:40:02.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.562+0000 7f62edffb640 1 -- 172.21.15.22:0/3559029224 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7f62f8063eb0 con 0x7f62fc09b6e0 2024-09-21T05:40:02.564 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:40:02.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f62e0078b90 msgr2=0x7f62e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f62e0078b90 0x7f62e007b0a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f62f0001b40 tx=0x7f62f0001ad0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 msgr2=0x7f62fc141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 0x7f62fc141a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f62f8011ae0 tx=0x7f62f8009e90 comp rx=0 tx=0).stop 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 shutdown_connections 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f62e0078b90 0x7f62e007b0a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62fc06f7a0 0x7f62fc06fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62fc09b6e0 0x7f62fc141a70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 --2- 172.21.15.22:0/3559029224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62e8004d90 0x7f62fc1e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 >> 172.21.15.22:0/3559029224 conn(0x7f62fc06d7d0 msgr2=0x7f62fc0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 shutdown_connections 2024-09-21T05:40:02.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.565+0000 7f630355e640 1 -- 172.21.15.22:0/3559029224 wait complete. 2024-09-21T05:40:02.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 -- 172.21.15.22:0/3538368946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4161020 msgr2=0x7f8fc40ccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 --2- 172.21.15.22:0/3538368946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4161020 0x7f8fc40ccb70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40b1d10 tx=0x7f8fb402a560 comp rx=0 tx=0).stop 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 -- 172.21.15.22:0/3538368946 shutdown_connections 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 --2- 172.21.15.22:0/3538368946 >> v2:172.21.15.134:3300/0 conn(0x7f8fc40cec50 0x7f8fc40cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 --2- 172.21.15.22:0/3538368946 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fc4156f10 0x7f8fc415d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 --2- 172.21.15.22:0/3538368946 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4161020 0x7f8fc40ccb70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 -- 172.21.15.22:0/3538368946 >> 172.21.15.22:0/3538368946 conn(0x7f8fc406d4f0 msgr2=0x7f8fc4071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 -- 172.21.15.22:0/3538368946 shutdown_connections 2024-09-21T05:40:02.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.704+0000 7f8fc9230640 1 -- 172.21.15.22:0/3538368946 wait complete. 2024-09-21T05:40:02.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.706+0000 7f8fc9230640 1 Processor -- start 2024-09-21T05:40:02.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.706+0000 7f8fc9230640 1 -- start start 2024-09-21T05:40:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc9230640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc40cec50 0x7f8fc40b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc9230640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 0x7f8fc41590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc9230640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fc4161020 0x7f8fc40cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc9230640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc40d06b0 con 0x7f8fc4156f10 2024-09-21T05:40:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc9230640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc40d0530 con 0x7f8fc40cec50 2024-09-21T05:40:02.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc9230640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc40d0830 con 0x7f8fc4161020 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 0x7f8fc41590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 0x7f8fc41590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36990/0 (socket says 172.21.15.22:36990) 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 -- 172.21.15.22:0/1518544230 learned_addr learned my addr 172.21.15.22:0/1518544230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc359e640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fc4161020 0x7f8fc40cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc2d9d640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc40cec50 0x7f8fc40b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 -- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fc4161020 msgr2=0x7f8fc40cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fc4161020 0x7f8fc40cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 -- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc40cec50 msgr2=0x7f8fc40b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc40cec50 0x7f8fc40b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.708+0000 7f8fc259c640 1 -- 172.21.15.22:0/1518544230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc40c93c0 con 0x7f8fc4156f10 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.709+0000 7f8fc259c640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 0x7f8fc41590e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800e750 tx=0x7f8fb800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.709+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800cab0 con 0x7f8fc4156f10 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.709+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8fb800cc50 con 0x7f8fc4156f10 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.709+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc40c96a0 con 0x7f8fc4156f10 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.709+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8005280 con 0x7f8fc4156f10 2024-09-21T05:40:02.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.709+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc40c9c60 con 0x7f8fc4156f10 2024-09-21T05:40:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.710+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc415d9c0 con 0x7f8fc4156f10 2024-09-21T05:40:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.711+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8fb8005420 con 0x7f8fc4156f10 2024-09-21T05:40:02.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.711+0000 7f8fab7fe640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8fb0078b90 0x7f8fb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.712+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f8fb809ba80 con 0x7f8fc4156f10 2024-09-21T05:40:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.712+0000 7f8fc2d9d640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8fb0078b90 0x7f8fb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:02.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.712+0000 7f8fc2d9d640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8fb0078b90 0x7f8fb007b0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4033000 tx=0x7f8fb4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:02.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.716+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8fb800ce20 con 0x7f8fc4156f10 2024-09-21T05:40:02.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:02.940+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f8fc40ccb70 con 0x7f8fc4156f10 2024-09-21T05:40:03.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.583+0000 7f8fab7fe640 1 -- 172.21.15.22:0/1518544230 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f8fb8063f20 con 0x7f8fc4156f10 2024-09-21T05:40:03.592 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 4 and data pool 5 2024-09-21T05:40:03.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.595+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8fb0078b90 msgr2=0x7f8fb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.595+0000 7f8fc9230640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8fb0078b90 0x7f8fb007b0a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4033000 tx=0x7f8fb4004730 comp rx=0 tx=0).stop 2024-09-21T05:40:03.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.595+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 msgr2=0x7f8fc41590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.595+0000 7f8fc9230640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 0x7f8fc41590e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800e750 tx=0x7f8fb800eb50 comp rx=0 tx=0).stop 2024-09-21T05:40:03.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 shutdown_connections 2024-09-21T05:40:03.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8fb0078b90 0x7f8fb007b0a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8fc4161020 0x7f8fc40cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8fc4156f10 0x7f8fc41590e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 --2- 172.21.15.22:0/1518544230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8fc40cec50 0x7f8fc40b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 >> 172.21.15.22:0/1518544230 conn(0x7f8fc406d4f0 msgr2=0x7f8fc4071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:03.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.598+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 shutdown_connections 2024-09-21T05:40:03.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.599+0000 7f8fc9230640 1 -- 172.21.15.22:0/1518544230 wait complete. 2024-09-21T05:40:03.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:03.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 -- 172.21.15.22:0/2005275997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a40b9d00 msgr2=0x7f72a40cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 --2- 172.21.15.22:0/2005275997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a40b9d00 0x7f72a40cfb40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7294005fd0 tx=0x7f729402aa10 comp rx=0 tx=0).stop 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 -- 172.21.15.22:0/2005275997 shutdown_connections 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 --2- 172.21.15.22:0/2005275997 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72a40b66b0 0x7f72a40d3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 --2- 172.21.15.22:0/2005275997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a40b9d00 0x7f72a40cfb40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 -- 172.21.15.22:0/2005275997 >> 172.21.15.22:0/2005275997 conn(0x7f72a409d870 msgr2=0x7f72a40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 -- 172.21.15.22:0/2005275997 shutdown_connections 2024-09-21T05:40:03.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.739+0000 7f72a8869640 1 -- 172.21.15.22:0/2005275997 wait complete. 2024-09-21T05:40:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.741+0000 7f72a8869640 1 Processor -- start 2024-09-21T05:40:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.741+0000 7f72a8869640 1 -- start start 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.742+0000 7f72a8869640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72a40b9d00 0x7f72a41e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.742+0000 7f72a8869640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72a41620e0 0x7f72a415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a8869640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 0x7f72a40ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a8869640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72a406e420 con 0x7f72a41f99b0 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a8869640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72a406e2a0 con 0x7f72a40b9d00 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a8869640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f72a406e5a0 con 0x7f72a41620e0 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 0x7f72a40ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:03.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 0x7f72a40ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36996/0 (socket says 172.21.15.22:36996) 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 -- 172.21.15.22:0/2678123133 learned_addr learned my addr 172.21.15.22:0/2678123133 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a1d9b640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72a41620e0 0x7f72a415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a259c640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72a40b9d00 0x7f72a41e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 -- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72a41620e0 msgr2=0x7f72a415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72a41620e0 0x7f72a415c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 -- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72a40b9d00 msgr2=0x7f72a41e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72a40b9d00 0x7f72a41e9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 -- 172.21.15.22:0/2678123133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a40ca060 con 0x7f72a41f99b0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a259c640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72a40b9d00 0x7f72a41e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a2d9d640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 0x7f72a40ca280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d5f0 tx=0x7f729c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c017070 con 0x7f72a41f99b0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f729c005330 con 0x7f72a41f99b0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a41f8f10 con 0x7f72a41f99b0 2024-09-21T05:40:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.743+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0047a0 con 0x7f72a41f99b0 2024-09-21T05:40:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.744+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a40c9d80 con 0x7f72a41f99b0 2024-09-21T05:40:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.744+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a40c9af0 con 0x7f72a41f99b0 2024-09-21T05:40:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.745+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f729c004940 con 0x7f72a41f99b0 2024-09-21T05:40:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.746+0000 7f7292ffd640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7284078bc0 0x7f728407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.746+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f729c09ac90 con 0x7f72a41f99b0 2024-09-21T05:40:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.746+0000 7f72a259c640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7284078bc0 0x7f728407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.746+0000 7f72a259c640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7284078bc0 0x7f728407b0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7294035000 tx=0x7f7294004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:03.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.751+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f729c062b10 con 0x7f72a41f99b0 2024-09-21T05:40:03.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.941+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f72a406eb30 con 0x7f72a41f99b0 2024-09-21T05:40:03.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.942+0000 7f7292ffd640 1 -- 172.21.15.22:0/2678123133 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16730 (secure 0 0 0) 0x7f729c067540 con 0x7f72a41f99b0 2024-09-21T05:40:03.944 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:03.944 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":31,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:02.941532+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:01.033773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:01.908775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:03.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7284078bc0 msgr2=0x7f728407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7284078bc0 0x7f728407b0d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7294035000 tx=0x7f7294004ab0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 msgr2=0x7f72a40ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:03.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 0x7f72a40ca280 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d5f0 tx=0x7f729c00dab0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 shutdown_connections 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72a41f99b0 0x7f72a40ca280 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7284078bc0 0x7f728407b0d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72a41620e0 0x7f72a415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 --2- 172.21.15.22:0/2678123133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72a40b9d00 0x7f72a41e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 >> 172.21.15.22:0/2678123133 conn(0x7f72a409d870 msgr2=0x7f72a40b5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 shutdown_connections 2024-09-21T05:40:03.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:03.944+0000 7f72a8869640 1 -- 172.21.15.22:0/2678123133 wait complete. 2024-09-21T05:40:03.958 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f71228a9640 1 -- 172.21.15.22:0/833633439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7104003a00 msgr2=0x7f7104003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f71228a9640 1 --2- 172.21.15.22:0/833633439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7104003a00 0x7f7104003e70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f710c009b50 tx=0x7f7104005830 comp rx=0 tx=0).stop 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f7120e46640 1 -- 172.21.15.22:0/833633439 reap_dead start 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f71228a9640 1 -- 172.21.15.22:0/833633439 shutdown_connections 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f71228a9640 1 -- 172.21.15.22:0/833633439 >> 172.21.15.22:0/833633439 conn(0x7f711c09d8f0 msgr2=0x7f711c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f71228a9640 1 -- 172.21.15.22:0/833633439 shutdown_connections 2024-09-21T05:40:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.072+0000 7f71228a9640 1 -- 172.21.15.22:0/833633439 wait complete. 2024-09-21T05:40:04.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.074+0000 7f71228a9640 1 Processor -- start 2024-09-21T05:40:04.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.074+0000 7f71228a9640 1 -- start start 2024-09-21T05:40:04.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f71228a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7104003a00 0x7f711c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f71228a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 0x7f711c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f71228a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f711c15c8a0 0x7f711c1f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f71228a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f711c163080 con 0x7f711c0c97b0 2024-09-21T05:40:04.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f71228a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f711c0d1fa0 con 0x7f711c15c8a0 2024-09-21T05:40:04.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f71228a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f711c163200 con 0x7f7104003a00 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 0x7f711c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 0x7f711c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:37022/0 (socket says 172.21.15.22:37022) 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 -- 172.21.15.22:0/1625511844 learned_addr learned my addr 172.21.15.22:0/1625511844 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f7120e46640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f711c15c8a0 0x7f711c1f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711bfff640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7104003a00 0x7f711c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 -- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7104003a00 msgr2=0x7f711c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7104003a00 0x7f711c0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 -- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f711c15c8a0 msgr2=0x7f711c1f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f711c15c8a0 0x7f711c1f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 -- 172.21.15.22:0/1625511844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c15ccb0 con 0x7f711c0c97b0 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f7120e46640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f711c15c8a0 0x7f711c1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711bfff640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7104003a00 0x7f711c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.076+0000 7f711b7fe640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 0x7f711c15c360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f711000e5c0 tx=0x7f711000ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.077+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110015070 con 0x7f711c0c97b0 2024-09-21T05:40:04.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.077+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7110010980 con 0x7f711c0c97b0 2024-09-21T05:40:04.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.077+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010d50 con 0x7f711c0c97b0 2024-09-21T05:40:04.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.077+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c1f96c0 con 0x7f711c0c97b0 2024-09-21T05:40:04.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.077+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1f9960 con 0x7f711c0c97b0 2024-09-21T05:40:04.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.078+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c0d11c0 con 0x7f711c0c97b0 2024-09-21T05:40:04.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.078+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7110018070 con 0x7f711c0c97b0 2024-09-21T05:40:04.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.079+0000 7f7118ff9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7108078bc0 0x7f710807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.079+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f711009b020 con 0x7f711c0c97b0 2024-09-21T05:40:04.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.079+0000 7f711bfff640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7108078bc0 0x7f710807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.080+0000 7f711bfff640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7108078bc0 0x7f710807b0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f710c009b50 tx=0x7f710c02d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:04.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.084+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f711c15ccb0 con 0x7f711c0c97b0 2024-09-21T05:40:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.275+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f711c0ba0d0 con 0x7f711c0c97b0 2024-09-21T05:40:04.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.276+0000 7f7118ff9640 1 -- 172.21.15.22:0/1625511844 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16730 (secure 0 0 0) 0x7f711c0d1900 con 0x7f711c0c97b0 2024-09-21T05:40:04.279 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:04.279 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":31,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:02.941532+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:01.033773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:01.908775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7108078bc0 msgr2=0x7f710807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7108078bc0 0x7f710807b0d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f710c009b50 tx=0x7f710c02d040 comp rx=0 tx=0).stop 2024-09-21T05:40:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 msgr2=0x7f711c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 0x7f711c15c360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f711000e5c0 tx=0x7f711000ea80 comp rx=0 tx=0).stop 2024-09-21T05:40:04.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 shutdown_connections 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f711c15c8a0 0x7f711c1f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711c0c97b0 0x7f711c15c360 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7108078bc0 0x7f710807b0d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 --2- 172.21.15.22:0/1625511844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7104003a00 0x7f711c0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 >> 172.21.15.22:0/1625511844 conn(0x7f711c09d8f0 msgr2=0x7f711c0b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 shutdown_connections 2024-09-21T05:40:04.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.279+0000 7f71228a9640 1 -- 172.21.15.22:0/1625511844 wait complete. 2024-09-21T05:40:04.292 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:40:04.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc349a64640 1 -- 172.21.15.22:0/3022989466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc32c003a00 msgr2=0x7fc32c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc349a64640 1 --2- 172.21.15.22:0/3022989466 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc32c003a00 0x7fc32c003e70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc334009b70 tx=0x7fc32c005830 comp rx=0 tx=0).stop 2024-09-21T05:40:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc343fff640 1 -- 172.21.15.22:0/3022989466 reap_dead start 2024-09-21T05:40:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc349a64640 1 -- 172.21.15.22:0/3022989466 shutdown_connections 2024-09-21T05:40:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc349a64640 1 -- 172.21.15.22:0/3022989466 >> 172.21.15.22:0/3022989466 conn(0x7fc34409d830 msgr2=0x7fc3440d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc349a64640 1 -- 172.21.15.22:0/3022989466 shutdown_connections 2024-09-21T05:40:04.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.410+0000 7fc349a64640 1 -- 172.21.15.22:0/3022989466 wait complete. 2024-09-21T05:40:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.412+0000 7fc349a64640 1 Processor -- start 2024-09-21T05:40:04.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.412+0000 7fc349a64640 1 -- start start 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.413+0000 7fc349a64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc34406f360 0x7fc34415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.413+0000 7fc349a64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 0x7fc3440b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.413+0000 7fc349a64640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3440b46f0 0x7fc3441f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.413+0000 7fc349a64640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3440d35f0 con 0x7fc3440b1190 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.413+0000 7fc349a64640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3440d3470 con 0x7fc34406f360 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.413+0000 7fc349a64640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3440d3770 con 0x7fc3440b46f0 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 0x7fc3440b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 0x7fc3440b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53580/0 (socket says 172.21.15.22:53580) 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 -- 172.21.15.22:0/1994168224 learned_addr learned my addr 172.21.15.22:0/1994168224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc3437fe640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc34406f360 0x7fc34415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc343fff640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3440b46f0 0x7fc3441f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 -- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3440b46f0 msgr2=0x7fc3441f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3440b46f0 0x7fc3441f8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 -- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc34406f360 msgr2=0x7fc34415edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc34406f360 0x7fc34415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 -- 172.21.15.22:0/1994168224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3440b2150 con 0x7fc3440b1190 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc342ffd640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 0x7fc3440b41b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc338011ae0 tx=0x7fc338009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc33801e070 con 0x7fc3440b1190 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc33800fe70 con 0x7fc3440b1190 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3440b2430 con 0x7fc3440b1190 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc338004290 con 0x7fc3440b1190 2024-09-21T05:40:04.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.414+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3440b29f0 con 0x7fc3440b1190 2024-09-21T05:40:04.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.415+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3440cfbe0 con 0x7fc3440b1190 2024-09-21T05:40:04.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.416+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc338021070 con 0x7fc3440b1190 2024-09-21T05:40:04.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.417+0000 7fc32bfff640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc330078b90 0x7fc33007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:04.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.417+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fc33809b720 con 0x7fc3440b1190 2024-09-21T05:40:04.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.417+0000 7fc3437fe640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc330078b90 0x7fc33007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:04.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.418+0000 7fc3437fe640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc330078b90 0x7fc33007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc334035000 tx=0x7fc33402d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:04.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.421+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc3380636f0 con 0x7fc3440b1190 2024-09-21T05:40:04.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:04.652+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc3440d1e80 con 0x7fc3440b1190 2024-09-21T05:40:05.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.586+0000 7fc32bfff640 1 -- 172.21.15.22:0/1994168224 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fc338068120 con 0x7fc3440b1190 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc330078b90 msgr2=0x7fc33007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc330078b90 0x7fc33007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc334035000 tx=0x7fc33402d040 comp rx=0 tx=0).stop 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 msgr2=0x7fc3440b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 0x7fc3440b41b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc338011ae0 tx=0x7fc338009e90 comp rx=0 tx=0).stop 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 shutdown_connections 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc330078b90 0x7fc33007b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3440b46f0 0x7fc3441f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3440b1190 0x7fc3440b41b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 --2- 172.21.15.22:0/1994168224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc34406f360 0x7fc34415edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 >> 172.21.15.22:0/1994168224 conn(0x7fc34409d830 msgr2=0x7fc344160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:05.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 shutdown_connections 2024-09-21T05:40:05.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.590+0000 7fc349a64640 1 -- 172.21.15.22:0/1994168224 wait complete. 2024-09-21T05:40:05.606 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:40:05.606 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 -- 172.21.15.22:0/1207257916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d0c70 msgr2=0x7f83180d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 --2- 172.21.15.22:0/1207257916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d0c70 0x7f83180d1080 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f830c009830 tx=0x7f830c02a510 comp rx=0 tx=0).stop 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 -- 172.21.15.22:0/1207257916 shutdown_connections 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 --2- 172.21.15.22:0/1207257916 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83180b4a50 0x7f83180b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 --2- 172.21.15.22:0/1207257916 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d0c70 0x7f83180d1080 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 -- 172.21.15.22:0/1207257916 >> 172.21.15.22:0/1207257916 conn(0x7f831809d880 msgr2=0x7f831808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 -- 172.21.15.22:0/1207257916 shutdown_connections 2024-09-21T05:40:05.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.731+0000 7f831cd20640 1 -- 172.21.15.22:0/1207257916 wait complete. 2024-09-21T05:40:05.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.733+0000 7f831cd20640 1 Processor -- start 2024-09-21T05:40:05.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.733+0000 7f831cd20640 1 -- start start 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831cd20640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83180b4a50 0x7f83180cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831cd20640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83180d0c70 0x7f831814e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831cd20640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 0x7f83180ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831cd20640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f83180b5520 con 0x7f83180d2430 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831cd20640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83180b53a0 con 0x7f83180d0c70 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831cd20640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83180b56a0 con 0x7f83180b4a50 2024-09-21T05:40:05.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 0x7f83180ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 0x7f83180ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53594/0 (socket says 172.21.15.22:53594) 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 -- 172.21.15.22:0/2700685165 learned_addr learned my addr 172.21.15.22:0/2700685165 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8315d9b640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83180d0c70 0x7f831814e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f831659c640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83180b4a50 0x7f83180cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 -- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83180b4a50 msgr2=0x7f83180cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83180b4a50 0x7f83180cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 -- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83180d0c70 msgr2=0x7f831814e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83180d0c70 0x7f831814e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 -- 172.21.15.22:0/2700685165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831814e870 con 0x7f83180d2430 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8315d9b640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83180d0c70 0x7f831814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8316d9d640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 0x7f83180ca450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f831000d5f0 tx=0x7f831000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310017070 con 0x7f83180d2430 2024-09-21T05:40:05.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.735+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8310005330 con 0x7f83180d2430 2024-09-21T05:40:05.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.736+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831815bc30 con 0x7f83180d2430 2024-09-21T05:40:05.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.736+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83100047a0 con 0x7f83180d2430 2024-09-21T05:40:05.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.736+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831815c1f0 con 0x7f83180d2430 2024-09-21T05:40:05.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.737+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83180b4e60 con 0x7f83180d2430 2024-09-21T05:40:05.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.737+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8310004940 con 0x7f83180d2430 2024-09-21T05:40:05.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.738+0000 7f8306ffd640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f82f8078b90 0x7f82f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:05.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.738+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f831009ab90 con 0x7f83180d2430 2024-09-21T05:40:05.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.738+0000 7f831659c640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f82f8078b90 0x7f82f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:05.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.739+0000 7f831659c640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f82f8078b90 0x7f82f807b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83080012e0 tx=0x7f8308009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:05.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.742+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8310062aa0 con 0x7f83180d2430 2024-09-21T05:40:05.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:05.975+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f831806e4e0 con 0x7f83180d2430 2024-09-21T05:40:06.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.592+0000 7f8306ffd640 1 -- 172.21.15.22:0/2700685165 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f83100674d0 con 0x7f83180d2430 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f82f8078b90 msgr2=0x7f82f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f82f8078b90 0x7f82f807b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83080012e0 tx=0x7f8308009040 comp rx=0 tx=0).stop 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 msgr2=0x7f83180ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 0x7f83180ca450 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f831000d5f0 tx=0x7f831000dab0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 shutdown_connections 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83180d2430 0x7f83180ca450 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83180d0c70 0x7f831814e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f82f8078b90 0x7f82f807b0a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 --2- 172.21.15.22:0/2700685165 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83180b4a50 0x7f83180cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 >> 172.21.15.22:0/2700685165 conn(0x7f831809d880 msgr2=0x7f83180d2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 shutdown_connections 2024-09-21T05:40:06.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.596+0000 7f831cd20640 1 -- 172.21.15.22:0/2700685165 wait complete. 2024-09-21T05:40:06.611 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:40:06.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.731+0000 7f5943cc5640 1 -- 172.21.15.22:0/3660055657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cf440 msgr2=0x7f593c15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.731+0000 7f5943cc5640 1 --2- 172.21.15.22:0/3660055657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cf440 0x7f593c15c6a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f593c0cc8f0 tx=0x7f592c02a560 comp rx=0 tx=0).stop 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.732+0000 7f5943cc5640 1 -- 172.21.15.22:0/3660055657 shutdown_connections 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.732+0000 7f5943cc5640 1 --2- 172.21.15.22:0/3660055657 >> v2:172.21.15.134:3300/0 conn(0x7f593c0b7fe0 0x7f593c0d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.732+0000 7f5943cc5640 1 --2- 172.21.15.22:0/3660055657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cf440 0x7f593c15c6a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.732+0000 7f5943cc5640 1 -- 172.21.15.22:0/3660055657 >> 172.21.15.22:0/3660055657 conn(0x7f593c0710b0 msgr2=0x7f593c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.732+0000 7f5943cc5640 1 -- 172.21.15.22:0/3660055657 shutdown_connections 2024-09-21T05:40:06.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.732+0000 7f5943cc5640 1 -- 172.21.15.22:0/3660055657 wait complete. 2024-09-21T05:40:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.733+0000 7f5943cc5640 1 Processor -- start 2024-09-21T05:40:06.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.733+0000 7f5943cc5640 1 -- start start 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5943cc5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f593c0b7fe0 0x7f593c0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5943cc5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 0x7f593c1476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5943cc5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f593c157530 0x7f593c157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5943cc5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0d0b40 con 0x7f593c0cd360 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5943cc5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f593c0d09c0 con 0x7f593c0b7fe0 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5943cc5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f593c0d0cc0 con 0x7f593c157530 2024-09-21T05:40:06.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 0x7f593c1476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 0x7f593c1476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53616/0 (socket says 172.21.15.22:53616) 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 -- 172.21.15.22:0/1963651414 learned_addr learned my addr 172.21.15.22:0/1963651414 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941a61640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f593c0b7fe0 0x7f593c0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5942262640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f593c157530 0x7f593c157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 -- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f593c157530 msgr2=0x7f593c157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f593c157530 0x7f593c157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 -- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f593c0b7fe0 msgr2=0x7f593c0c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f593c0b7fe0 0x7f593c0c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 -- 172.21.15.22:0/1963651414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c147c30 con 0x7f593c0cd360 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941a61640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f593c0b7fe0 0x7f593c0c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5942262640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f593c157530 0x7f593c157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.735+0000 7f5941260640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 0x7f593c1476f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f592c034040 tx=0x7f592c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.736+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c03e030 con 0x7f593c0cd360 2024-09-21T05:40:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.736+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592c003db0 con 0x7f593c0cd360 2024-09-21T05:40:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.736+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c035040 con 0x7f593c0cd360 2024-09-21T05:40:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.736+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c0c91e0 con 0x7f593c0cd360 2024-09-21T05:40:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.736+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c0c97a0 con 0x7f593c0cd360 2024-09-21T05:40:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.737+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593c158010 con 0x7f593c0cd360 2024-09-21T05:40:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.738+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f592c041030 con 0x7f593c0cd360 2024-09-21T05:40:06.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.738+0000 7f59327fc640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5920078b90 0x7f592007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.738+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f592c0bb8e0 con 0x7f593c0cd360 2024-09-21T05:40:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.739+0000 7f5941a61640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5920078b90 0x7f592007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:06.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.739+0000 7f5941a61640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5920078b90 0x7f592007b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5938007fd0 tx=0x7f5938009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:06.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.742+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f592c083f20 con 0x7f593c0cd360 2024-09-21T05:40:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.966+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f593c06a5e0 con 0x7f593c0cd360 2024-09-21T05:40:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.966+0000 7f59327fc640 1 -- 172.21.15.22:0/1963651414 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f593c06a5e0 con 0x7f593c0cd360 2024-09-21T05:40:06.968 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:06.968 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:06.969 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5920078b90 msgr2=0x7f592007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5920078b90 0x7f592007b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5938007fd0 tx=0x7f5938009040 comp rx=0 tx=0).stop 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 msgr2=0x7f593c1476f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 0x7f593c1476f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f592c034040 tx=0x7f592c0038a0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 shutdown_connections 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5920078b90 0x7f592007b0a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f593c157530 0x7f593c157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f593c0cd360 0x7f593c1476f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 --2- 172.21.15.22:0/1963651414 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f593c0b7fe0 0x7f593c0c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:06.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 >> 172.21.15.22:0/1963651414 conn(0x7f593c0710b0 msgr2=0x7f593c0b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:06.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 shutdown_connections 2024-09-21T05:40:06.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:06.969+0000 7f5943cc5640 1 -- 172.21.15.22:0/1963651414 wait complete. 2024-09-21T05:40:06.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:07.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/2372299525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc806e3c0 msgr2=0x7f0cc806f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/2372299525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc806e3c0 0x7f0cc806f7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0cc808c950 tx=0x7f0cbc02a560 comp rx=0 tx=0).stop 2024-09-21T05:40:07.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/2372299525 shutdown_connections 2024-09-21T05:40:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/2372299525 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cc80d0e10 0x7f0cc80d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/2372299525 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc806e3c0 0x7f0cc806f7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/2372299525 >> v2:172.21.15.134:3300/0 conn(0x7f0cc80b3a60 0x7f0cc80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/2372299525 >> 172.21.15.22:0/2372299525 conn(0x7f0cc808bfc0 msgr2=0x7f0cc809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/2372299525 shutdown_connections 2024-09-21T05:40:07.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.096+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/2372299525 wait complete. 2024-09-21T05:40:07.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.098+0000 7f0ccd7f1640 1 Processor -- start 2024-09-21T05:40:07.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.098+0000 7f0ccd7f1640 1 -- start start 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.099+0000 7f0ccd7f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cc806e3c0 0x7f0cc815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.099+0000 7f0ccd7f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cc80b3a60 0x7f0cc81ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0ccd7f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 0x7f0cc81eada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0ccd7f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80d2ad0 con 0x7f0cc80d0e10 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0ccd7f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc80d2950 con 0x7f0cc806e3c0 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0ccd7f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80d2c50 con 0x7f0cc80b3a60 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 0x7f0cc81eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 0x7f0cc81eada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53634/0 (socket says 172.21.15.22:53634) 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 -- 172.21.15.22:0/1281337900 learned_addr learned my addr 172.21.15.22:0/1281337900 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc6ffd640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cc806e3c0 0x7f0cc815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc67fc640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cc80b3a60 0x7f0cc81ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 -- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cc80b3a60 msgr2=0x7f0cc81ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cc80b3a60 0x7f0cc81ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 -- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cc806e3c0 msgr2=0x7f0cc815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cc806e3c0 0x7f0cc815d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 -- 172.21.15.22:0/1281337900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc81f6960 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc6ffd640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cc806e3c0 0x7f0cc815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0cc77fe640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 0x7f0cc81eada0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000ea40 tx=0x7f0cc000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc000ce40 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cc0004c10 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc81f5950 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc0005020 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.100+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc81f5e80 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.102+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc80d31e0 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.102+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0cc00051c0 con 0x7f0cc80d0e10 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.103+0000 7f0caf7fe640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cb4078b90 0x7f0cb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.103+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0cc009bc40 con 0x7f0cc80d0e10 2024-09-21T05:40:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.103+0000 7f0cc6ffd640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cb4078b90 0x7f0cb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.104+0000 7f0cc6ffd640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cb4078b90 0x7f0cb407b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009b00 tx=0x7f0cb8005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.107+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0cc0064080 con 0x7f0cc80d0e10 2024-09-21T05:40:07.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.296+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cc80a20b0 con 0x7f0cc80d0e10 2024-09-21T05:40:07.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.298+0000 7f0caf7fe640 1 -- 172.21.15.22:0/1281337900 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16717 (secure 0 0 0) 0x7f0cc0063980 con 0x7f0cc80d0e10 2024-09-21T05:40:07.300 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:07.300 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":32,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:04.576394+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:01.033773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:01.908775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:07.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cb4078b90 msgr2=0x7f0cb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cb4078b90 0x7f0cb407b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8009b00 tx=0x7f0cb8005f90 comp rx=0 tx=0).stop 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 msgr2=0x7f0cc81eada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 0x7f0cc81eada0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0cc000ea40 tx=0x7f0cc000ef00 comp rx=0 tx=0).stop 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 shutdown_connections 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0cc80d0e10 0x7f0cc81eada0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cb4078b90 0x7f0cb407b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0cc80b3a60 0x7f0cc81ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 --2- 172.21.15.22:0/1281337900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0cc806e3c0 0x7f0cc815d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 >> 172.21.15.22:0/1281337900 conn(0x7f0cc808bfc0 msgr2=0x7f0cc80cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 shutdown_connections 2024-09-21T05:40:07.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.300+0000 7f0ccd7f1640 1 -- 172.21.15.22:0/1281337900 wait complete. 2024-09-21T05:40:07.314 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:40:07.314 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:40:07.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 -- 172.21.15.22:0/1040292851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 msgr2=0x7fabac06f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 --2- 172.21.15.22:0/1040292851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac06f6e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faba0005fd0 tx=0x7faba002a560 comp rx=0 tx=0).stop 2024-09-21T05:40:07.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 -- 172.21.15.22:0/1040292851 shutdown_connections 2024-09-21T05:40:07.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 --2- 172.21.15.22:0/1040292851 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac06f6e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 -- 172.21.15.22:0/1040292851 >> 172.21.15.22:0/1040292851 conn(0x7fabac09d840 msgr2=0x7fabac08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:07.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 -- 172.21.15.22:0/1040292851 shutdown_connections 2024-09-21T05:40:07.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.431+0000 7fabb14dc640 1 -- 172.21.15.22:0/1040292851 wait complete. 2024-09-21T05:40:07.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.433+0000 7fabb14dc640 1 Processor -- start 2024-09-21T05:40:07.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.433+0000 7fabb14dc640 1 -- start start 2024-09-21T05:40:07.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabb14dc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabac06e0a0 0x7fabac1512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabb14dc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabb14dc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabac0d0e50 0x7fabac154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabb14dc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0cff00 con 0x7fabac0b97e0 2024-09-21T05:40:07.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabb14dc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabac0cfd80 con 0x7fabac06e0a0 2024-09-21T05:40:07.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabb14dc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fabac0d0080 con 0x7fabac0d0e50 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53642/0 (socket says 172.21.15.22:53642) 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 -- 172.21.15.22:0/4034958540 learned_addr learned my addr 172.21.15.22:0/4034958540 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaaffd640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabac06e0a0 0x7fabac1512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabab7fe640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabac0d0e50 0x7fabac154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 -- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabac0d0e50 msgr2=0x7fabac154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabac0d0e50 0x7fabac154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 -- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabac06e0a0 msgr2=0x7fabac1512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabac06e0a0 0x7fabac1512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.434+0000 7fabaa7fc640 1 -- 172.21.15.22:0/4034958540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabac160610 con 0x7fabac0b97e0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.435+0000 7fabaa7fc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac153f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faba0034040 tx=0x7faba0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.435+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba003e030 con 0x7fabac0b97e0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.435+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faba0003e00 con 0x7fabac0b97e0 2024-09-21T05:40:07.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.435+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabac0b4ba0 con 0x7fabac0b97e0 2024-09-21T05:40:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.435+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0035040 con 0x7fabac0b97e0 2024-09-21T05:40:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.435+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabac0b5160 con 0x7fabac0b97e0 2024-09-21T05:40:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.436+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabac0d0430 con 0x7fabac0b97e0 2024-09-21T05:40:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.437+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faba0041030 con 0x7fabac0b97e0 2024-09-21T05:40:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.437+0000 7fab937fe640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab98078b90 0x7fab9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.437+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7faba00bb5d0 con 0x7fabac0b97e0 2024-09-21T05:40:07.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.437+0000 7fabaaffd640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab98078b90 0x7fab9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.438+0000 7fabaaffd640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab98078b90 0x7fab9807b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0016b0 tx=0x7fab9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:07.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.442+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faba00833c0 con 0x7fabac0b97e0 2024-09-21T05:40:07.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.628+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabac0cf580 con 0x7fab98078b90 2024-09-21T05:40:07.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.633+0000 7fab937fe640 1 -- 172.21.15.22:0/4034958540 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247789 (secure 0 0 0) 0x7fabac0cf580 con 0x7fab98078b90 2024-09-21T05:40:07.637 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:07.639 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab98078b90 msgr2=0x7fab9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab98078b90 0x7fab9807b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0016b0 tx=0x7fab9c009040 comp rx=0 tx=0).stop 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 msgr2=0x7fabac153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac153f20 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faba0034040 tx=0x7faba0003910 comp rx=0 tx=0).stop 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 shutdown_connections 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab98078b90 0x7fab9807b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabac0d0e50 0x7fabac154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabac0b97e0 0x7fabac153f20 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 --2- 172.21.15.22:0/4034958540 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabac06e0a0 0x7fabac1512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 >> 172.21.15.22:0/4034958540 conn(0x7fabac09d840 msgr2=0x7fabac0b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:07.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 shutdown_connections 2024-09-21T05:40:07.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.636+0000 7fabb14dc640 1 -- 172.21.15.22:0/4034958540 wait complete. 2024-09-21T05:40:07.649 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-09-21T05:40:05.746812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":139,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104,"ondisk_log_size":104,"up":138,"acting":138,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":43,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":240900,"kb_used_data":3844,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733820,"statfs":{"total":773094113280,"available":772847431680,"internally_reserved":0,"allocated":3936256,"data_stored":2216212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.188170"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585391+0000","last_change":"2024-09-21T05:40:03.591951+0000","last_active":"2024-09-21T05:40:04.585391+0000","last_peered":"2024-09-21T05:40:04.585391+0000","last_clean":"2024-09-21T05:40:04.585391+0000","last_became_active":"2024-09-21T05:40:03.591582+0000","last_became_peered":"2024-09-21T05:40:03.591582+0000","last_unstale":"2024-09-21T05:40:04.585391+0000","last_undegraded":"2024-09-21T05:40:04.585391+0000","last_fullsized":"2024-09-21T05:40:04.585391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:42.963566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245090+0000","last_change":"2024-09-21T05:40:03.588128+0000","last_active":"2024-09-21T05:40:05.245090+0000","last_peered":"2024-09-21T05:40:05.245090+0000","last_clean":"2024-09-21T05:40:05.245090+0000","last_became_active":"2024-09-21T05:40:03.587771+0000","last_became_peered":"2024-09-21T05:40:03.587771+0000","last_unstale":"2024-09-21T05:40:05.245090+0000","last_undegraded":"2024-09-21T05:40:05.245090+0000","last_fullsized":"2024-09-21T05:40:05.245090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:56.242972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245367+0000","last_change":"2024-09-21T05:40:03.597254+0000","last_active":"2024-09-21T05:40:05.245367+0000","last_peered":"2024-09-21T05:40:05.245367+0000","last_clean":"2024-09-21T05:40:05.245367+0000","last_became_active":"2024-09-21T05:40:03.596911+0000","last_became_peered":"2024-09-21T05:40:03.596911+0000","last_unstale":"2024-09-21T05:40:05.245367+0000","last_undegraded":"2024-09-21T05:40:05.245367+0000","last_fullsized":"2024-09-21T05:40:05.245367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:56.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583607+0000","last_change":"2024-09-21T05:40:03.580710+0000","last_active":"2024-09-21T05:40:04.583607+0000","last_peered":"2024-09-21T05:40:04.583607+0000","last_clean":"2024-09-21T05:40:04.583607+0000","last_became_active":"2024-09-21T05:40:03.578193+0000","last_became_peered":"2024-09-21T05:40:03.578193+0000","last_unstale":"2024-09-21T05:40:04.583607+0000","last_undegraded":"2024-09-21T05:40:04.583607+0000","last_fullsized":"2024-09-21T05:40:04.583607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:12.963462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163380+0000","last_change":"2024-09-21T05:40:03.583377+0000","last_active":"2024-09-21T05:40:05.163380+0000","last_peered":"2024-09-21T05:40:05.163380+0000","last_clean":"2024-09-21T05:40:05.163380+0000","last_became_active":"2024-09-21T05:40:03.582743+0000","last_became_peered":"2024-09-21T05:40:03.582743+0000","last_unstale":"2024-09-21T05:40:05.163380+0000","last_undegraded":"2024-09-21T05:40:05.163380+0000","last_fullsized":"2024-09-21T05:40:05.163380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:37.774264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244660+0000","last_change":"2024-09-21T05:40:03.594538+0000","last_active":"2024-09-21T05:40:05.244660+0000","last_peered":"2024-09-21T05:40:05.244660+0000","last_clean":"2024-09-21T05:40:05.244660+0000","last_became_active":"2024-09-21T05:40:03.593800+0000","last_became_peered":"2024-09-21T05:40:03.593800+0000","last_unstale":"2024-09-21T05:40:05.244660+0000","last_undegraded":"2024-09-21T05:40:05.244660+0000","last_fullsized":"2024-09-21T05:40:05.244660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:48.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583924+0000","last_change":"2024-09-21T05:40:03.582035+0000","last_active":"2024-09-21T05:40:04.583924+0000","last_peered":"2024-09-21T05:40:04.583924+0000","last_clean":"2024-09-21T05:40:04.583924+0000","last_became_active":"2024-09-21T05:40:03.581662+0000","last_became_peered":"2024-09-21T05:40:03.581662+0000","last_unstale":"2024-09-21T05:40:04.583924+0000","last_undegraded":"2024-09-21T05:40:04.583924+0000","last_fullsized":"2024-09-21T05:40:04.583924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:46.211686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245189+0000","last_change":"2024-09-21T05:40:03.595910+0000","last_active":"2024-09-21T05:40:05.245189+0000","last_peered":"2024-09-21T05:40:05.245189+0000","last_clean":"2024-09-21T05:40:05.245189+0000","last_became_active":"2024-09-21T05:40:03.595184+0000","last_became_peered":"2024-09-21T05:40:03.595184+0000","last_unstale":"2024-09-21T05:40:05.245189+0000","last_undegraded":"2024-09-21T05:40:05.245189+0000","last_fullsized":"2024-09-21T05:40:05.245189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:38.803584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163421+0000","last_change":"2024-09-21T05:40:03.585159+0000","last_active":"2024-09-21T05:40:05.163421+0000","last_peered":"2024-09-21T05:40:05.163421+0000","last_clean":"2024-09-21T05:40:05.163421+0000","last_became_active":"2024-09-21T05:40:03.584369+0000","last_became_peered":"2024-09-21T05:40:03.584369+0000","last_unstale":"2024-09-21T05:40:05.163421+0000","last_undegraded":"2024-09-21T05:40:05.163421+0000","last_fullsized":"2024-09-21T05:40:05.163421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:23.359967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244592+0000","last_change":"2024-09-21T05:40:03.594146+0000","last_active":"2024-09-21T05:40:05.244592+0000","last_peered":"2024-09-21T05:40:05.244592+0000","last_clean":"2024-09-21T05:40:05.244592+0000","last_became_active":"2024-09-21T05:40:03.593367+0000","last_became_peered":"2024-09-21T05:40:03.593367+0000","last_unstale":"2024-09-21T05:40:05.244592+0000","last_undegraded":"2024-09-21T05:40:05.244592+0000","last_fullsized":"2024-09-21T05:40:05.244592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.401173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244494+0000","last_change":"2024-09-21T05:40:03.593210+0000","last_active":"2024-09-21T05:40:05.244494+0000","last_peered":"2024-09-21T05:40:05.244494+0000","last_clean":"2024-09-21T05:40:05.244494+0000","last_became_active":"2024-09-21T05:40:03.592872+0000","last_became_peered":"2024-09-21T05:40:03.592872+0000","last_unstale":"2024-09-21T05:40:05.244494+0000","last_undegraded":"2024-09-21T05:40:05.244494+0000","last_fullsized":"2024-09-21T05:40:05.244494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:21.107732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245145+0000","last_change":"2024-09-21T05:40:03.584578+0000","last_active":"2024-09-21T05:40:05.245145+0000","last_peered":"2024-09-21T05:40:05.245145+0000","last_clean":"2024-09-21T05:40:05.245145+0000","last_became_active":"2024-09-21T05:40:03.584009+0000","last_became_peered":"2024-09-21T05:40:03.584009+0000","last_unstale":"2024-09-21T05:40:05.245145+0000","last_undegraded":"2024-09-21T05:40:05.245145+0000","last_fullsized":"2024-09-21T05:40:05.245145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:41.244043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244890+0000","last_change":"2024-09-21T05:40:03.584102+0000","last_active":"2024-09-21T05:40:05.244890+0000","last_peered":"2024-09-21T05:40:05.244890+0000","last_clean":"2024-09-21T05:40:05.244890+0000","last_became_active":"2024-09-21T05:40:03.583776+0000","last_became_peered":"2024-09-21T05:40:03.583776+0000","last_unstale":"2024-09-21T05:40:05.244890+0000","last_undegraded":"2024-09-21T05:40:05.244890+0000","last_fullsized":"2024-09-21T05:40:05.244890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:55.295143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163648+0000","last_change":"2024-09-21T05:40:03.595491+0000","last_active":"2024-09-21T05:40:05.163648+0000","last_peered":"2024-09-21T05:40:05.163648+0000","last_clean":"2024-09-21T05:40:05.163648+0000","last_became_active":"2024-09-21T05:40:03.595062+0000","last_became_peered":"2024-09-21T05:40:03.595062+0000","last_unstale":"2024-09-21T05:40:05.163648+0000","last_undegraded":"2024-09-21T05:40:05.163648+0000","last_fullsized":"2024-09-21T05:40:05.163648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:41.038623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587668+0000","last_change":"2024-09-21T05:40:03.591516+0000","last_active":"2024-09-21T05:40:04.587668+0000","last_peered":"2024-09-21T05:40:04.587668+0000","last_clean":"2024-09-21T05:40:04.587668+0000","last_became_active":"2024-09-21T05:40:03.591036+0000","last_became_peered":"2024-09-21T05:40:03.591036+0000","last_unstale":"2024-09-21T05:40:04.587668+0000","last_undegraded":"2024-09-21T05:40:04.587668+0000","last_fullsized":"2024-09-21T05:40:04.587668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:42.963566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244323+0000","last_change":"2024-09-21T05:40:03.597051+0000","last_active":"2024-09-21T05:40:05.244323+0000","last_peered":"2024-09-21T05:40:05.244323+0000","last_clean":"2024-09-21T05:40:05.244323+0000","last_became_active":"2024-09-21T05:40:03.596560+0000","last_became_peered":"2024-09-21T05:40:03.596560+0000","last_unstale":"2024-09-21T05:40:05.244323+0000","last_undegraded":"2024-09-21T05:40:05.244323+0000","last_fullsized":"2024-09-21T05:40:05.244323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:53.219876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584169+0000","last_change":"2024-09-21T05:40:03.587689+0000","last_active":"2024-09-21T05:40:04.584169+0000","last_peered":"2024-09-21T05:40:04.584169+0000","last_clean":"2024-09-21T05:40:04.584169+0000","last_became_active":"2024-09-21T05:40:03.587347+0000","last_became_peered":"2024-09-21T05:40:03.587347+0000","last_unstale":"2024-09-21T05:40:04.584169+0000","last_undegraded":"2024-09-21T05:40:04.584169+0000","last_fullsized":"2024-09-21T05:40:04.584169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:32.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245235+0000","last_change":"2024-09-21T05:40:03.587262+0000","last_active":"2024-09-21T05:40:05.245235+0000","last_peered":"2024-09-21T05:40:05.245235+0000","last_clean":"2024-09-21T05:40:05.245235+0000","last_became_active":"2024-09-21T05:40:03.586326+0000","last_became_peered":"2024-09-21T05:40:03.586326+0000","last_unstale":"2024-09-21T05:40:05.245235+0000","last_undegraded":"2024-09-21T05:40:05.245235+0000","last_fullsized":"2024-09-21T05:40:05.245235+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:08.160006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583927+0000","last_change":"2024-09-21T05:40:03.578020+0000","last_active":"2024-09-21T05:40:04.583927+0000","last_peered":"2024-09-21T05:40:04.583927+0000","last_clean":"2024-09-21T05:40:04.583927+0000","last_became_active":"2024-09-21T05:40:03.577015+0000","last_became_peered":"2024-09-21T05:40:03.577015+0000","last_unstale":"2024-09-21T05:40:04.583927+0000","last_undegraded":"2024-09-21T05:40:04.583927+0000","last_fullsized":"2024-09-21T05:40:04.583927+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:30.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245401+0000","last_change":"2024-09-21T05:40:03.596127+0000","last_active":"2024-09-21T05:40:05.245401+0000","last_peered":"2024-09-21T05:40:05.245401+0000","last_clean":"2024-09-21T05:40:05.245401+0000","last_became_active":"2024-09-21T05:40:03.595710+0000","last_became_peered":"2024-09-21T05:40:03.595710+0000","last_unstale":"2024-09-21T05:40:05.245401+0000","last_undegraded":"2024-09-21T05:40:05.245401+0000","last_fullsized":"2024-09-21T05:40:05.245401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:08.055220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584738+0000","last_change":"2024-09-21T05:40:03.577513+0000","last_active":"2024-09-21T05:40:04.584738+0000","last_peered":"2024-09-21T05:40:04.584738+0000","last_clean":"2024-09-21T05:40:04.584738+0000","last_became_active":"2024-09-21T05:40:03.577197+0000","last_became_peered":"2024-09-21T05:40:03.577197+0000","last_unstale":"2024-09-21T05:40:04.584738+0000","last_undegraded":"2024-09-21T05:40:04.584738+0000","last_fullsized":"2024-09-21T05:40:04.584738+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:34.165185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587871+0000","last_change":"2024-09-21T05:40:03.592754+0000","last_active":"2024-09-21T05:40:04.587871+0000","last_peered":"2024-09-21T05:40:04.587871+0000","last_clean":"2024-09-21T05:40:04.587871+0000","last_became_active":"2024-09-21T05:40:03.592183+0000","last_became_peered":"2024-09-21T05:40:03.592183+0000","last_unstale":"2024-09-21T05:40:04.587871+0000","last_undegraded":"2024-09-21T05:40:04.587871+0000","last_fullsized":"2024-09-21T05:40:04.587871+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:30.497627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585534+0000","last_change":"2024-09-21T05:40:02.581436+0000","last_active":"2024-09-21T05:40:04.585534+0000","last_peered":"2024-09-21T05:40:04.585534+0000","last_clean":"2024-09-21T05:40:04.585534+0000","last_became_active":"2024-09-21T05:40:02.581120+0000","last_became_peered":"2024-09-21T05:40:02.581120+0000","last_unstale":"2024-09-21T05:40:04.585534+0000","last_undegraded":"2024-09-21T05:40:04.585534+0000","last_fullsized":"2024-09-21T05:40:04.585534+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:16.673104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585463+0000","last_change":"2024-09-21T05:40:03.574748+0000","last_active":"2024-09-21T05:40:04.585463+0000","last_peered":"2024-09-21T05:40:04.585463+0000","last_clean":"2024-09-21T05:40:04.585463+0000","last_became_active":"2024-09-21T05:40:03.574374+0000","last_became_peered":"2024-09-21T05:40:03.574374+0000","last_unstale":"2024-09-21T05:40:04.585463+0000","last_undegraded":"2024-09-21T05:40:04.585463+0000","last_fullsized":"2024-09-21T05:40:04.585463+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.176357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587751+0000","last_change":"2024-09-21T05:40:02.587449+0000","last_active":"2024-09-21T05:40:04.587751+0000","last_peered":"2024-09-21T05:40:04.587751+0000","last_clean":"2024-09-21T05:40:04.587751+0000","last_became_active":"2024-09-21T05:40:02.587151+0000","last_became_peered":"2024-09-21T05:40:02.587151+0000","last_unstale":"2024-09-21T05:40:04.587751+0000","last_undegraded":"2024-09-21T05:40:04.587751+0000","last_fullsized":"2024-09-21T05:40:04.587751+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:34.268429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244786+0000","last_change":"2024-09-21T05:40:02.581247+0000","last_active":"2024-09-21T05:40:05.244786+0000","last_peered":"2024-09-21T05:40:05.244786+0000","last_clean":"2024-09-21T05:40:05.244786+0000","last_became_active":"2024-09-21T05:40:02.580490+0000","last_became_peered":"2024-09-21T05:40:02.580490+0000","last_unstale":"2024-09-21T05:40:05.244786+0000","last_undegraded":"2024-09-21T05:40:05.244786+0000","last_fullsized":"2024-09-21T05:40:05.244786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:01.905529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584010+0000","last_change":"2024-09-21T05:40:03.576536+0000","last_active":"2024-09-21T05:40:04.584010+0000","last_peered":"2024-09-21T05:40:04.584010+0000","last_clean":"2024-09-21T05:40:04.584010+0000","last_became_active":"2024-09-21T05:40:03.576157+0000","last_became_peered":"2024-09-21T05:40:03.576157+0000","last_unstale":"2024-09-21T05:40:04.584010+0000","last_undegraded":"2024-09-21T05:40:04.584010+0000","last_fullsized":"2024-09-21T05:40:04.584010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:58.586041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163515+0000","last_change":"2024-09-21T05:40:02.576102+0000","last_active":"2024-09-21T05:40:05.163515+0000","last_peered":"2024-09-21T05:40:05.163515+0000","last_clean":"2024-09-21T05:40:05.163515+0000","last_became_active":"2024-09-21T05:40:02.575795+0000","last_became_peered":"2024-09-21T05:40:02.575795+0000","last_unstale":"2024-09-21T05:40:05.163515+0000","last_undegraded":"2024-09-21T05:40:05.163515+0000","last_fullsized":"2024-09-21T05:40:05.163515+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:43.899292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245147+0000","last_change":"2024-09-21T05:40:03.593834+0000","last_active":"2024-09-21T05:40:05.245147+0000","last_peered":"2024-09-21T05:40:05.245147+0000","last_clean":"2024-09-21T05:40:05.245147+0000","last_became_active":"2024-09-21T05:40:03.593476+0000","last_became_peered":"2024-09-21T05:40:03.593476+0000","last_unstale":"2024-09-21T05:40:05.245147+0000","last_undegraded":"2024-09-21T05:40:05.245147+0000","last_fullsized":"2024-09-21T05:40:05.245147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:36.095605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244292+0000","last_change":"2024-09-21T05:40:02.582062+0000","last_active":"2024-09-21T05:40:05.244292+0000","last_peered":"2024-09-21T05:40:05.244292+0000","last_clean":"2024-09-21T05:40:05.244292+0000","last_became_active":"2024-09-21T05:40:02.581737+0000","last_became_peered":"2024-09-21T05:40:02.581737+0000","last_unstale":"2024-09-21T05:40:05.244292+0000","last_undegraded":"2024-09-21T05:40:05.244292+0000","last_fullsized":"2024-09-21T05:40:05.244292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:09.507441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583468+0000","last_change":"2024-09-21T05:40:03.598467+0000","last_active":"2024-09-21T05:40:04.583468+0000","last_peered":"2024-09-21T05:40:04.583468+0000","last_clean":"2024-09-21T05:40:04.583468+0000","last_became_active":"2024-09-21T05:40:03.598054+0000","last_became_peered":"2024-09-21T05:40:03.598054+0000","last_unstale":"2024-09-21T05:40:04.583468+0000","last_undegraded":"2024-09-21T05:40:04.583468+0000","last_fullsized":"2024-09-21T05:40:04.583468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:57.683487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245499+0000","last_change":"2024-09-21T05:40:03.595491+0000","last_active":"2024-09-21T05:40:05.245499+0000","last_peered":"2024-09-21T05:40:05.245499+0000","last_clean":"2024-09-21T05:40:05.245499+0000","last_became_active":"2024-09-21T05:40:03.595148+0000","last_became_peered":"2024-09-21T05:40:03.595148+0000","last_unstale":"2024-09-21T05:40:05.245499+0000","last_undegraded":"2024-09-21T05:40:05.245499+0000","last_fullsized":"2024-09-21T05:40:05.245499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:16.922990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245032+0000","last_change":"2024-09-21T05:40:02.587265+0000","last_active":"2024-09-21T05:40:05.245032+0000","last_peered":"2024-09-21T05:40:05.245032+0000","last_clean":"2024-09-21T05:40:05.245032+0000","last_became_active":"2024-09-21T05:40:02.586947+0000","last_became_peered":"2024-09-21T05:40:02.586947+0000","last_unstale":"2024-09-21T05:40:05.245032+0000","last_undegraded":"2024-09-21T05:40:05.245032+0000","last_fullsized":"2024-09-21T05:40:05.245032+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:04.995697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245051+0000","last_change":"2024-09-21T05:40:03.583507+0000","last_active":"2024-09-21T05:40:05.245051+0000","last_peered":"2024-09-21T05:40:05.245051+0000","last_clean":"2024-09-21T05:40:05.245051+0000","last_became_active":"2024-09-21T05:40:03.583188+0000","last_became_peered":"2024-09-21T05:40:03.583188+0000","last_unstale":"2024-09-21T05:40:05.245051+0000","last_undegraded":"2024-09-21T05:40:05.245051+0000","last_fullsized":"2024-09-21T05:40:05.245051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:07.340885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587892+0000","last_change":"2024-09-21T05:40:02.585079+0000","last_active":"2024-09-21T05:40:04.587892+0000","last_peered":"2024-09-21T05:40:04.587892+0000","last_clean":"2024-09-21T05:40:04.587892+0000","last_became_active":"2024-09-21T05:40:02.584690+0000","last_became_peered":"2024-09-21T05:40:02.584690+0000","last_unstale":"2024-09-21T05:40:04.587892+0000","last_undegraded":"2024-09-21T05:40:04.587892+0000","last_fullsized":"2024-09-21T05:40:04.587892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:10.788740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244961+0000","last_change":"2024-09-21T05:40:03.585755+0000","last_active":"2024-09-21T05:40:05.244961+0000","last_peered":"2024-09-21T05:40:05.244961+0000","last_clean":"2024-09-21T05:40:05.244961+0000","last_became_active":"2024-09-21T05:40:03.585409+0000","last_became_peered":"2024-09-21T05:40:03.585409+0000","last_unstale":"2024-09-21T05:40:05.244961+0000","last_undegraded":"2024-09-21T05:40:05.244961+0000","last_fullsized":"2024-09-21T05:40:05.244961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:21.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245521+0000","last_change":"2024-09-21T05:40:02.587490+0000","last_active":"2024-09-21T05:40:05.245521+0000","last_peered":"2024-09-21T05:40:05.245521+0000","last_clean":"2024-09-21T05:40:05.245521+0000","last_became_active":"2024-09-21T05:40:02.587151+0000","last_became_peered":"2024-09-21T05:40:02.587151+0000","last_unstale":"2024-09-21T05:40:05.245521+0000","last_undegraded":"2024-09-21T05:40:05.245521+0000","last_fullsized":"2024-09-21T05:40:05.245521+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.951231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245012+0000","last_change":"2024-09-21T05:40:03.597363+0000","last_active":"2024-09-21T05:40:05.245012+0000","last_peered":"2024-09-21T05:40:05.245012+0000","last_clean":"2024-09-21T05:40:05.245012+0000","last_became_active":"2024-09-21T05:40:03.596922+0000","last_became_peered":"2024-09-21T05:40:03.596922+0000","last_unstale":"2024-09-21T05:40:05.245012+0000","last_undegraded":"2024-09-21T05:40:05.245012+0000","last_fullsized":"2024-09-21T05:40:05.245012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:18.614162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245115+0000","last_change":"2024-09-21T05:40:02.582587+0000","last_active":"2024-09-21T05:40:05.245115+0000","last_peered":"2024-09-21T05:40:05.245115+0000","last_clean":"2024-09-21T05:40:05.245115+0000","last_became_active":"2024-09-21T05:40:02.580120+0000","last_became_peered":"2024-09-21T05:40:02.580120+0000","last_unstale":"2024-09-21T05:40:05.245115+0000","last_undegraded":"2024-09-21T05:40:05.245115+0000","last_fullsized":"2024-09-21T05:40:05.245115+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:20.438203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583421+0000","last_change":"2024-09-21T05:40:03.593160+0000","last_active":"2024-09-21T05:40:04.583421+0000","last_peered":"2024-09-21T05:40:04.583421+0000","last_clean":"2024-09-21T05:40:04.583421+0000","last_became_active":"2024-09-21T05:40:03.592817+0000","last_became_peered":"2024-09-21T05:40:03.592817+0000","last_unstale":"2024-09-21T05:40:04.583421+0000","last_undegraded":"2024-09-21T05:40:04.583421+0000","last_fullsized":"2024-09-21T05:40:04.583421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:09.792067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244714+0000","last_change":"2024-09-21T05:40:03.588128+0000","last_active":"2024-09-21T05:40:05.244714+0000","last_peered":"2024-09-21T05:40:05.244714+0000","last_clean":"2024-09-21T05:40:05.244714+0000","last_became_active":"2024-09-21T05:40:03.587800+0000","last_became_peered":"2024-09-21T05:40:03.587800+0000","last_unstale":"2024-09-21T05:40:05.244714+0000","last_undegraded":"2024-09-21T05:40:05.244714+0000","last_fullsized":"2024-09-21T05:40:05.244714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:19.540979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583319+0000","last_change":"2024-09-21T05:40:02.579425+0000","last_active":"2024-09-21T05:40:04.583319+0000","last_peered":"2024-09-21T05:40:04.583319+0000","last_clean":"2024-09-21T05:40:04.583319+0000","last_became_active":"2024-09-21T05:40:02.579121+0000","last_became_peered":"2024-09-21T05:40:02.579121+0000","last_unstale":"2024-09-21T05:40:04.583319+0000","last_undegraded":"2024-09-21T05:40:04.583319+0000","last_fullsized":"2024-09-21T05:40:04.583319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:17.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244414+0000","last_change":"2024-09-21T05:40:02.583950+0000","last_active":"2024-09-21T05:40:05.244414+0000","last_peered":"2024-09-21T05:40:05.244414+0000","last_clean":"2024-09-21T05:40:05.244414+0000","last_became_active":"2024-09-21T05:40:02.583565+0000","last_became_peered":"2024-09-21T05:40:02.583565+0000","last_unstale":"2024-09-21T05:40:05.244414+0000","last_undegraded":"2024-09-21T05:40:05.244414+0000","last_fullsized":"2024-09-21T05:40:05.244414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:24.340836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584985+0000","last_change":"2024-09-21T05:40:03.574794+0000","last_active":"2024-09-21T05:40:04.584985+0000","last_peered":"2024-09-21T05:40:04.584985+0000","last_clean":"2024-09-21T05:40:04.584985+0000","last_became_active":"2024-09-21T05:40:03.574445+0000","last_became_peered":"2024-09-21T05:40:03.574445+0000","last_unstale":"2024-09-21T05:40:04.584985+0000","last_undegraded":"2024-09-21T05:40:04.584985+0000","last_fullsized":"2024-09-21T05:40:04.584985+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:25.544807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244295+0000","last_change":"2024-09-21T05:40:02.580478+0000","last_active":"2024-09-21T05:40:05.244295+0000","last_peered":"2024-09-21T05:40:05.244295+0000","last_clean":"2024-09-21T05:40:05.244295+0000","last_became_active":"2024-09-21T05:40:02.580172+0000","last_became_peered":"2024-09-21T05:40:02.580172+0000","last_unstale":"2024-09-21T05:40:05.244295+0000","last_undegraded":"2024-09-21T05:40:05.244295+0000","last_fullsized":"2024-09-21T05:40:05.244295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:24.539189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584285+0000","last_change":"2024-09-21T05:40:02.586220+0000","last_active":"2024-09-21T05:40:04.584285+0000","last_peered":"2024-09-21T05:40:04.584285+0000","last_clean":"2024-09-21T05:40:04.584285+0000","last_became_active":"2024-09-21T05:40:02.585561+0000","last_became_peered":"2024-09-21T05:40:02.585561+0000","last_unstale":"2024-09-21T05:40:04.584285+0000","last_undegraded":"2024-09-21T05:40:04.584285+0000","last_fullsized":"2024-09-21T05:40:04.584285+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:01.014995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245528+0000","last_change":"2024-09-21T05:40:03.591303+0000","last_active":"2024-09-21T05:40:05.245528+0000","last_peered":"2024-09-21T05:40:05.245528+0000","last_clean":"2024-09-21T05:40:05.245528+0000","last_became_active":"2024-09-21T05:40:03.590934+0000","last_became_peered":"2024-09-21T05:40:03.590934+0000","last_unstale":"2024-09-21T05:40:05.245528+0000","last_undegraded":"2024-09-21T05:40:05.245528+0000","last_fullsized":"2024-09-21T05:40:05.245528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:27.355864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583610+0000","last_change":"2024-09-21T05:40:03.579661+0000","last_active":"2024-09-21T05:40:04.583610+0000","last_peered":"2024-09-21T05:40:04.583610+0000","last_clean":"2024-09-21T05:40:04.583610+0000","last_became_active":"2024-09-21T05:40:03.577844+0000","last_became_peered":"2024-09-21T05:40:03.577844+0000","last_unstale":"2024-09-21T05:40:04.583610+0000","last_undegraded":"2024-09-21T05:40:04.583610+0000","last_fullsized":"2024-09-21T05:40:04.583610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:13.602757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163673+0000","last_change":"2024-09-21T05:40:03.590622+0000","last_active":"2024-09-21T05:40:05.163673+0000","last_peered":"2024-09-21T05:40:05.163673+0000","last_clean":"2024-09-21T05:40:05.163673+0000","last_became_active":"2024-09-21T05:40:03.590218+0000","last_became_peered":"2024-09-21T05:40:03.590218+0000","last_unstale":"2024-09-21T05:40:05.163673+0000","last_undegraded":"2024-09-21T05:40:05.163673+0000","last_fullsized":"2024-09-21T05:40:05.163673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:58.082728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163364+0000","last_change":"2024-09-21T05:40:03.583528+0000","last_active":"2024-09-21T05:40:05.163364+0000","last_peered":"2024-09-21T05:40:05.163364+0000","last_clean":"2024-09-21T05:40:05.163364+0000","last_became_active":"2024-09-21T05:40:03.583147+0000","last_became_peered":"2024-09-21T05:40:03.583147+0000","last_unstale":"2024-09-21T05:40:05.163364+0000","last_undegraded":"2024-09-21T05:40:05.163364+0000","last_fullsized":"2024-09-21T05:40:05.163364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:43.561897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587821+0000","last_change":"2024-09-21T05:40:02.585866+0000","last_active":"2024-09-21T05:40:04.587821+0000","last_peered":"2024-09-21T05:40:04.587821+0000","last_clean":"2024-09-21T05:40:04.587821+0000","last_became_active":"2024-09-21T05:40:02.585588+0000","last_became_peered":"2024-09-21T05:40:02.585588+0000","last_unstale":"2024-09-21T05:40:04.587821+0000","last_undegraded":"2024-09-21T05:40:04.587821+0000","last_fullsized":"2024-09-21T05:40:04.587821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.409276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163667+0000","last_change":"2024-09-21T05:40:03.587182+0000","last_active":"2024-09-21T05:40:05.163667+0000","last_peered":"2024-09-21T05:40:05.163667+0000","last_clean":"2024-09-21T05:40:05.163667+0000","last_became_active":"2024-09-21T05:40:03.584816+0000","last_became_peered":"2024-09-21T05:40:03.584816+0000","last_unstale":"2024-09-21T05:40:05.163667+0000","last_undegraded":"2024-09-21T05:40:05.163667+0000","last_fullsized":"2024-09-21T05:40:05.163667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:22.728465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584922+0000","last_change":"2024-09-21T05:40:02.580632+0000","last_active":"2024-09-21T05:40:04.584922+0000","last_peered":"2024-09-21T05:40:04.584922+0000","last_clean":"2024-09-21T05:40:04.584922+0000","last_became_active":"2024-09-21T05:40:02.580318+0000","last_became_peered":"2024-09-21T05:40:02.580318+0000","last_unstale":"2024-09-21T05:40:04.584922+0000","last_undegraded":"2024-09-21T05:40:04.584922+0000","last_fullsized":"2024-09-21T05:40:04.584922+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:02.332464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163383+0000","last_change":"2024-09-21T05:40:03.583922+0000","last_active":"2024-09-21T05:40:05.163383+0000","last_peered":"2024-09-21T05:40:05.163383+0000","last_clean":"2024-09-21T05:40:05.163383+0000","last_became_active":"2024-09-21T05:40:03.583568+0000","last_became_peered":"2024-09-21T05:40:03.583568+0000","last_unstale":"2024-09-21T05:40:05.163383+0000","last_undegraded":"2024-09-21T05:40:05.163383+0000","last_fullsized":"2024-09-21T05:40:05.163383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:31.304310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244187+0000","last_change":"2024-09-21T05:40:02.583065+0000","last_active":"2024-09-21T05:40:05.244187+0000","last_peered":"2024-09-21T05:40:05.244187+0000","last_clean":"2024-09-21T05:40:05.244187+0000","last_became_active":"2024-09-21T05:40:02.582693+0000","last_became_peered":"2024-09-21T05:40:02.582693+0000","last_unstale":"2024-09-21T05:40:05.244187+0000","last_undegraded":"2024-09-21T05:40:05.244187+0000","last_fullsized":"2024-09-21T05:40:05.244187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:33.159567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163687+0000","last_change":"2024-09-21T05:40:03.594519+0000","last_active":"2024-09-21T05:40:05.163687+0000","last_peered":"2024-09-21T05:40:05.163687+0000","last_clean":"2024-09-21T05:40:05.163687+0000","last_became_active":"2024-09-21T05:40:03.594162+0000","last_became_peered":"2024-09-21T05:40:03.594162+0000","last_unstale":"2024-09-21T05:40:05.163687+0000","last_undegraded":"2024-09-21T05:40:05.163687+0000","last_fullsized":"2024-09-21T05:40:05.163687+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:11.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584299+0000","last_change":"2024-09-21T05:40:02.586383+0000","last_active":"2024-09-21T05:40:04.584299+0000","last_peered":"2024-09-21T05:40:04.584299+0000","last_clean":"2024-09-21T05:40:04.584299+0000","last_became_active":"2024-09-21T05:40:02.585939+0000","last_became_peered":"2024-09-21T05:40:02.585939+0000","last_unstale":"2024-09-21T05:40:04.584299+0000","last_undegraded":"2024-09-21T05:40:04.584299+0000","last_fullsized":"2024-09-21T05:40:04.584299+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:28.955499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245711+0000","last_change":"2024-09-21T05:40:03.597292+0000","last_active":"2024-09-21T05:40:05.245711+0000","last_peered":"2024-09-21T05:40:05.245711+0000","last_clean":"2024-09-21T05:40:05.245711+0000","last_became_active":"2024-09-21T05:40:03.596914+0000","last_became_peered":"2024-09-21T05:40:03.596914+0000","last_unstale":"2024-09-21T05:40:05.245711+0000","last_undegraded":"2024-09-21T05:40:05.245711+0000","last_fullsized":"2024-09-21T05:40:05.245711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:07.345872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.589442+0000","last_change":"2024-09-21T05:40:02.580955+0000","last_active":"2024-09-21T05:40:04.589442+0000","last_peered":"2024-09-21T05:40:04.589442+0000","last_clean":"2024-09-21T05:40:04.589442+0000","last_became_active":"2024-09-21T05:40:02.580625+0000","last_became_peered":"2024-09-21T05:40:02.580625+0000","last_unstale":"2024-09-21T05:40:04.589442+0000","last_undegraded":"2024-09-21T05:40:04.589442+0000","last_fullsized":"2024-09-21T05:40:04.589442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:06.416483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587745+0000","last_change":"2024-09-21T05:40:03.597758+0000","last_active":"2024-09-21T05:40:04.587745+0000","last_peered":"2024-09-21T05:40:04.587745+0000","last_clean":"2024-09-21T05:40:04.587745+0000","last_became_active":"2024-09-21T05:40:03.597153+0000","last_became_peered":"2024-09-21T05:40:03.597153+0000","last_unstale":"2024-09-21T05:40:04.587745+0000","last_undegraded":"2024-09-21T05:40:04.587745+0000","last_fullsized":"2024-09-21T05:40:04.587745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.176357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583744+0000","last_change":"2024-09-21T05:40:02.580377+0000","last_active":"2024-09-21T05:40:04.583744+0000","last_peered":"2024-09-21T05:40:04.583744+0000","last_clean":"2024-09-21T05:40:04.583744+0000","last_became_active":"2024-09-21T05:40:02.580032+0000","last_became_peered":"2024-09-21T05:40:02.580032+0000","last_unstale":"2024-09-21T05:40:04.583744+0000","last_undegraded":"2024-09-21T05:40:04.583744+0000","last_fullsized":"2024-09-21T05:40:04.583744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.133340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584784+0000","last_change":"2024-09-21T05:40:03.586169+0000","last_active":"2024-09-21T05:40:04.584784+0000","last_peered":"2024-09-21T05:40:04.584784+0000","last_clean":"2024-09-21T05:40:04.584784+0000","last_became_active":"2024-09-21T05:40:03.585827+0000","last_became_peered":"2024-09-21T05:40:03.585827+0000","last_unstale":"2024-09-21T05:40:04.584784+0000","last_undegraded":"2024-09-21T05:40:04.584784+0000","last_fullsized":"2024-09-21T05:40:04.584784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.217251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587957+0000","last_change":"2024-09-21T05:40:02.585651+0000","last_active":"2024-09-21T05:40:04.587957+0000","last_peered":"2024-09-21T05:40:04.587957+0000","last_clean":"2024-09-21T05:40:04.587957+0000","last_became_active":"2024-09-21T05:40:02.585338+0000","last_became_peered":"2024-09-21T05:40:02.585338+0000","last_unstale":"2024-09-21T05:40:04.587957+0000","last_undegraded":"2024-09-21T05:40:04.587957+0000","last_fullsized":"2024-09-21T05:40:04.587957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:20.805434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587893+0000","last_change":"2024-09-21T05:40:03.594038+0000","last_active":"2024-09-21T05:40:04.587893+0000","last_peered":"2024-09-21T05:40:04.587893+0000","last_clean":"2024-09-21T05:40:04.587893+0000","last_became_active":"2024-09-21T05:40:03.593643+0000","last_became_peered":"2024-09-21T05:40:03.593643+0000","last_unstale":"2024-09-21T05:40:04.587893+0000","last_undegraded":"2024-09-21T05:40:04.587893+0000","last_fullsized":"2024-09-21T05:40:04.587893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:16.968181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583440+0000","last_change":"2024-09-21T05:40:02.587534+0000","last_active":"2024-09-21T05:40:04.583440+0000","last_peered":"2024-09-21T05:40:04.583440+0000","last_clean":"2024-09-21T05:40:04.583440+0000","last_became_active":"2024-09-21T05:40:02.586822+0000","last_became_peered":"2024-09-21T05:40:02.586822+0000","last_unstale":"2024-09-21T05:40:04.583440+0000","last_undegraded":"2024-09-21T05:40:04.583440+0000","last_fullsized":"2024-09-21T05:40:04.583440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.443912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244356+0000","last_change":"2024-09-21T05:40:03.582258+0000","last_active":"2024-09-21T05:40:05.244356+0000","last_peered":"2024-09-21T05:40:05.244356+0000","last_clean":"2024-09-21T05:40:05.244356+0000","last_became_active":"2024-09-21T05:40:03.581871+0000","last_became_peered":"2024-09-21T05:40:03.581871+0000","last_unstale":"2024-09-21T05:40:05.244356+0000","last_undegraded":"2024-09-21T05:40:05.244356+0000","last_fullsized":"2024-09-21T05:40:05.244356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:17.462467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:01.552730+0000","last_change":"2024-09-21T05:40:01.552730+0000","last_active":"2024-09-21T05:40:01.552730+0000","last_peered":"2024-09-21T05:40:01.552730+0000","last_clean":"2024-09-21T05:40:01.552730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:01.552730+0000","last_undegraded":"2024-09-21T05:40:01.552730+0000","last_fullsized":"2024-09-21T05:40:01.552730+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244461+0000","last_change":"2024-09-21T05:40:02.582884+0000","last_active":"2024-09-21T05:40:05.244461+0000","last_peered":"2024-09-21T05:40:05.244461+0000","last_clean":"2024-09-21T05:40:05.244461+0000","last_became_active":"2024-09-21T05:40:02.582586+0000","last_became_peered":"2024-09-21T05:40:02.582586+0000","last_unstale":"2024-09-21T05:40:05.244461+0000","last_undegraded":"2024-09-21T05:40:05.244461+0000","last_fullsized":"2024-09-21T05:40:05.244461+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:10.954780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:40:02.558348+0000","last_change":"2024-09-21T05:40:02.558348+0000","last_active":"2024-09-21T05:40:02.558348+0000","last_peered":"2024-09-21T05:40:02.558348+0000","last_clean":"2024-09-21T05:40:02.558348+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:40:02.558348+0000","last_undegraded":"2024-09-21T05:40:02.558348+0000","last_fullsized":"2024-09-21T05:40:02.558348+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584477+0000","last_change":"2024-09-21T05:40:02.580170+0000","last_active":"2024-09-21T05:40:04.584477+0000","last_peered":"2024-09-21T05:40:04.584477+0000","last_clean":"2024-09-21T05:40:04.584477+0000","last_became_active":"2024-09-21T05:40:02.579822+0000","last_became_peered":"2024-09-21T05:40:02.579822+0000","last_unstale":"2024-09-21T05:40:04.584477+0000","last_undegraded":"2024-09-21T05:40:04.584477+0000","last_fullsized":"2024-09-21T05:40:04.584477+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:10.529215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":121,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-21T05:39:57.521139+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:39:57.521139+0000","last_peered":"2024-09-21T05:39:57.521139+0000","last_clean":"2024-09-21T05:39:57.521139+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:39:57.521139+0000","last_undegraded":"2024-09-21T05:39:57.521139+0000","last_fullsized":"2024-09-21T05:39:57.521139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":90,"acting":90,"num_store_stats":6},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":4626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":46,"acting":46,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27908,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343932,"statfs":{"total":96636764160,"available":96608186368,"internally_reserved":0,"allocated":897024,"data_stored":701930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":111562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":137337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":132055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":344064,"data_stored":130491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35712,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336128,"statfs":{"total":96636764160,"available":96600195072,"internally_reserved":0,"allocated":368640,"data_stored":138355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":727089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:40:07.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 -- 172.21.15.22:0/2196711668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0b4530 msgr2=0x7f4fdc15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 --2- 172.21.15.22:0/2196711668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0b4530 0x7f4fdc15c830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009830 tx=0x7f4fd002a480 comp rx=0 tx=0).stop 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 -- 172.21.15.22:0/2196711668 shutdown_connections 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 --2- 172.21.15.22:0/2196711668 >> v2:172.21.15.134:3300/0 conn(0x7f4fdc160c90 0x7f4fdc0d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 --2- 172.21.15.22:0/2196711668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0b4530 0x7f4fdc15c830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 -- 172.21.15.22:0/2196711668 >> 172.21.15.22:0/2196711668 conn(0x7f4fdc0cee20 msgr2=0x7f4fdc0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 -- 172.21.15.22:0/2196711668 shutdown_connections 2024-09-21T05:40:07.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.805+0000 7f4fe2056640 1 -- 172.21.15.22:0/2196711668 wait complete. 2024-09-21T05:40:07.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.807+0000 7f4fe2056640 1 Processor -- start 2024-09-21T05:40:07.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.807+0000 7f4fe2056640 1 -- start start 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.808+0000 7f4fe2056640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0cff70 0x7f4fdc0b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.808+0000 7f4fe2056640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fdc160c90 0x7f4fdc147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.808+0000 7f4fe2056640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 0x7f4fdc14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.808+0000 7f4fe2056640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0d17d0 con 0x7f4fdc1483c0 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.808+0000 7f4fe2056640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4fdc0d1650 con 0x7f4fdc160c90 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.808+0000 7f4fe2056640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4fdc0d1950 con 0x7f4fdc0cff70 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 0x7f4fdc14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 0x7f4fdc14f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53652/0 (socket says 172.21.15.22:53652) 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 -- 172.21.15.22:0/863101385 learned_addr learned my addr 172.21.15.22:0/863101385 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdb7fe640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0cff70 0x7f4fdc0b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdaffd640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fdc160c90 0x7f4fdc147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 -- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0cff70 msgr2=0x7f4fdc0b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0cff70 0x7f4fdc0b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 -- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fdc160c90 msgr2=0x7f4fdc147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fdc160c90 0x7f4fdc147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 -- 172.21.15.22:0/863101385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fdc0b78b0 con 0x7f4fdc1483c0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fdbfff640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 0x7f4fdc14f000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400ebf0 tx=0x7f4fd400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4003b70 con 0x7f4fdc1483c0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fdc0c8650 con 0x7f4fdc1483c0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fd4003d10 con 0x7f4fdc1483c0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004280 con 0x7f4fdc1483c0 2024-09-21T05:40:07.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.809+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fdc0b6b70 con 0x7f4fdc1483c0 2024-09-21T05:40:07.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.811+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fdc0d1e00 con 0x7f4fdc1483c0 2024-09-21T05:40:07.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.811+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4fd4004420 con 0x7f4fdc1483c0 2024-09-21T05:40:07.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.812+0000 7f4fc3fff640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fc8078b90 0x7f4fc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:07.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.812+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4fd409b8c0 con 0x7f4fdc1483c0 2024-09-21T05:40:07.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.812+0000 7f4fdb7fe640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fc8078b90 0x7f4fc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:07.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.813+0000 7f4fdb7fe640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fc8078b90 0x7f4fc807b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc007fd0 tx=0x7f4fcc001370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:07.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:07.816+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4fd4063630 con 0x7f4fdc1483c0 2024-09-21T05:40:08.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.003+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fdc158790 con 0x7f4fc8078b90 2024-09-21T05:40:08.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.008+0000 7f4fc3fff640 1 -- 172.21.15.22:0/863101385 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f4fdc158790 con 0x7f4fc8078b90 2024-09-21T05:40:08.012 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:08.014 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:40:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fc8078b90 msgr2=0x7f4fc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fc8078b90 0x7f4fc807b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc007fd0 tx=0x7f4fcc001370 comp rx=0 tx=0).stop 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 msgr2=0x7f4fdc14f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 0x7f4fdc14f000 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400ebf0 tx=0x7f4fd400c6a0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 shutdown_connections 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4fdc1483c0 0x7f4fdc14f000 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4fdc160c90 0x7f4fdc147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fc8078b90 0x7f4fc807b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 --2- 172.21.15.22:0/863101385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4fdc0cff70 0x7f4fdc0b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 >> 172.21.15.22:0/863101385 conn(0x7f4fdc0cee20 msgr2=0x7f4fdc0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 shutdown_connections 2024-09-21T05:40:08.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.011+0000 7f4fe2056640 1 -- 172.21.15.22:0/863101385 wait complete. 2024-09-21T05:40:08.024 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-09-21T05:40:07.747309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241004,"kb_used_data":3952,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733716,"statfs":{"total":773094113280,"available":772847325184,"internally_reserved":0,"allocated":4046848,"data_stored":2266724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001281"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585391+0000","last_change":"2024-09-21T05:40:03.591951+0000","last_active":"2024-09-21T05:40:04.585391+0000","last_peered":"2024-09-21T05:40:04.585391+0000","last_clean":"2024-09-21T05:40:04.585391+0000","last_became_active":"2024-09-21T05:40:03.591582+0000","last_became_peered":"2024-09-21T05:40:03.591582+0000","last_unstale":"2024-09-21T05:40:04.585391+0000","last_undegraded":"2024-09-21T05:40:04.585391+0000","last_fullsized":"2024-09-21T05:40:04.585391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:42.963566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245090+0000","last_change":"2024-09-21T05:40:03.588128+0000","last_active":"2024-09-21T05:40:05.245090+0000","last_peered":"2024-09-21T05:40:05.245090+0000","last_clean":"2024-09-21T05:40:05.245090+0000","last_became_active":"2024-09-21T05:40:03.587771+0000","last_became_peered":"2024-09-21T05:40:03.587771+0000","last_unstale":"2024-09-21T05:40:05.245090+0000","last_undegraded":"2024-09-21T05:40:05.245090+0000","last_fullsized":"2024-09-21T05:40:05.245090+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:56.242972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245367+0000","last_change":"2024-09-21T05:40:03.597254+0000","last_active":"2024-09-21T05:40:05.245367+0000","last_peered":"2024-09-21T05:40:05.245367+0000","last_clean":"2024-09-21T05:40:05.245367+0000","last_became_active":"2024-09-21T05:40:03.596911+0000","last_became_peered":"2024-09-21T05:40:03.596911+0000","last_unstale":"2024-09-21T05:40:05.245367+0000","last_undegraded":"2024-09-21T05:40:05.245367+0000","last_fullsized":"2024-09-21T05:40:05.245367+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:56.414713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583607+0000","last_change":"2024-09-21T05:40:03.580710+0000","last_active":"2024-09-21T05:40:04.583607+0000","last_peered":"2024-09-21T05:40:04.583607+0000","last_clean":"2024-09-21T05:40:04.583607+0000","last_became_active":"2024-09-21T05:40:03.578193+0000","last_became_peered":"2024-09-21T05:40:03.578193+0000","last_unstale":"2024-09-21T05:40:04.583607+0000","last_undegraded":"2024-09-21T05:40:04.583607+0000","last_fullsized":"2024-09-21T05:40:04.583607+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:12.963462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163380+0000","last_change":"2024-09-21T05:40:03.583377+0000","last_active":"2024-09-21T05:40:05.163380+0000","last_peered":"2024-09-21T05:40:05.163380+0000","last_clean":"2024-09-21T05:40:05.163380+0000","last_became_active":"2024-09-21T05:40:03.582743+0000","last_became_peered":"2024-09-21T05:40:03.582743+0000","last_unstale":"2024-09-21T05:40:05.163380+0000","last_undegraded":"2024-09-21T05:40:05.163380+0000","last_fullsized":"2024-09-21T05:40:05.163380+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:37.774264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244660+0000","last_change":"2024-09-21T05:40:03.594538+0000","last_active":"2024-09-21T05:40:05.244660+0000","last_peered":"2024-09-21T05:40:05.244660+0000","last_clean":"2024-09-21T05:40:05.244660+0000","last_became_active":"2024-09-21T05:40:03.593800+0000","last_became_peered":"2024-09-21T05:40:03.593800+0000","last_unstale":"2024-09-21T05:40:05.244660+0000","last_undegraded":"2024-09-21T05:40:05.244660+0000","last_fullsized":"2024-09-21T05:40:05.244660+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:48.460419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245581+0000","last_change":"2024-09-21T05:40:03.586501+0000","last_active":"2024-09-21T05:40:05.245581+0000","last_peered":"2024-09-21T05:40:05.245581+0000","last_clean":"2024-09-21T05:40:05.245581+0000","last_became_active":"2024-09-21T05:40:03.586126+0000","last_became_peered":"2024-09-21T05:40:03.586126+0000","last_unstale":"2024-09-21T05:40:05.245581+0000","last_undegraded":"2024-09-21T05:40:05.245581+0000","last_fullsized":"2024-09-21T05:40:05.245581+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:23:35.875838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244442+0000","last_change":"2024-09-21T05:40:03.602549+0000","last_active":"2024-09-21T05:40:05.244442+0000","last_peered":"2024-09-21T05:40:05.244442+0000","last_clean":"2024-09-21T05:40:05.244442+0000","last_became_active":"2024-09-21T05:40:03.602146+0000","last_became_peered":"2024-09-21T05:40:03.602146+0000","last_unstale":"2024-09-21T05:40:05.244442+0000","last_undegraded":"2024-09-21T05:40:05.244442+0000","last_fullsized":"2024-09-21T05:40:05.244442+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:57:41.105916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585642+0000","last_change":"2024-09-21T05:40:03.577504+0000","last_active":"2024-09-21T05:40:04.585642+0000","last_peered":"2024-09-21T05:40:04.585642+0000","last_clean":"2024-09-21T05:40:04.585642+0000","last_became_active":"2024-09-21T05:40:03.577176+0000","last_became_peered":"2024-09-21T05:40:03.577176+0000","last_unstale":"2024-09-21T05:40:04.585642+0000","last_undegraded":"2024-09-21T05:40:04.585642+0000","last_fullsized":"2024-09-21T05:40:04.585642+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:09.792067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583924+0000","last_change":"2024-09-21T05:40:03.582035+0000","last_active":"2024-09-21T05:40:04.583924+0000","last_peered":"2024-09-21T05:40:04.583924+0000","last_clean":"2024-09-21T05:40:04.583924+0000","last_became_active":"2024-09-21T05:40:03.581662+0000","last_became_peered":"2024-09-21T05:40:03.581662+0000","last_unstale":"2024-09-21T05:40:04.583924+0000","last_undegraded":"2024-09-21T05:40:04.583924+0000","last_fullsized":"2024-09-21T05:40:04.583924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:46.211686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245457+0000","last_change":"2024-09-21T05:40:03.597175+0000","last_active":"2024-09-21T05:40:05.245457+0000","last_peered":"2024-09-21T05:40:05.245457+0000","last_clean":"2024-09-21T05:40:05.245457+0000","last_became_active":"2024-09-21T05:40:03.596867+0000","last_became_peered":"2024-09-21T05:40:03.596867+0000","last_unstale":"2024-09-21T05:40:05.245457+0000","last_undegraded":"2024-09-21T05:40:05.245457+0000","last_fullsized":"2024-09-21T05:40:05.245457+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:26.502910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245189+0000","last_change":"2024-09-21T05:40:03.595910+0000","last_active":"2024-09-21T05:40:05.245189+0000","last_peered":"2024-09-21T05:40:05.245189+0000","last_clean":"2024-09-21T05:40:05.245189+0000","last_became_active":"2024-09-21T05:40:03.595184+0000","last_became_peered":"2024-09-21T05:40:03.595184+0000","last_unstale":"2024-09-21T05:40:05.245189+0000","last_undegraded":"2024-09-21T05:40:05.245189+0000","last_fullsized":"2024-09-21T05:40:05.245189+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:38.803584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163421+0000","last_change":"2024-09-21T05:40:03.585159+0000","last_active":"2024-09-21T05:40:05.163421+0000","last_peered":"2024-09-21T05:40:05.163421+0000","last_clean":"2024-09-21T05:40:05.163421+0000","last_became_active":"2024-09-21T05:40:03.584369+0000","last_became_peered":"2024-09-21T05:40:03.584369+0000","last_unstale":"2024-09-21T05:40:05.163421+0000","last_undegraded":"2024-09-21T05:40:05.163421+0000","last_fullsized":"2024-09-21T05:40:05.163421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:23.359967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244592+0000","last_change":"2024-09-21T05:40:03.594146+0000","last_active":"2024-09-21T05:40:05.244592+0000","last_peered":"2024-09-21T05:40:05.244592+0000","last_clean":"2024-09-21T05:40:05.244592+0000","last_became_active":"2024-09-21T05:40:03.593367+0000","last_became_peered":"2024-09-21T05:40:03.593367+0000","last_unstale":"2024-09-21T05:40:05.244592+0000","last_undegraded":"2024-09-21T05:40:05.244592+0000","last_fullsized":"2024-09-21T05:40:05.244592+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:19.401173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244494+0000","last_change":"2024-09-21T05:40:03.593210+0000","last_active":"2024-09-21T05:40:05.244494+0000","last_peered":"2024-09-21T05:40:05.244494+0000","last_clean":"2024-09-21T05:40:05.244494+0000","last_became_active":"2024-09-21T05:40:03.592872+0000","last_became_peered":"2024-09-21T05:40:03.592872+0000","last_unstale":"2024-09-21T05:40:05.244494+0000","last_undegraded":"2024-09-21T05:40:05.244494+0000","last_fullsized":"2024-09-21T05:40:05.244494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:21.107732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244845+0000","last_change":"2024-09-21T05:40:03.589948+0000","last_active":"2024-09-21T05:40:05.244845+0000","last_peered":"2024-09-21T05:40:05.244845+0000","last_clean":"2024-09-21T05:40:05.244845+0000","last_became_active":"2024-09-21T05:40:03.589609+0000","last_became_peered":"2024-09-21T05:40:03.589609+0000","last_unstale":"2024-09-21T05:40:05.244845+0000","last_undegraded":"2024-09-21T05:40:05.244845+0000","last_fullsized":"2024-09-21T05:40:05.244845+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:02.033028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245524+0000","last_change":"2024-09-21T05:40:03.591893+0000","last_active":"2024-09-21T05:40:05.245524+0000","last_peered":"2024-09-21T05:40:05.245524+0000","last_clean":"2024-09-21T05:40:05.245524+0000","last_became_active":"2024-09-21T05:40:03.591445+0000","last_became_peered":"2024-09-21T05:40:03.591445+0000","last_unstale":"2024-09-21T05:40:05.245524+0000","last_undegraded":"2024-09-21T05:40:05.245524+0000","last_fullsized":"2024-09-21T05:40:05.245524+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:57:32.892080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245145+0000","last_change":"2024-09-21T05:40:03.584578+0000","last_active":"2024-09-21T05:40:05.245145+0000","last_peered":"2024-09-21T05:40:05.245145+0000","last_clean":"2024-09-21T05:40:05.245145+0000","last_became_active":"2024-09-21T05:40:03.584009+0000","last_became_peered":"2024-09-21T05:40:03.584009+0000","last_unstale":"2024-09-21T05:40:05.245145+0000","last_undegraded":"2024-09-21T05:40:05.245145+0000","last_fullsized":"2024-09-21T05:40:05.245145+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:41.244043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244890+0000","last_change":"2024-09-21T05:40:03.584102+0000","last_active":"2024-09-21T05:40:05.244890+0000","last_peered":"2024-09-21T05:40:05.244890+0000","last_clean":"2024-09-21T05:40:05.244890+0000","last_became_active":"2024-09-21T05:40:03.583776+0000","last_became_peered":"2024-09-21T05:40:03.583776+0000","last_unstale":"2024-09-21T05:40:05.244890+0000","last_undegraded":"2024-09-21T05:40:05.244890+0000","last_fullsized":"2024-09-21T05:40:05.244890+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:55.295143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163648+0000","last_change":"2024-09-21T05:40:03.595491+0000","last_active":"2024-09-21T05:40:05.163648+0000","last_peered":"2024-09-21T05:40:05.163648+0000","last_clean":"2024-09-21T05:40:05.163648+0000","last_became_active":"2024-09-21T05:40:03.595062+0000","last_became_peered":"2024-09-21T05:40:03.595062+0000","last_unstale":"2024-09-21T05:40:05.163648+0000","last_undegraded":"2024-09-21T05:40:05.163648+0000","last_fullsized":"2024-09-21T05:40:05.163648+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:41.038623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585638+0000","last_change":"2024-09-21T05:40:03.584545+0000","last_active":"2024-09-21T05:40:04.585638+0000","last_peered":"2024-09-21T05:40:04.585638+0000","last_clean":"2024-09-21T05:40:04.585638+0000","last_became_active":"2024-09-21T05:40:03.584222+0000","last_became_peered":"2024-09-21T05:40:03.584222+0000","last_unstale":"2024-09-21T05:40:04.585638+0000","last_undegraded":"2024-09-21T05:40:04.585638+0000","last_fullsized":"2024-09-21T05:40:04.585638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:27.700490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585211+0000","last_change":"2024-09-21T05:40:03.576475+0000","last_active":"2024-09-21T05:40:04.585211+0000","last_peered":"2024-09-21T05:40:04.585211+0000","last_clean":"2024-09-21T05:40:04.585211+0000","last_became_active":"2024-09-21T05:40:03.576079+0000","last_became_peered":"2024-09-21T05:40:03.576079+0000","last_unstale":"2024-09-21T05:40:04.585211+0000","last_undegraded":"2024-09-21T05:40:04.585211+0000","last_fullsized":"2024-09-21T05:40:04.585211+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:03.273830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587668+0000","last_change":"2024-09-21T05:40:03.591516+0000","last_active":"2024-09-21T05:40:04.587668+0000","last_peered":"2024-09-21T05:40:04.587668+0000","last_clean":"2024-09-21T05:40:04.587668+0000","last_became_active":"2024-09-21T05:40:03.591036+0000","last_became_peered":"2024-09-21T05:40:03.591036+0000","last_unstale":"2024-09-21T05:40:04.587668+0000","last_undegraded":"2024-09-21T05:40:04.587668+0000","last_fullsized":"2024-09-21T05:40:04.587668+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:42.963566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244323+0000","last_change":"2024-09-21T05:40:03.597051+0000","last_active":"2024-09-21T05:40:05.244323+0000","last_peered":"2024-09-21T05:40:05.244323+0000","last_clean":"2024-09-21T05:40:05.244323+0000","last_became_active":"2024-09-21T05:40:03.596560+0000","last_became_peered":"2024-09-21T05:40:03.596560+0000","last_unstale":"2024-09-21T05:40:05.244323+0000","last_undegraded":"2024-09-21T05:40:05.244323+0000","last_fullsized":"2024-09-21T05:40:05.244323+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:53.219876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584169+0000","last_change":"2024-09-21T05:40:03.587689+0000","last_active":"2024-09-21T05:40:04.584169+0000","last_peered":"2024-09-21T05:40:04.584169+0000","last_clean":"2024-09-21T05:40:04.584169+0000","last_became_active":"2024-09-21T05:40:03.587347+0000","last_became_peered":"2024-09-21T05:40:03.587347+0000","last_unstale":"2024-09-21T05:40:04.584169+0000","last_undegraded":"2024-09-21T05:40:04.584169+0000","last_fullsized":"2024-09-21T05:40:04.584169+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:12:32.396286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245235+0000","last_change":"2024-09-21T05:40:03.587262+0000","last_active":"2024-09-21T05:40:05.245235+0000","last_peered":"2024-09-21T05:40:05.245235+0000","last_clean":"2024-09-21T05:40:05.245235+0000","last_became_active":"2024-09-21T05:40:03.586326+0000","last_became_peered":"2024-09-21T05:40:03.586326+0000","last_unstale":"2024-09-21T05:40:05.245235+0000","last_undegraded":"2024-09-21T05:40:05.245235+0000","last_fullsized":"2024-09-21T05:40:05.245235+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:08.160006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583927+0000","last_change":"2024-09-21T05:40:03.578020+0000","last_active":"2024-09-21T05:40:04.583927+0000","last_peered":"2024-09-21T05:40:04.583927+0000","last_clean":"2024-09-21T05:40:04.583927+0000","last_became_active":"2024-09-21T05:40:03.577015+0000","last_became_peered":"2024-09-21T05:40:03.577015+0000","last_unstale":"2024-09-21T05:40:04.583927+0000","last_undegraded":"2024-09-21T05:40:04.583927+0000","last_fullsized":"2024-09-21T05:40:04.583927+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:30.218264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245401+0000","last_change":"2024-09-21T05:40:03.596127+0000","last_active":"2024-09-21T05:40:05.245401+0000","last_peered":"2024-09-21T05:40:05.245401+0000","last_clean":"2024-09-21T05:40:05.245401+0000","last_became_active":"2024-09-21T05:40:03.595710+0000","last_became_peered":"2024-09-21T05:40:03.595710+0000","last_unstale":"2024-09-21T05:40:05.245401+0000","last_undegraded":"2024-09-21T05:40:05.245401+0000","last_fullsized":"2024-09-21T05:40:05.245401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:08.055220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585741+0000","last_change":"2024-09-21T05:40:03.580932+0000","last_active":"2024-09-21T05:40:04.585741+0000","last_peered":"2024-09-21T05:40:04.585741+0000","last_clean":"2024-09-21T05:40:04.585741+0000","last_became_active":"2024-09-21T05:40:03.580586+0000","last_became_peered":"2024-09-21T05:40:03.580586+0000","last_unstale":"2024-09-21T05:40:04.585741+0000","last_undegraded":"2024-09-21T05:40:04.585741+0000","last_fullsized":"2024-09-21T05:40:04.585741+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:12.963462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245625+0000","last_change":"2024-09-21T05:40:03.584814+0000","last_active":"2024-09-21T05:40:05.245625+0000","last_peered":"2024-09-21T05:40:05.245625+0000","last_clean":"2024-09-21T05:40:05.245625+0000","last_became_active":"2024-09-21T05:40:03.584508+0000","last_became_peered":"2024-09-21T05:40:03.584508+0000","last_unstale":"2024-09-21T05:40:05.245625+0000","last_undegraded":"2024-09-21T05:40:05.245625+0000","last_fullsized":"2024-09-21T05:40:05.245625+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:01.161113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584738+0000","last_change":"2024-09-21T05:40:03.577513+0000","last_active":"2024-09-21T05:40:04.584738+0000","last_peered":"2024-09-21T05:40:04.584738+0000","last_clean":"2024-09-21T05:40:04.584738+0000","last_became_active":"2024-09-21T05:40:03.577197+0000","last_became_peered":"2024-09-21T05:40:03.577197+0000","last_unstale":"2024-09-21T05:40:04.584738+0000","last_undegraded":"2024-09-21T05:40:04.584738+0000","last_fullsized":"2024-09-21T05:40:04.584738+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:34.165185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244527+0000","last_change":"2024-09-21T05:40:03.589121+0000","last_active":"2024-09-21T05:40:05.244527+0000","last_peered":"2024-09-21T05:40:05.244527+0000","last_clean":"2024-09-21T05:40:05.244527+0000","last_became_active":"2024-09-21T05:40:03.588750+0000","last_became_peered":"2024-09-21T05:40:03.588750+0000","last_unstale":"2024-09-21T05:40:05.244527+0000","last_undegraded":"2024-09-21T05:40:05.244527+0000","last_fullsized":"2024-09-21T05:40:05.244527+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:57.621012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587871+0000","last_change":"2024-09-21T05:40:03.592754+0000","last_active":"2024-09-21T05:40:04.587871+0000","last_peered":"2024-09-21T05:40:04.587871+0000","last_clean":"2024-09-21T05:40:04.587871+0000","last_became_active":"2024-09-21T05:40:03.592183+0000","last_became_peered":"2024-09-21T05:40:03.592183+0000","last_unstale":"2024-09-21T05:40:04.587871+0000","last_undegraded":"2024-09-21T05:40:04.587871+0000","last_fullsized":"2024-09-21T05:40:04.587871+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:30.497627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585534+0000","last_change":"2024-09-21T05:40:02.581436+0000","last_active":"2024-09-21T05:40:04.585534+0000","last_peered":"2024-09-21T05:40:04.585534+0000","last_clean":"2024-09-21T05:40:04.585534+0000","last_became_active":"2024-09-21T05:40:02.581120+0000","last_became_peered":"2024-09-21T05:40:02.581120+0000","last_unstale":"2024-09-21T05:40:04.585534+0000","last_undegraded":"2024-09-21T05:40:04.585534+0000","last_fullsized":"2024-09-21T05:40:04.585534+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:00:16.673104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585463+0000","last_change":"2024-09-21T05:40:03.574748+0000","last_active":"2024-09-21T05:40:04.585463+0000","last_peered":"2024-09-21T05:40:04.585463+0000","last_clean":"2024-09-21T05:40:04.585463+0000","last_became_active":"2024-09-21T05:40:03.574374+0000","last_became_peered":"2024-09-21T05:40:03.574374+0000","last_unstale":"2024-09-21T05:40:04.585463+0000","last_undegraded":"2024-09-21T05:40:04.585463+0000","last_fullsized":"2024-09-21T05:40:04.585463+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.176357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587751+0000","last_change":"2024-09-21T05:40:02.587449+0000","last_active":"2024-09-21T05:40:04.587751+0000","last_peered":"2024-09-21T05:40:04.587751+0000","last_clean":"2024-09-21T05:40:04.587751+0000","last_became_active":"2024-09-21T05:40:02.587151+0000","last_became_peered":"2024-09-21T05:40:02.587151+0000","last_unstale":"2024-09-21T05:40:04.587751+0000","last_undegraded":"2024-09-21T05:40:04.587751+0000","last_fullsized":"2024-09-21T05:40:04.587751+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:34.268429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.586024+0000","last_change":"2024-09-21T05:40:03.581555+0000","last_active":"2024-09-21T05:40:04.586024+0000","last_peered":"2024-09-21T05:40:04.586024+0000","last_clean":"2024-09-21T05:40:04.586024+0000","last_became_active":"2024-09-21T05:40:03.581229+0000","last_became_peered":"2024-09-21T05:40:03.581229+0000","last_unstale":"2024-09-21T05:40:04.586024+0000","last_undegraded":"2024-09-21T05:40:04.586024+0000","last_fullsized":"2024-09-21T05:40:04.586024+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:11.525306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244786+0000","last_change":"2024-09-21T05:40:02.581247+0000","last_active":"2024-09-21T05:40:05.244786+0000","last_peered":"2024-09-21T05:40:05.244786+0000","last_clean":"2024-09-21T05:40:05.244786+0000","last_became_active":"2024-09-21T05:40:02.580490+0000","last_became_peered":"2024-09-21T05:40:02.580490+0000","last_unstale":"2024-09-21T05:40:05.244786+0000","last_undegraded":"2024-09-21T05:40:05.244786+0000","last_fullsized":"2024-09-21T05:40:05.244786+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:01.905529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584010+0000","last_change":"2024-09-21T05:40:03.576536+0000","last_active":"2024-09-21T05:40:04.584010+0000","last_peered":"2024-09-21T05:40:04.584010+0000","last_clean":"2024-09-21T05:40:04.584010+0000","last_became_active":"2024-09-21T05:40:03.576157+0000","last_became_peered":"2024-09-21T05:40:03.576157+0000","last_unstale":"2024-09-21T05:40:04.584010+0000","last_undegraded":"2024-09-21T05:40:04.584010+0000","last_fullsized":"2024-09-21T05:40:04.584010+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:58.586041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163515+0000","last_change":"2024-09-21T05:40:02.576102+0000","last_active":"2024-09-21T05:40:05.163515+0000","last_peered":"2024-09-21T05:40:05.163515+0000","last_clean":"2024-09-21T05:40:05.163515+0000","last_became_active":"2024-09-21T05:40:02.575795+0000","last_became_peered":"2024-09-21T05:40:02.575795+0000","last_unstale":"2024-09-21T05:40:05.163515+0000","last_undegraded":"2024-09-21T05:40:05.163515+0000","last_fullsized":"2024-09-21T05:40:05.163515+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:31:43.899292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245695+0000","last_change":"2024-09-21T05:40:03.589422+0000","last_active":"2024-09-21T05:40:05.245695+0000","last_peered":"2024-09-21T05:40:05.245695+0000","last_clean":"2024-09-21T05:40:05.245695+0000","last_became_active":"2024-09-21T05:40:03.589096+0000","last_became_peered":"2024-09-21T05:40:03.589096+0000","last_unstale":"2024-09-21T05:40:05.245695+0000","last_undegraded":"2024-09-21T05:40:05.245695+0000","last_fullsized":"2024-09-21T05:40:05.245695+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:12.842917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585771+0000","last_change":"2024-09-21T05:40:02.583760+0000","last_active":"2024-09-21T05:40:04.585771+0000","last_peered":"2024-09-21T05:40:04.585771+0000","last_clean":"2024-09-21T05:40:04.585771+0000","last_became_active":"2024-09-21T05:40:02.582875+0000","last_became_peered":"2024-09-21T05:40:02.582875+0000","last_unstale":"2024-09-21T05:40:04.585771+0000","last_undegraded":"2024-09-21T05:40:04.585771+0000","last_fullsized":"2024-09-21T05:40:04.585771+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:07.218081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245147+0000","last_change":"2024-09-21T05:40:03.593834+0000","last_active":"2024-09-21T05:40:05.245147+0000","last_peered":"2024-09-21T05:40:05.245147+0000","last_clean":"2024-09-21T05:40:05.245147+0000","last_became_active":"2024-09-21T05:40:03.593476+0000","last_became_peered":"2024-09-21T05:40:03.593476+0000","last_unstale":"2024-09-21T05:40:05.245147+0000","last_undegraded":"2024-09-21T05:40:05.245147+0000","last_fullsized":"2024-09-21T05:40:05.245147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:36.095605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585340+0000","last_change":"2024-09-21T05:40:02.581590+0000","last_active":"2024-09-21T05:40:04.585340+0000","last_peered":"2024-09-21T05:40:04.585340+0000","last_clean":"2024-09-21T05:40:04.585340+0000","last_became_active":"2024-09-21T05:40:02.581263+0000","last_became_peered":"2024-09-21T05:40:02.581263+0000","last_unstale":"2024-09-21T05:40:04.585340+0000","last_undegraded":"2024-09-21T05:40:04.585340+0000","last_fullsized":"2024-09-21T05:40:04.585340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:37.833705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585042+0000","last_change":"2024-09-21T05:40:03.577545+0000","last_active":"2024-09-21T05:40:04.585042+0000","last_peered":"2024-09-21T05:40:04.585042+0000","last_clean":"2024-09-21T05:40:04.585042+0000","last_became_active":"2024-09-21T05:40:03.577211+0000","last_became_peered":"2024-09-21T05:40:03.577211+0000","last_unstale":"2024-09-21T05:40:04.585042+0000","last_undegraded":"2024-09-21T05:40:04.585042+0000","last_fullsized":"2024-09-21T05:40:04.585042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:39.531039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244292+0000","last_change":"2024-09-21T05:40:02.582062+0000","last_active":"2024-09-21T05:40:05.244292+0000","last_peered":"2024-09-21T05:40:05.244292+0000","last_clean":"2024-09-21T05:40:05.244292+0000","last_became_active":"2024-09-21T05:40:02.581737+0000","last_became_peered":"2024-09-21T05:40:02.581737+0000","last_unstale":"2024-09-21T05:40:05.244292+0000","last_undegraded":"2024-09-21T05:40:05.244292+0000","last_fullsized":"2024-09-21T05:40:05.244292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:09.507441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583468+0000","last_change":"2024-09-21T05:40:03.598467+0000","last_active":"2024-09-21T05:40:04.583468+0000","last_peered":"2024-09-21T05:40:04.583468+0000","last_clean":"2024-09-21T05:40:04.583468+0000","last_became_active":"2024-09-21T05:40:03.598054+0000","last_became_peered":"2024-09-21T05:40:03.598054+0000","last_unstale":"2024-09-21T05:40:04.583468+0000","last_undegraded":"2024-09-21T05:40:04.583468+0000","last_fullsized":"2024-09-21T05:40:04.583468+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:57.683487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585177+0000","last_change":"2024-09-21T05:40:02.576540+0000","last_active":"2024-09-21T05:40:04.585177+0000","last_peered":"2024-09-21T05:40:04.585177+0000","last_clean":"2024-09-21T05:40:04.585177+0000","last_became_active":"2024-09-21T05:40:02.576231+0000","last_became_peered":"2024-09-21T05:40:02.576231+0000","last_unstale":"2024-09-21T05:40:04.585177+0000","last_undegraded":"2024-09-21T05:40:04.585177+0000","last_fullsized":"2024-09-21T05:40:04.585177+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:25:06.784274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245499+0000","last_change":"2024-09-21T05:40:03.595491+0000","last_active":"2024-09-21T05:40:05.245499+0000","last_peered":"2024-09-21T05:40:05.245499+0000","last_clean":"2024-09-21T05:40:05.245499+0000","last_became_active":"2024-09-21T05:40:03.595148+0000","last_became_peered":"2024-09-21T05:40:03.595148+0000","last_unstale":"2024-09-21T05:40:05.245499+0000","last_undegraded":"2024-09-21T05:40:05.245499+0000","last_fullsized":"2024-09-21T05:40:05.245499+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:16.922990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245032+0000","last_change":"2024-09-21T05:40:02.587265+0000","last_active":"2024-09-21T05:40:05.245032+0000","last_peered":"2024-09-21T05:40:05.245032+0000","last_clean":"2024-09-21T05:40:05.245032+0000","last_became_active":"2024-09-21T05:40:02.586947+0000","last_became_peered":"2024-09-21T05:40:02.586947+0000","last_unstale":"2024-09-21T05:40:05.245032+0000","last_undegraded":"2024-09-21T05:40:05.245032+0000","last_fullsized":"2024-09-21T05:40:05.245032+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:58:04.995697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245051+0000","last_change":"2024-09-21T05:40:03.583507+0000","last_active":"2024-09-21T05:40:05.245051+0000","last_peered":"2024-09-21T05:40:05.245051+0000","last_clean":"2024-09-21T05:40:05.245051+0000","last_became_active":"2024-09-21T05:40:03.583188+0000","last_became_peered":"2024-09-21T05:40:03.583188+0000","last_unstale":"2024-09-21T05:40:05.245051+0000","last_undegraded":"2024-09-21T05:40:05.245051+0000","last_fullsized":"2024-09-21T05:40:05.245051+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:24:07.340885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587892+0000","last_change":"2024-09-21T05:40:02.585079+0000","last_active":"2024-09-21T05:40:04.587892+0000","last_peered":"2024-09-21T05:40:04.587892+0000","last_clean":"2024-09-21T05:40:04.587892+0000","last_became_active":"2024-09-21T05:40:02.584690+0000","last_became_peered":"2024-09-21T05:40:02.584690+0000","last_unstale":"2024-09-21T05:40:04.587892+0000","last_undegraded":"2024-09-21T05:40:04.587892+0000","last_fullsized":"2024-09-21T05:40:04.587892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:10.788740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244961+0000","last_change":"2024-09-21T05:40:03.585755+0000","last_active":"2024-09-21T05:40:05.244961+0000","last_peered":"2024-09-21T05:40:05.244961+0000","last_clean":"2024-09-21T05:40:05.244961+0000","last_became_active":"2024-09-21T05:40:03.585409+0000","last_became_peered":"2024-09-21T05:40:03.585409+0000","last_unstale":"2024-09-21T05:40:05.244961+0000","last_undegraded":"2024-09-21T05:40:05.244961+0000","last_fullsized":"2024-09-21T05:40:05.244961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:21.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245521+0000","last_change":"2024-09-21T05:40:02.587490+0000","last_active":"2024-09-21T05:40:05.245521+0000","last_peered":"2024-09-21T05:40:05.245521+0000","last_clean":"2024-09-21T05:40:05.245521+0000","last_became_active":"2024-09-21T05:40:02.587151+0000","last_became_peered":"2024-09-21T05:40:02.587151+0000","last_unstale":"2024-09-21T05:40:05.245521+0000","last_undegraded":"2024-09-21T05:40:05.245521+0000","last_fullsized":"2024-09-21T05:40:05.245521+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:29.951231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245012+0000","last_change":"2024-09-21T05:40:03.597363+0000","last_active":"2024-09-21T05:40:05.245012+0000","last_peered":"2024-09-21T05:40:05.245012+0000","last_clean":"2024-09-21T05:40:05.245012+0000","last_became_active":"2024-09-21T05:40:03.596922+0000","last_became_peered":"2024-09-21T05:40:03.596922+0000","last_unstale":"2024-09-21T05:40:05.245012+0000","last_undegraded":"2024-09-21T05:40:05.245012+0000","last_fullsized":"2024-09-21T05:40:05.245012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:39:18.614162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245115+0000","last_change":"2024-09-21T05:40:02.582587+0000","last_active":"2024-09-21T05:40:05.245115+0000","last_peered":"2024-09-21T05:40:05.245115+0000","last_clean":"2024-09-21T05:40:05.245115+0000","last_became_active":"2024-09-21T05:40:02.580120+0000","last_became_peered":"2024-09-21T05:40:02.580120+0000","last_unstale":"2024-09-21T05:40:05.245115+0000","last_undegraded":"2024-09-21T05:40:05.245115+0000","last_fullsized":"2024-09-21T05:40:05.245115+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:20.438203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583421+0000","last_change":"2024-09-21T05:40:03.593160+0000","last_active":"2024-09-21T05:40:04.583421+0000","last_peered":"2024-09-21T05:40:04.583421+0000","last_clean":"2024-09-21T05:40:04.583421+0000","last_became_active":"2024-09-21T05:40:03.592817+0000","last_became_peered":"2024-09-21T05:40:03.592817+0000","last_unstale":"2024-09-21T05:40:04.583421+0000","last_undegraded":"2024-09-21T05:40:04.583421+0000","last_fullsized":"2024-09-21T05:40:04.583421+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:09.792067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585892+0000","last_change":"2024-09-21T05:40:02.578012+0000","last_active":"2024-09-21T05:40:04.585892+0000","last_peered":"2024-09-21T05:40:04.585892+0000","last_clean":"2024-09-21T05:40:04.585892+0000","last_became_active":"2024-09-21T05:40:02.577686+0000","last_became_peered":"2024-09-21T05:40:02.577686+0000","last_unstale":"2024-09-21T05:40:04.585892+0000","last_undegraded":"2024-09-21T05:40:04.585892+0000","last_fullsized":"2024-09-21T05:40:04.585892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:49.518408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244714+0000","last_change":"2024-09-21T05:40:03.588128+0000","last_active":"2024-09-21T05:40:05.244714+0000","last_peered":"2024-09-21T05:40:05.244714+0000","last_clean":"2024-09-21T05:40:05.244714+0000","last_became_active":"2024-09-21T05:40:03.587800+0000","last_became_peered":"2024-09-21T05:40:03.587800+0000","last_unstale":"2024-09-21T05:40:05.244714+0000","last_undegraded":"2024-09-21T05:40:05.244714+0000","last_fullsized":"2024-09-21T05:40:05.244714+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:19.540979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583319+0000","last_change":"2024-09-21T05:40:02.579425+0000","last_active":"2024-09-21T05:40:04.583319+0000","last_peered":"2024-09-21T05:40:04.583319+0000","last_clean":"2024-09-21T05:40:04.583319+0000","last_became_active":"2024-09-21T05:40:02.579121+0000","last_became_peered":"2024-09-21T05:40:02.579121+0000","last_unstale":"2024-09-21T05:40:04.583319+0000","last_undegraded":"2024-09-21T05:40:04.583319+0000","last_fullsized":"2024-09-21T05:40:04.583319+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:17.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244691+0000","last_change":"2024-09-21T05:40:03.589530+0000","last_active":"2024-09-21T05:40:05.244691+0000","last_peered":"2024-09-21T05:40:05.244691+0000","last_clean":"2024-09-21T05:40:05.244691+0000","last_became_active":"2024-09-21T05:40:03.589231+0000","last_became_peered":"2024-09-21T05:40:03.589231+0000","last_unstale":"2024-09-21T05:40:05.244691+0000","last_undegraded":"2024-09-21T05:40:05.244691+0000","last_fullsized":"2024-09-21T05:40:05.244691+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:34.321587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244414+0000","last_change":"2024-09-21T05:40:02.583950+0000","last_active":"2024-09-21T05:40:05.244414+0000","last_peered":"2024-09-21T05:40:05.244414+0000","last_clean":"2024-09-21T05:40:05.244414+0000","last_became_active":"2024-09-21T05:40:02.583565+0000","last_became_peered":"2024-09-21T05:40:02.583565+0000","last_unstale":"2024-09-21T05:40:05.244414+0000","last_undegraded":"2024-09-21T05:40:05.244414+0000","last_fullsized":"2024-09-21T05:40:05.244414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:24.340836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584985+0000","last_change":"2024-09-21T05:40:03.574794+0000","last_active":"2024-09-21T05:40:04.584985+0000","last_peered":"2024-09-21T05:40:04.584985+0000","last_clean":"2024-09-21T05:40:04.584985+0000","last_became_active":"2024-09-21T05:40:03.574445+0000","last_became_peered":"2024-09-21T05:40:03.574445+0000","last_unstale":"2024-09-21T05:40:04.584985+0000","last_undegraded":"2024-09-21T05:40:04.584985+0000","last_fullsized":"2024-09-21T05:40:04.584985+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:25.544807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":23,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244295+0000","last_change":"2024-09-21T05:40:02.580478+0000","last_active":"2024-09-21T05:40:05.244295+0000","last_peered":"2024-09-21T05:40:05.244295+0000","last_clean":"2024-09-21T05:40:05.244295+0000","last_became_active":"2024-09-21T05:40:02.580172+0000","last_became_peered":"2024-09-21T05:40:02.580172+0000","last_unstale":"2024-09-21T05:40:05.244295+0000","last_undegraded":"2024-09-21T05:40:05.244295+0000","last_fullsized":"2024-09-21T05:40:05.244295+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:24.539189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584880+0000","last_change":"2024-09-21T05:40:03.591539+0000","last_active":"2024-09-21T05:40:04.584880+0000","last_peered":"2024-09-21T05:40:04.584880+0000","last_clean":"2024-09-21T05:40:04.584880+0000","last_became_active":"2024-09-21T05:40:03.591187+0000","last_became_peered":"2024-09-21T05:40:03.591187+0000","last_unstale":"2024-09-21T05:40:04.584880+0000","last_undegraded":"2024-09-21T05:40:04.584880+0000","last_fullsized":"2024-09-21T05:40:04.584880+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:01:41.485992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584285+0000","last_change":"2024-09-21T05:40:02.586220+0000","last_active":"2024-09-21T05:40:04.584285+0000","last_peered":"2024-09-21T05:40:04.584285+0000","last_clean":"2024-09-21T05:40:04.584285+0000","last_became_active":"2024-09-21T05:40:02.585561+0000","last_became_peered":"2024-09-21T05:40:02.585561+0000","last_unstale":"2024-09-21T05:40:04.584285+0000","last_undegraded":"2024-09-21T05:40:04.584285+0000","last_fullsized":"2024-09-21T05:40:04.584285+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:01.014995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245528+0000","last_change":"2024-09-21T05:40:03.591303+0000","last_active":"2024-09-21T05:40:05.245528+0000","last_peered":"2024-09-21T05:40:05.245528+0000","last_clean":"2024-09-21T05:40:05.245528+0000","last_became_active":"2024-09-21T05:40:03.590934+0000","last_became_peered":"2024-09-21T05:40:03.590934+0000","last_unstale":"2024-09-21T05:40:05.245528+0000","last_undegraded":"2024-09-21T05:40:05.245528+0000","last_fullsized":"2024-09-21T05:40:05.245528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:27.355864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585011+0000","last_change":"2024-09-21T05:40:02.576280+0000","last_active":"2024-09-21T05:40:04.585011+0000","last_peered":"2024-09-21T05:40:04.585011+0000","last_clean":"2024-09-21T05:40:04.585011+0000","last_became_active":"2024-09-21T05:40:02.576000+0000","last_became_peered":"2024-09-21T05:40:02.576000+0000","last_unstale":"2024-09-21T05:40:04.585011+0000","last_undegraded":"2024-09-21T05:40:04.585011+0000","last_fullsized":"2024-09-21T05:40:04.585011+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:25.979410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583610+0000","last_change":"2024-09-21T05:40:03.579661+0000","last_active":"2024-09-21T05:40:04.583610+0000","last_peered":"2024-09-21T05:40:04.583610+0000","last_clean":"2024-09-21T05:40:04.583610+0000","last_became_active":"2024-09-21T05:40:03.577844+0000","last_became_peered":"2024-09-21T05:40:03.577844+0000","last_unstale":"2024-09-21T05:40:04.583610+0000","last_undegraded":"2024-09-21T05:40:04.583610+0000","last_fullsized":"2024-09-21T05:40:04.583610+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:13.602757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584721+0000","last_change":"2024-09-21T05:40:02.575810+0000","last_active":"2024-09-21T05:40:04.584721+0000","last_peered":"2024-09-21T05:40:04.584721+0000","last_clean":"2024-09-21T05:40:04.584721+0000","last_became_active":"2024-09-21T05:40:02.575507+0000","last_became_peered":"2024-09-21T05:40:02.575507+0000","last_unstale":"2024-09-21T05:40:04.584721+0000","last_undegraded":"2024-09-21T05:40:04.584721+0000","last_fullsized":"2024-09-21T05:40:04.584721+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:24.161211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163673+0000","last_change":"2024-09-21T05:40:03.590622+0000","last_active":"2024-09-21T05:40:05.163673+0000","last_peered":"2024-09-21T05:40:05.163673+0000","last_clean":"2024-09-21T05:40:05.163673+0000","last_became_active":"2024-09-21T05:40:03.590218+0000","last_became_peered":"2024-09-21T05:40:03.590218+0000","last_unstale":"2024-09-21T05:40:05.163673+0000","last_undegraded":"2024-09-21T05:40:05.163673+0000","last_fullsized":"2024-09-21T05:40:05.163673+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:58.082728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.585867+0000","last_change":"2024-09-21T05:40:02.582440+0000","last_active":"2024-09-21T05:40:04.585867+0000","last_peered":"2024-09-21T05:40:04.585867+0000","last_clean":"2024-09-21T05:40:04.585867+0000","last_became_active":"2024-09-21T05:40:02.582141+0000","last_became_peered":"2024-09-21T05:40:02.582141+0000","last_unstale":"2024-09-21T05:40:04.585867+0000","last_undegraded":"2024-09-21T05:40:04.585867+0000","last_fullsized":"2024-09-21T05:40:04.585867+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.443912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163364+0000","last_change":"2024-09-21T05:40:03.583528+0000","last_active":"2024-09-21T05:40:05.163364+0000","last_peered":"2024-09-21T05:40:05.163364+0000","last_clean":"2024-09-21T05:40:05.163364+0000","last_became_active":"2024-09-21T05:40:03.583147+0000","last_became_peered":"2024-09-21T05:40:03.583147+0000","last_unstale":"2024-09-21T05:40:05.163364+0000","last_undegraded":"2024-09-21T05:40:05.163364+0000","last_fullsized":"2024-09-21T05:40:05.163364+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:43.561897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587821+0000","last_change":"2024-09-21T05:40:02.585866+0000","last_active":"2024-09-21T05:40:04.587821+0000","last_peered":"2024-09-21T05:40:04.587821+0000","last_clean":"2024-09-21T05:40:04.587821+0000","last_became_active":"2024-09-21T05:40:02.585588+0000","last_became_peered":"2024-09-21T05:40:02.585588+0000","last_unstale":"2024-09-21T05:40:04.587821+0000","last_undegraded":"2024-09-21T05:40:04.587821+0000","last_fullsized":"2024-09-21T05:40:04.587821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:30.409276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163667+0000","last_change":"2024-09-21T05:40:03.587182+0000","last_active":"2024-09-21T05:40:05.163667+0000","last_peered":"2024-09-21T05:40:05.163667+0000","last_clean":"2024-09-21T05:40:05.163667+0000","last_became_active":"2024-09-21T05:40:03.584816+0000","last_became_peered":"2024-09-21T05:40:03.584816+0000","last_unstale":"2024-09-21T05:40:05.163667+0000","last_undegraded":"2024-09-21T05:40:05.163667+0000","last_fullsized":"2024-09-21T05:40:05.163667+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:22.728465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584922+0000","last_change":"2024-09-21T05:40:02.580632+0000","last_active":"2024-09-21T05:40:04.584922+0000","last_peered":"2024-09-21T05:40:04.584922+0000","last_clean":"2024-09-21T05:40:04.584922+0000","last_became_active":"2024-09-21T05:40:02.580318+0000","last_became_peered":"2024-09-21T05:40:02.580318+0000","last_unstale":"2024-09-21T05:40:04.584922+0000","last_undegraded":"2024-09-21T05:40:04.584922+0000","last_fullsized":"2024-09-21T05:40:04.584922+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:02.332464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163383+0000","last_change":"2024-09-21T05:40:03.583922+0000","last_active":"2024-09-21T05:40:05.163383+0000","last_peered":"2024-09-21T05:40:05.163383+0000","last_clean":"2024-09-21T05:40:05.163383+0000","last_became_active":"2024-09-21T05:40:03.583568+0000","last_became_peered":"2024-09-21T05:40:03.583568+0000","last_unstale":"2024-09-21T05:40:05.163383+0000","last_undegraded":"2024-09-21T05:40:05.163383+0000","last_fullsized":"2024-09-21T05:40:05.163383+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:45:31.304310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.586034+0000","last_change":"2024-09-21T05:40:02.581966+0000","last_active":"2024-09-21T05:40:04.586034+0000","last_peered":"2024-09-21T05:40:04.586034+0000","last_clean":"2024-09-21T05:40:04.586034+0000","last_became_active":"2024-09-21T05:40:02.581643+0000","last_became_peered":"2024-09-21T05:40:02.581643+0000","last_unstale":"2024-09-21T05:40:04.586034+0000","last_undegraded":"2024-09-21T05:40:04.586034+0000","last_fullsized":"2024-09-21T05:40:04.586034+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:17.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244504+0000","last_change":"2024-09-21T05:40:03.592685+0000","last_active":"2024-09-21T05:40:05.244504+0000","last_peered":"2024-09-21T05:40:05.244504+0000","last_clean":"2024-09-21T05:40:05.244504+0000","last_became_active":"2024-09-21T05:40:03.592344+0000","last_became_peered":"2024-09-21T05:40:03.592344+0000","last_unstale":"2024-09-21T05:40:05.244504+0000","last_undegraded":"2024-09-21T05:40:05.244504+0000","last_fullsized":"2024-09-21T05:40:05.244504+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:45.673952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244187+0000","last_change":"2024-09-21T05:40:02.583065+0000","last_active":"2024-09-21T05:40:05.244187+0000","last_peered":"2024-09-21T05:40:05.244187+0000","last_clean":"2024-09-21T05:40:05.244187+0000","last_became_active":"2024-09-21T05:40:02.582693+0000","last_became_peered":"2024-09-21T05:40:02.582693+0000","last_unstale":"2024-09-21T05:40:05.244187+0000","last_undegraded":"2024-09-21T05:40:05.244187+0000","last_fullsized":"2024-09-21T05:40:05.244187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:33.159567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.163687+0000","last_change":"2024-09-21T05:40:03.594519+0000","last_active":"2024-09-21T05:40:05.163687+0000","last_peered":"2024-09-21T05:40:05.163687+0000","last_clean":"2024-09-21T05:40:05.163687+0000","last_became_active":"2024-09-21T05:40:03.594162+0000","last_became_peered":"2024-09-21T05:40:03.594162+0000","last_unstale":"2024-09-21T05:40:05.163687+0000","last_undegraded":"2024-09-21T05:40:05.163687+0000","last_fullsized":"2024-09-21T05:40:05.163687+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:11.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584299+0000","last_change":"2024-09-21T05:40:02.586383+0000","last_active":"2024-09-21T05:40:04.584299+0000","last_peered":"2024-09-21T05:40:04.584299+0000","last_clean":"2024-09-21T05:40:04.584299+0000","last_became_active":"2024-09-21T05:40:02.585939+0000","last_became_peered":"2024-09-21T05:40:02.585939+0000","last_unstale":"2024-09-21T05:40:04.584299+0000","last_undegraded":"2024-09-21T05:40:04.584299+0000","last_fullsized":"2024-09-21T05:40:04.584299+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:28.955499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245711+0000","last_change":"2024-09-21T05:40:03.597292+0000","last_active":"2024-09-21T05:40:05.245711+0000","last_peered":"2024-09-21T05:40:05.245711+0000","last_clean":"2024-09-21T05:40:05.245711+0000","last_became_active":"2024-09-21T05:40:03.596914+0000","last_became_peered":"2024-09-21T05:40:03.596914+0000","last_unstale":"2024-09-21T05:40:05.245711+0000","last_undegraded":"2024-09-21T05:40:05.245711+0000","last_fullsized":"2024-09-21T05:40:05.245711+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:07.345872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.589442+0000","last_change":"2024-09-21T05:40:02.580955+0000","last_active":"2024-09-21T05:40:04.589442+0000","last_peered":"2024-09-21T05:40:04.589442+0000","last_clean":"2024-09-21T05:40:04.589442+0000","last_became_active":"2024-09-21T05:40:02.580625+0000","last_became_peered":"2024-09-21T05:40:02.580625+0000","last_unstale":"2024-09-21T05:40:04.589442+0000","last_undegraded":"2024-09-21T05:40:04.589442+0000","last_fullsized":"2024-09-21T05:40:04.589442+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:06.416483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587745+0000","last_change":"2024-09-21T05:40:03.597758+0000","last_active":"2024-09-21T05:40:04.587745+0000","last_peered":"2024-09-21T05:40:04.587745+0000","last_clean":"2024-09-21T05:40:04.587745+0000","last_became_active":"2024-09-21T05:40:03.597153+0000","last_became_peered":"2024-09-21T05:40:03.597153+0000","last_unstale":"2024-09-21T05:40:04.587745+0000","last_undegraded":"2024-09-21T05:40:04.587745+0000","last_fullsized":"2024-09-21T05:40:04.587745+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:05.176357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583744+0000","last_change":"2024-09-21T05:40:02.580377+0000","last_active":"2024-09-21T05:40:04.583744+0000","last_peered":"2024-09-21T05:40:04.583744+0000","last_clean":"2024-09-21T05:40:04.583744+0000","last_became_active":"2024-09-21T05:40:02.580032+0000","last_became_peered":"2024-09-21T05:40:02.580032+0000","last_unstale":"2024-09-21T05:40:04.583744+0000","last_undegraded":"2024-09-21T05:40:04.583744+0000","last_fullsized":"2024-09-21T05:40:04.583744+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:31.133340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584784+0000","last_change":"2024-09-21T05:40:03.586169+0000","last_active":"2024-09-21T05:40:04.584784+0000","last_peered":"2024-09-21T05:40:04.584784+0000","last_clean":"2024-09-21T05:40:04.584784+0000","last_became_active":"2024-09-21T05:40:03.585827+0000","last_became_peered":"2024-09-21T05:40:03.585827+0000","last_unstale":"2024-09-21T05:40:04.584784+0000","last_undegraded":"2024-09-21T05:40:04.584784+0000","last_fullsized":"2024-09-21T05:40:04.584784+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:41.217251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587957+0000","last_change":"2024-09-21T05:40:02.585651+0000","last_active":"2024-09-21T05:40:04.587957+0000","last_peered":"2024-09-21T05:40:04.587957+0000","last_clean":"2024-09-21T05:40:04.587957+0000","last_became_active":"2024-09-21T05:40:02.585338+0000","last_became_peered":"2024-09-21T05:40:02.585338+0000","last_unstale":"2024-09-21T05:40:04.587957+0000","last_undegraded":"2024-09-21T05:40:04.587957+0000","last_fullsized":"2024-09-21T05:40:04.587957+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:20.805434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.587893+0000","last_change":"2024-09-21T05:40:03.594038+0000","last_active":"2024-09-21T05:40:04.587893+0000","last_peered":"2024-09-21T05:40:04.587893+0000","last_clean":"2024-09-21T05:40:04.587893+0000","last_became_active":"2024-09-21T05:40:03.593643+0000","last_became_peered":"2024-09-21T05:40:03.593643+0000","last_unstale":"2024-09-21T05:40:04.587893+0000","last_undegraded":"2024-09-21T05:40:04.587893+0000","last_fullsized":"2024-09-21T05:40:04.587893+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:16.968181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.583440+0000","last_change":"2024-09-21T05:40:02.587534+0000","last_active":"2024-09-21T05:40:04.583440+0000","last_peered":"2024-09-21T05:40:04.583440+0000","last_clean":"2024-09-21T05:40:04.583440+0000","last_became_active":"2024-09-21T05:40:02.586822+0000","last_became_peered":"2024-09-21T05:40:02.586822+0000","last_unstale":"2024-09-21T05:40:04.583440+0000","last_undegraded":"2024-09-21T05:40:04.583440+0000","last_fullsized":"2024-09-21T05:40:04.583440+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.443912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244356+0000","last_change":"2024-09-21T05:40:03.582258+0000","last_active":"2024-09-21T05:40:05.244356+0000","last_peered":"2024-09-21T05:40:05.244356+0000","last_clean":"2024-09-21T05:40:05.244356+0000","last_became_active":"2024-09-21T05:40:03.581871+0000","last_became_peered":"2024-09-21T05:40:03.581871+0000","last_unstale":"2024-09-21T05:40:05.244356+0000","last_undegraded":"2024-09-21T05:40:05.244356+0000","last_fullsized":"2024-09-21T05:40:05.244356+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:17.462467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.586187+0000","last_change":"2024-09-21T05:40:02.581927+0000","last_active":"2024-09-21T05:40:04.586187+0000","last_peered":"2024-09-21T05:40:04.586187+0000","last_clean":"2024-09-21T05:40:04.586187+0000","last_became_active":"2024-09-21T05:40:02.581208+0000","last_became_peered":"2024-09-21T05:40:02.581208+0000","last_unstale":"2024-09-21T05:40:04.586187+0000","last_undegraded":"2024-09-21T05:40:04.586187+0000","last_fullsized":"2024-09-21T05:40:04.586187+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:09:57.096716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.586147+0000","last_change":"2024-09-21T05:40:03.578676+0000","last_active":"2024-09-21T05:40:04.586147+0000","last_peered":"2024-09-21T05:40:04.586147+0000","last_clean":"2024-09-21T05:40:04.586147+0000","last_became_active":"2024-09-21T05:40:03.578339+0000","last_became_peered":"2024-09-21T05:40:03.578339+0000","last_unstale":"2024-09-21T05:40:04.586147+0000","last_undegraded":"2024-09-21T05:40:04.586147+0000","last_fullsized":"2024-09-21T05:40:04.586147+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:46.211686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244461+0000","last_change":"2024-09-21T05:40:02.582884+0000","last_active":"2024-09-21T05:40:05.244461+0000","last_peered":"2024-09-21T05:40:05.244461+0000","last_clean":"2024-09-21T05:40:05.244461+0000","last_became_active":"2024-09-21T05:40:02.582586+0000","last_became_peered":"2024-09-21T05:40:02.582586+0000","last_unstale":"2024-09-21T05:40:05.244461+0000","last_undegraded":"2024-09-21T05:40:05.244461+0000","last_fullsized":"2024-09-21T05:40:05.244461+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:10.954780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.245028+0000","last_change":"2024-09-21T05:40:03.597485+0000","last_active":"2024-09-21T05:40:05.245028+0000","last_peered":"2024-09-21T05:40:05.245028+0000","last_clean":"2024-09-21T05:40:05.245028+0000","last_became_active":"2024-09-21T05:40:03.597164+0000","last_became_peered":"2024-09-21T05:40:03.597164+0000","last_unstale":"2024-09-21T05:40:05.245028+0000","last_undegraded":"2024-09-21T05:40:05.245028+0000","last_fullsized":"2024-09-21T05:40:05.245028+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:02.558348+0000","last_clean_scrub_stamp":"2024-09-21T05:40:02.558348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:43.378842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:04.584477+0000","last_change":"2024-09-21T05:40:02.580170+0000","last_active":"2024-09-21T05:40:04.584477+0000","last_peered":"2024-09-21T05:40:04.584477+0000","last_clean":"2024-09-21T05:40:04.584477+0000","last_became_active":"2024-09-21T05:40:02.579822+0000","last_became_peered":"2024-09-21T05:40:02.579822+0000","last_unstale":"2024-09-21T05:40:04.584477+0000","last_undegraded":"2024-09-21T05:40:04.584477+0000","last_fullsized":"2024-09-21T05:40:04.584477+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:01.552730+0000","last_clean_scrub_stamp":"2024-09-21T05:40:01.552730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:10.529215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":129,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-21T05:40:05.244766+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:40:05.244766+0000","last_peered":"2024-09-21T05:40:05.244766+0000","last_clean":"2024-09-21T05:40:05.244766+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:40:05.244766+0000","last_undegraded":"2024-09-21T05:40:05.244766+0000","last_fullsized":"2024-09-21T05:40:05.244766+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":950272,"data_stored":727179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":360448,"data_stored":136825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":137337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":360448,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":132055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":344064,"data_stored":130491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35712,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336128,"statfs":{"total":96636764160,"available":96600195072,"internally_reserved":0,"allocated":368640,"data_stored":138355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":727089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:40:08.028 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:40:08.028 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:40:08.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 -- 172.21.15.22:0/3218345354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 msgr2=0x7fa4480d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 --2- 172.21.15.22:0/3218345354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4480d31e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa43c009830 tx=0x7fa43c02a460 comp rx=0 tx=0).stop 2024-09-21T05:40:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 -- 172.21.15.22:0/3218345354 shutdown_connections 2024-09-21T05:40:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 --2- 172.21.15.22:0/3218345354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4480d31e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 -- 172.21.15.22:0/3218345354 >> 172.21.15.22:0/3218345354 conn(0x7fa44809d830 msgr2=0x7fa4480d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 -- 172.21.15.22:0/3218345354 shutdown_connections 2024-09-21T05:40:08.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.187+0000 7fa44f240640 1 -- 172.21.15.22:0/3218345354 wait complete. 2024-09-21T05:40:08.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.189+0000 7fa44f240640 1 Processor -- start 2024-09-21T05:40:08.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.189+0000 7fa44f240640 1 -- start start 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44f240640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480b5fa0 0x7fa4481543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44f240640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4481510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44f240640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 0x7fa4480b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44f240640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa44806e340 con 0x7fa4480d4fd0 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44f240640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa44806e1c0 con 0x7fa4480b9cf0 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44f240640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa44806e4c0 con 0x7fa4480b5fa0 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.190+0000 7fa44d7dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 0x7fa4480b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 0x7fa4480b7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53666/0 (socket says 172.21.15.22:53666) 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 -- 172.21.15.22:0/2360547729 learned_addr learned my addr 172.21.15.22:0/2360547729 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa447fff640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4481510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44cfdc640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480b5fa0 0x7fa4481543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 -- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480b5fa0 msgr2=0x7fa4481543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480b5fa0 0x7fa4481543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 -- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 msgr2=0x7fa4481510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4481510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 -- 172.21.15.22:0/2360547729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4480cafe0 con 0x7fa4480d4fd0 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa447fff640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4481510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44d7dd640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 0x7fa4480b7b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c7a0 tx=0x7fa44000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440017070 con 0x7fa4480d4fd0 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa440005070 con 0x7fa4480d4fd0 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa448151ae0 con 0x7fa4480d4fd0 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4400054e0 con 0x7fa4480d4fd0 2024-09-21T05:40:08.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.191+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4480cad00 con 0x7fa4480d4fd0 2024-09-21T05:40:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.193+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44809e660 con 0x7fa4480d4fd0 2024-09-21T05:40:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.193+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa44001a070 con 0x7fa4480d4fd0 2024-09-21T05:40:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.194+0000 7fa4457fa640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa434078b90 0x7fa43407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.194+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa44009a380 con 0x7fa4480d4fd0 2024-09-21T05:40:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.194+0000 7fa44cfdc640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa434078b90 0x7fa43407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.195+0000 7fa44cfdc640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa434078b90 0x7fa43407b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4380014f0 tx=0x7fa438009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:08.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.198+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa440062180 con 0x7fa4480d4fd0 2024-09-21T05:40:08.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.457+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa44806e9f0 con 0x7fa4480d4fd0 2024-09-21T05:40:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.460+0000 7fa4457fa640 1 -- 172.21.15.22:0/2360547729 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fa440066bb0 con 0x7fa4480d4fd0 2024-09-21T05:40:08.462 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:40:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:40:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:40:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:40:08.462 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:40:08.463 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:40:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa434078b90 msgr2=0x7fa43407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa434078b90 0x7fa43407b0a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa4380014f0 tx=0x7fa438009040 comp rx=0 tx=0).stop 2024-09-21T05:40:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 msgr2=0x7fa4480b7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 0x7fa4480b7b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c7a0 tx=0x7fa44000cc60 comp rx=0 tx=0).stop 2024-09-21T05:40:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 shutdown_connections 2024-09-21T05:40:08.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa4480d4fd0 0x7fa4480b7b50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa4480b9cf0 0x7fa4481510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa434078b90 0x7fa43407b0a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 --2- 172.21.15.22:0/2360547729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa4480b5fa0 0x7fa4481543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 >> 172.21.15.22:0/2360547729 conn(0x7fa44809d830 msgr2=0x7fa448068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:08.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 shutdown_connections 2024-09-21T05:40:08.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.462+0000 7fa44f240640 1 -- 172.21.15.22:0/2360547729 wait complete. 2024-09-21T05:40:08.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:08.508 INFO:tasks.ceph.mds.a.smithi022.stderr:2024-09-21T05:40:08.506+0000 7f4d5997e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:40:08.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 -- 172.21.15.22:0/2783261997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 msgr2=0x7f15ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 --2- 172.21.15.22:0/2783261997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15ac00b9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f15b800c630 tx=0x7f15ac005830 comp rx=0 tx=0).stop 2024-09-21T05:40:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 -- 172.21.15.22:0/2783261997 shutdown_connections 2024-09-21T05:40:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 --2- 172.21.15.22:0/2783261997 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15ac00b9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 -- 172.21.15.22:0/2783261997 >> 172.21.15.22:0/2783261997 conn(0x7f15c006d230 msgr2=0x7f15c006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 -- 172.21.15.22:0/2783261997 shutdown_connections 2024-09-21T05:40:08.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.591+0000 7f15c6558640 1 -- 172.21.15.22:0/2783261997 wait complete. 2024-09-21T05:40:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.593+0000 7f15c6558640 1 Processor -- start 2024-09-21T05:40:08.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.593+0000 7f15c6558640 1 -- start start 2024-09-21T05:40:08.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.594+0000 7f15c6558640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15c00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.594+0000 7f15c6558640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15c01f5990 0x7f15c00ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.594+0000 7f15c6558640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c01f5da0 0x7f15c00caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15c6558640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f15c00ccc60 con 0x7f15ac005300 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15c6558640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15c015dcb0 con 0x7f15c01f5da0 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15c6558640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15c00ccde0 con 0x7f15c01f5990 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15c00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15c00b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53684/0 (socket says 172.21.15.22:53684) 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 -- 172.21.15.22:0/3666053835 learned_addr learned my addr 172.21.15.22:0/3666053835 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bf7fe640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15c01f5990 0x7f15c00ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15c4af5640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c01f5da0 0x7f15c00caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 -- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15c01f5990 msgr2=0x7f15c00ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15c01f5990 0x7f15c00ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 -- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c01f5da0 msgr2=0x7f15c00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c01f5da0 0x7f15c00caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 -- 172.21.15.22:0/3666053835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c00b1010 con 0x7f15ac005300 2024-09-21T05:40:08.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.595+0000 7f15bffff640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15c00b4530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f15b0003f00 tx=0x7f15b0001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.597+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0011640 con 0x7f15ac005300 2024-09-21T05:40:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.597+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15b0016020 con 0x7f15ac005300 2024-09-21T05:40:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.597+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c00b12f0 con 0x7f15ac005300 2024-09-21T05:40:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.597+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0011a80 con 0x7f15ac005300 2024-09-21T05:40:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.597+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c006e1d0 con 0x7f15ac005300 2024-09-21T05:40:08.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.598+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c00cd2f0 con 0x7f15ac005300 2024-09-21T05:40:08.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.599+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15b0011c20 con 0x7f15ac005300 2024-09-21T05:40:08.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.600+0000 7f15bcff9640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15a0078b90 0x7f15a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:08.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.600+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f15b009b7a0 con 0x7f15ac005300 2024-09-21T05:40:08.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.600+0000 7f15bf7fe640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15a0078b90 0x7f15a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:08.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.601+0000 7f15bf7fe640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15a0078b90 0x7f15a007b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f15c006fc90 tx=0x7f15a800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:08.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.606+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15b0020020 con 0x7f15ac005300 2024-09-21T05:40:08.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.832+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15c00d15b0 con 0x7f15ac005300 2024-09-21T05:40:08.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.833+0000 7f15bcff9640 1 -- 172.21.15.22:0/3666053835 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f15b0063620 con 0x7f15ac005300 2024-09-21T05:40:08.835 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:08.835 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:08.836 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15a0078b90 msgr2=0x7f15a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15a0078b90 0x7f15a007b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f15c006fc90 tx=0x7f15a800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 msgr2=0x7f15c00b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15c00b4530 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f15b0003f00 tx=0x7f15b0001630 comp rx=0 tx=0).stop 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 shutdown_connections 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15c01f5da0 0x7f15c00caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15a0078b90 0x7f15a007b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15c01f5990 0x7f15c00ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 --2- 172.21.15.22:0/3666053835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15ac005300 0x7f15c00b4530 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.835+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 >> 172.21.15.22:0/3666053835 conn(0x7f15c006d230 msgr2=0x7f15c015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.836+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 shutdown_connections 2024-09-21T05:40:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:08.836+0000 7f15c6558640 1 -- 172.21.15.22:0/3666053835 wait complete. 2024-09-21T05:40:09.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:40:02.941475+0000', 'modified': '2024-09-21T05:40:05.976698+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4926}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4926': {'gid': 4926, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6834/1454316694', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 1454316694}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 1454316694}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4926, 'qdb_cluster': [4926]} 2024-09-21T05:40:09.251 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:40:09.252 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T05:40:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.372+0000 7f9ddf694640 1 -- 172.21.15.22:0/512557698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 msgr2=0x7f9dd815c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.372+0000 7f9ddf694640 1 --2- 172.21.15.22:0/512557698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815c830 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9dd4009830 tx=0x7f9dd402a530 comp rx=0 tx=0).stop 2024-09-21T05:40:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.372+0000 7f9ddf694640 1 -- 172.21.15.22:0/512557698 shutdown_connections 2024-09-21T05:40:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.372+0000 7f9ddf694640 1 --2- 172.21.15.22:0/512557698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815c830 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.372+0000 7f9ddf694640 1 -- 172.21.15.22:0/512557698 >> 172.21.15.22:0/512557698 conn(0x7f9dd80cee20 msgr2=0x7f9dd80cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.373+0000 7f9ddf694640 1 -- 172.21.15.22:0/512557698 shutdown_connections 2024-09-21T05:40:09.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.373+0000 7f9ddf694640 1 -- 172.21.15.22:0/512557698 wait complete. 2024-09-21T05:40:09.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.374+0000 7f9ddf694640 1 Processor -- start 2024-09-21T05:40:09.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.374+0000 7f9ddf694640 1 -- start start 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddf694640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddf694640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dd8160c90 0x7f9dd80cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddf694640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dd81f76b0 0x7f9dd81f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddf694640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd80d1900 con 0x7f9dd80cff70 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddf694640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd80d1780 con 0x7f9dd8160c90 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddf694640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd80d1a80 con 0x7f9dd81f76b0 2024-09-21T05:40:09.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53700/0 (socket says 172.21.15.22:53700) 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 -- 172.21.15.22:0/2211100477 learned_addr learned my addr 172.21.15.22:0/2211100477 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dd8160c90 0x7f9dd80cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9dddc31640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dd81f76b0 0x7f9dd81f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 -- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dd81f76b0 msgr2=0x7f9dd81f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dd81f76b0 0x7f9dd81f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 -- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dd8160c90 msgr2=0x7f9dd80cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dd8160c90 0x7f9dd80cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 -- 172.21.15.22:0/2211100477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd815e490 con 0x7f9dd80cff70 2024-09-21T05:40:09.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9dddc31640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dd81f76b0 0x7f9dd81f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9ddd430640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815da60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ab20 tx=0x7f9dd4033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.376+0000 7f9dcdffb640 1 -- 172.21.15.22:0/2211100477 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4031b60 con 0x7f9dd80cff70 2024-09-21T05:40:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.377+0000 7f9dcdffb640 1 -- 172.21.15.22:0/2211100477 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd4031df0 con 0x7f9dd80cff70 2024-09-21T05:40:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.377+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd815e770 con 0x7f9dd80cff70 2024-09-21T05:40:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.377+0000 7f9dcdffb640 1 -- 172.21.15.22:0/2211100477 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4005280 con 0x7f9dd80cff70 2024-09-21T05:40:09.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.377+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd815ed30 con 0x7f9dd80cff70 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.379+0000 7f9dcdffb640 1 -- 172.21.15.22:0/2211100477 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9dd4005420 con 0x7f9dd80cff70 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.379+0000 7f9dcdffb640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9dbc078bc0 0x7f9dbc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.379+0000 7f9dcdffb640 1 -- 172.21.15.22:0/2211100477 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9dd40bbf50 con 0x7f9dd80cff70 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.379+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9dbc078bc0 0x7f9dbc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.380+0000 7f9de6600740 1 Processor -- start 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.380+0000 7f9de6600740 1 -- start start 2024-09-21T05:40:09.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.380+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9dbc078bc0 0x7f9dbc07b0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80b4830 tx=0x7f9dc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9de6600740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7a02970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7a10e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9de6600740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5640d795c820 con 0x5640d7642270 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5640d794bc50 con 0x5640d7621590 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5640d794fa60 con 0x5640d76193b0 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9ddd430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7a02970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9ddd430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7a02970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53712/0 (socket says 172.21.15.22:53712) 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9ddd430640 1 -- 172.21.15.22:0/3240488668 learned_addr learned my addr 172.21.15.22:0/3240488668 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7a10e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.381+0000 7f9dddc31640 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9ddd430640 1 -- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 msgr2=0x5640d7a10e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7a10e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9ddd430640 1 -- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 msgr2=0x5640d7a113b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a113b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9ddd430640 1 -- 172.21.15.22:0/3240488668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d76ad850 con 0x5640d7642270 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7a02970 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40ba4a0 tx=0x7f9dd40837e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38898d303b035ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9dc2ffd640 1 -- 172.21.15.22:0/3240488668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4048030 con 0x5640d7642270 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9dc2ffd640 1 -- 172.21.15.22:0/3240488668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd40aa0d0 con 0x5640d7642270 2024-09-21T05:40:09.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.382+0000 7f9dc2ffd640 1 -- 172.21.15.22:0/3240488668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4031770 con 0x5640d7642270 2024-09-21T05:40:09.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 -- 172.21.15.22:0/3240488668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 msgr2=0x5640d7a02970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7a02970 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40ba4a0 tx=0x7f9dd40837e0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 -- 172.21.15.22:0/3240488668 shutdown_connections 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7a10e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 --2- 172.21.15.22:0/3240488668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7a02970 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 -- 172.21.15.22:0/3240488668 >> 172.21.15.22:0/3240488668 conn(0x5640d76363c0 msgr2=0x5640d79ea260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 -- 172.21.15.22:0/3240488668 shutdown_connections 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.383+0000 7f9de6600740 1 -- 172.21.15.22:0/3240488668 wait complete. 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.384+0000 7f9de6600740 1 -- start start 2024-09-21T05:40:09.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.384+0000 7f9de6600740 1 Processor -- start 2024-09-21T05:40:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d76193b0 0x5640d7a26670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 0x5640d7a27dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5640d76bc620 con 0x5640d7621590 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5640d78f15f0 con 0x5640d76193b0 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5640d7a43a80 con 0x5640d7642270 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9de6600740 10 client.0 _ll_register_callbacks cb 0x5640d78f50a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9ddcc2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 0x5640d7a27dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9ddcc2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 0x5640d7a27dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53720/0 (socket says 172.21.15.22:53720) 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.385+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/3114421524 learned_addr learned my addr 172.21.15.22:0/3114421524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d76193b0 0x5640d7a26670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9dddc31640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 msgr2=0x5640d79e89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d76193b0 msgr2=0x5640d7a26670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d76193b0 0x5640d7a26670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/3114421524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d77246b0 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d76193b0 0x5640d7a26670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9dddc31640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 0x5640d7a27dc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8014730 tx=0x7f9dc8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b9e9bec25caf3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9db7fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9db7fff640 1 -- 172.21.15.22:0/3114421524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8025020 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9db7fff640 1 -- 172.21.15.22:0/3114421524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9dc80216d0 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd40aa0d0 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9db7fff640 1 -- 172.21.15.22:0/3114421524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9dc802f050 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9de6600740 10 client.5082 fetch_fsmap learned FSMap version 17 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5640d7a26bb0 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.386+0000 7f9db7fff640 1 -- 172.21.15.22:0/3114421524 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd40aa0d0 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9db7fff640 1 -- 172.21.15.22:0/3114421524 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f9dc8032020 con 0x5640d7621590 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9de6600740 10 client.5082 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T05:40:09.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9de6600740 10 client.5082 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9de6600740 20 client.5082 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9de6600740 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d7600e40 0x5640d7a4ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9de6600740 4 client.5082 mds_command: new command op to 4926 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 --> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d795e770 con 0x5640d7600e40 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d7600e40 0x5640d7a4ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9db7fff640 10 client.5082 ms_handle_connect on v2:172.21.15.22:6832/1454316694 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.387+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d7600e40 0x5640d7a4ffd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bbdb39f7a089 server_cookie=f8d0b79fba4fc59c in_seq=0 out_seq=0 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9db7fff640 1 -- 172.21.15.22:0/3114421524 <== mds.0 v2:172.21.15.22:6832/1454316694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5640d795e770 con 0x5640d7600e40 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9db7fff640 10 client.5082 handle_command_reply: tid=0 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 client.5082 shutdown 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 msgr2=0x5640d7a27dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 0x5640d7a27dc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8014730 tx=0x7f9dc8000ed0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 shutdown_connections 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d7600e40 0x5640d7a4ffd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7621590 0x5640d7a27dc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9db7fff640 0 client.5082 ms_handle_reset on v2:172.21.15.22:6832/1454316694 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 --2- 172.21.15.22:0/3114421524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d76193b0 0x5640d7a26670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 >> 172.21.15.22:0/3114421524 conn(0x5640d76363c0 msgr2=0x5640d7a02030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 shutdown_connections 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.388+0000 7f9de6600740 1 -- 172.21.15.22:0/3114421524 wait complete. 2024-09-21T05:40:09.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.389+0000 7f9de6600740 20 client.5082 trim_cache size 0 max 16384 2024-09-21T05:40:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.407+0000 7f9de6600740 1 Processor -- start 2024-09-21T05:40:09.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.407+0000 7f9de6600740 1 -- start start 2024-09-21T05:40:09.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.408+0000 7f9de6600740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 0x5640d7a110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.408+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d79e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.408+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.408+0000 7f9de6600740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd806f4e0 con 0x5640d76193b0 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.408+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd806e540 con 0x5640d7621590 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.408+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd81f3e10 con 0x5640d7642270 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 0x5640d7a110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 0x5640d7a110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53722/0 (socket says 172.21.15.22:53722) 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 -- 172.21.15.22:0/3777609657 learned_addr learned my addr 172.21.15.22:0/3777609657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 -- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 msgr2=0x5640d79e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 -- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 msgr2=0x5640d79e3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d79e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d79e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 -- 172.21.15.22:0/3777609657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d7a04080 con 0x5640d76193b0 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9ddd430640 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 0x5640d7a110c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40ba420 tx=0x7f9dd402a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a505f533c03a548 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9dc27fc640 1 -- 172.21.15.22:0/3777609657 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4048070 con 0x5640d76193b0 2024-09-21T05:40:09.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9dc27fc640 1 -- 172.21.15.22:0/3777609657 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd4083dd0 con 0x5640d76193b0 2024-09-21T05:40:09.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.409+0000 7f9dc27fc640 1 -- 172.21.15.22:0/3777609657 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4088800 con 0x5640d76193b0 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 -- 172.21.15.22:0/3777609657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 msgr2=0x5640d7a110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 0x5640d7a110c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9dd40ba420 tx=0x7f9dd402a530 comp rx=0 tx=0).stop 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 -- 172.21.15.22:0/3777609657 shutdown_connections 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d7642270 0x5640d79e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d79e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 --2- 172.21.15.22:0/3777609657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d76193b0 0x5640d7a110c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 -- 172.21.15.22:0/3777609657 >> 172.21.15.22:0/3777609657 conn(0x5640d76363c0 msgr2=0x5640d7a251e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 -- 172.21.15.22:0/3777609657 shutdown_connections 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.410+0000 7f9de6600740 1 -- 172.21.15.22:0/3777609657 wait complete. 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.411+0000 7f9de6600740 1 -- start start 2024-09-21T05:40:09.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.411+0000 7f9de6600740 1 Processor -- start 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7ab0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a27ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7ab3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5640d7a28300 con 0x5640d7642270 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd806def0 con 0x5640d7621590 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5640d79e9620 con 0x5640d76193b0 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.412+0000 7f9de6600740 10 client.0 _ll_register_callbacks cb 0x7f9dc8001440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:09.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9dddc31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7ab3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9dddc31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7ab3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53732/0 (socket says 172.21.15.22:53732) 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddd430640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7ab0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a27ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9dddc31640 1 -- 172.21.15.22:0/1059446705 learned_addr learned my addr 172.21.15.22:0/1059446705 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 msgr2=0x5640d7ab0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7ab0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 msgr2=0x5640d7ab3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7ab3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 -- 172.21.15.22:0/1059446705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d79dfb70 con 0x5640d7621590 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9dddc31640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7ab3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddd430640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7ab0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9ddcc2f640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a27ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5640d7a142c0 tx=0x7f9dc802bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac5b9ddcb466c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9db7fff640 10 client.0 ms_handle_connect on v2:172.21.15.134:3300/0 2024-09-21T05:40:09.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.413+0000 7f9db7fff640 1 -- 172.21.15.22:0/1059446705 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8037070 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd4083dd0 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9db7fff640 1 -- 172.21.15.22:0/1059446705 <== mon.1 v2:172.21.15.134:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f9dc80216d0 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9db7fff640 1 -- 172.21.15.22:0/1059446705 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9dc8042050 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 10 client.4936 fetch_fsmap learned FSMap version 17 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5640d7ab3c80 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9db7fff640 1 -- 172.21.15.22:0/1059446705 <== mon.1 v2:172.21.15.134:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dd4083dd0 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9db7fff640 1 -- 172.21.15.22:0/1059446705 <== mon.1 v2:172.21.15.134:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f9dc8045020 con 0x5640d7621590 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 10 client.4936 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 10 client.4936 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 20 client.4936 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d79e38d0 0x5640d7a30730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 4 client.4936 mds_command: new command op to 4926 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 --> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] -- command(tid 0: {"prefix": "status"}) -- 0x5640d795a220 con 0x5640d79e38d0 2024-09-21T05:40:09.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.414+0000 7f9ddd430640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d79e38d0 0x5640d7a30730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9db7fff640 10 client.4936 ms_handle_connect on v2:172.21.15.22:6832/1454316694 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9ddd430640 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d79e38d0 0x5640d7a30730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b2ff317657e2de server_cookie=379ef16a697c04a3 in_seq=0 out_seq=0 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9db7fff640 1 -- 172.21.15.22:0/1059446705 <== mds.0 v2:172.21.15.22:6832/1454316694 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5640d795a220 con 0x5640d79e38d0 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9db7fff640 10 client.4936 handle_command_reply: tid=0 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 client.4936 shutdown 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 msgr2=0x5640d7a27ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a27ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x5640d7a142c0 tx=0x7f9dc802bc40 comp rx=0 tx=0).stop 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 shutdown_connections 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:6832/1454316694,v1:172.21.15.22:6834/1454316694] conn(0x5640d79e38d0 0x5640d7a30730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5640d7642270 0x5640d7ab3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5640d7621590 0x5640d7a27ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 --2- 172.21.15.22:0/1059446705 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5640d76193b0 0x5640d7ab0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9db7fff640 0 client.4936 ms_handle_reset on v2:172.21.15.22:6832/1454316694 2024-09-21T05:40:09.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 >> 172.21.15.22:0/1059446705 conn(0x5640d76363c0 msgr2=0x5640d79e5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 shutdown_connections 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 1 -- 172.21.15.22:0/1059446705 wait complete. 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.415+0000 7f9de6600740 20 client.4936 trim_cache size 0 max 16384 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 4926, 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:40:09.422 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.8385179489999999, 2024-09-21T05:40:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 17, 2024-09-21T05:40:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 32, 2024-09-21T05:40:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 32, 2024-09-21T05:40:09.423 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 12.752062107 2024-09-21T05:40:09.423 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9dbc078bc0 msgr2=0x7f9dbc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9dbc078bc0 0x7f9dbc07b0d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80b4830 tx=0x7f9dc8009040 comp rx=0 tx=0).stop 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 msgr2=0x7f9dd815da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815da60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ab20 tx=0x7f9dd4033c10 comp rx=0 tx=0).stop 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 shutdown_connections 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9dbc078bc0 0x7f9dbc07b0d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9dd81f76b0 0x7f9dd81f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9dd8160c90 0x7f9dd80cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 --2- 172.21.15.22:0/2211100477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9dd80cff70 0x7f9dd815da60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 >> 172.21.15.22:0/2211100477 conn(0x7f9dd80cee20 msgr2=0x7f9dd8072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 shutdown_connections 2024-09-21T05:40:09.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.416+0000 7f9ddf694640 1 -- 172.21.15.22:0/2211100477 wait complete. 2024-09-21T05:40:09.429 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:40:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.543+0000 7f616140e640 1 -- 172.21.15.22:0/3887568390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 msgr2=0x7f6148004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.543+0000 7f616140e640 1 --2- 172.21.15.22:0/3887568390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f6148004800 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6144009b20 tx=0x7f6148005830 comp rx=0 tx=0).stop 2024-09-21T05:40:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.543+0000 7f615b7fe640 1 -- 172.21.15.22:0/3887568390 reap_dead start 2024-09-21T05:40:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.544+0000 7f616140e640 1 -- 172.21.15.22:0/3887568390 shutdown_connections 2024-09-21T05:40:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.544+0000 7f616140e640 1 -- 172.21.15.22:0/3887568390 >> 172.21.15.22:0/3887568390 conn(0x7f615c09d840 msgr2=0x7f615c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.544+0000 7f616140e640 1 -- 172.21.15.22:0/3887568390 shutdown_connections 2024-09-21T05:40:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.544+0000 7f616140e640 1 -- 172.21.15.22:0/3887568390 wait complete. 2024-09-21T05:40:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.545+0000 7f616140e640 1 Processor -- start 2024-09-21T05:40:09.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.546+0000 7f616140e640 1 -- start start 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f616140e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f615c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f616140e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f615c1f8900 0x7f615c1f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f616140e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f615c1f9250 0x7f615c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f616140e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f615c06eae0 con 0x7f6148003fd0 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f616140e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f615c06e960 con 0x7f615c1f8900 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f616140e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f615c06ec60 con 0x7f615c1f9250 2024-09-21T05:40:09.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f615c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f615c15e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53750/0 (socket says 172.21.15.22:53750) 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 -- 172.21.15.22:0/3847198561 learned_addr learned my addr 172.21.15.22:0/3847198561 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615b7fe640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f615c1f9250 0x7f615c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615a7fc640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f615c1f8900 0x7f615c1f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 -- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f615c1f9250 msgr2=0x7f615c15a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f615c1f9250 0x7f615c15a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 -- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f615c1f8900 msgr2=0x7f615c1f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f615c1f8900 0x7f615c1f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 -- 172.21.15.22:0/3847198561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615c1600a0 con 0x7f6148003fd0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615a7fc640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f615c1f8900 0x7f615c1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.547+0000 7f615affd640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f615c15e4b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f614c011f10 tx=0x7f614c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.548+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c01e070 con 0x7f6148003fd0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.548+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f614c00e5c0 con 0x7f6148003fd0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.548+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c00e9f0 con 0x7f6148003fd0 2024-09-21T05:40:09.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.548+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615c15b170 con 0x7f6148003fd0 2024-09-21T05:40:09.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.548+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f615c15c610 con 0x7f6148003fd0 2024-09-21T05:40:09.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.549+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f615c0b97e0 con 0x7f6148003fd0 2024-09-21T05:40:09.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.550+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f614c021070 con 0x7f6148003fd0 2024-09-21T05:40:09.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.550+0000 7f61537fe640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f613c078b90 0x7f613c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:09.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.550+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f614c09bc30 con 0x7f6148003fd0 2024-09-21T05:40:09.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.551+0000 7f615a7fc640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f613c078b90 0x7f613c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:09.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.551+0000 7f615a7fc640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f613c078b90 0x7f613c07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6144035000 tx=0x7f61440046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:09.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.555+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f614c063a20 con 0x7f6148003fd0 2024-09-21T05:40:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.782+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f615c069230 con 0x7f6148003fd0 2024-09-21T05:40:09.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.782+0000 7f61537fe640 1 -- 172.21.15.22:0/3847198561 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f614c068450 con 0x7f6148003fd0 2024-09-21T05:40:09.785 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:40:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f613c078b90 msgr2=0x7f613c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f613c078b90 0x7f613c07b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6144035000 tx=0x7f61440046c0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 msgr2=0x7f615c15e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f615c15e4b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f614c011f10 tx=0x7f614c00f790 comp rx=0 tx=0).stop 2024-09-21T05:40:09.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 shutdown_connections 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f613c078b90 0x7f613c07b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f615c1f9250 0x7f615c15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f615c1f8900 0x7f615c1f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 --2- 172.21.15.22:0/3847198561 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6148003fd0 0x7f615c15e4b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 >> 172.21.15.22:0/3847198561 conn(0x7f615c09d840 msgr2=0x7f615c0c7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 shutdown_connections 2024-09-21T05:40:09.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:09.785+0000 7f616140e640 1 -- 172.21.15.22:0/3847198561 wait complete. 2024-09-21T05:40:09.798 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:40:09.798 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:40:09.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:40:09.798 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:40:09.799 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:09.799 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:40:09.799 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:40:09.799 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:40:09.799 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:40:09.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:09.799 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:40:09.815 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:40:09.815 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:40:09.815 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:40:09.816 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:40:09.817 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.818 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:40:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:40:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:40:09.818 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:09.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:09.818 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:40:09.879 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:40:09.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:09.881 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:40:09.938 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:40:09.938 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:40:09.938 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:40:09.938 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:40:09.938 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:40:09.938 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:40:09.939 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:40:09.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:40:09.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:40:09.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:40:09.939 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:40:10.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:40:10.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:10.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.067 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:40:10.083 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:40:10.083 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:40:10.083 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:40:10.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.084 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:40:10.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:40:10.149 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:10.205 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:40:10.205 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:40:10.205 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:10.261 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:40:10.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:10.356 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:10.357 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:40:10.357 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:40:10.368 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:10.368 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:40:10.368 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:40:10.368 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:40:10.368 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:40:10.373 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:10.373 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:40:10.374 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:40:10.374 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:40:10.374 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:40:10.374 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:40:10.374 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:40:10.374 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:40:10.376 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:40:10.376 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:40:10.376 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:40:10.376 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:40:10.377 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:40:10.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:40:10.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.377 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:10.408 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:40:10.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:40:10.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:10.510 INFO:teuthology.orchestra.run.smithi022.stdout:client4754 2024-09-21T05:40:10.510 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:10.510 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:10.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.526 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-09-21T05:40:10.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.589 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-09-21T05:40:10.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.646 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-09-21T05:40:10.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.704 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-09-21T05:40:10.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.763 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-09-21T05:40:10.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:10.820 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-21T05:40:10.881 INFO:teuthology.orchestra.run.smithi022.stdout:./ 2024-09-21T05:40:10.882 INFO:teuthology.orchestra.run.smithi022.stdout:./rfile 2024-09-21T05:40:10.882 INFO:teuthology.orchestra.run.smithi022.stdout:./parentdir 2024-09-21T05:40:10.882 INFO:teuthology.orchestra.run.smithi022.stdout:./parentdir/pfile 2024-09-21T05:40:10.882 INFO:teuthology.orchestra.run.smithi022.stdout:./parentdir/childdir 2024-09-21T05:40:10.882 INFO:teuthology.orchestra.run.smithi022.stdout:./parentdir/childdir/cfile 2024-09-21T05:40:10.883 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:10.883 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:10.883 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:10.884 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:10.884 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-21T05:40:10.884 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:10.988 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:40:10.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:10.997 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:10.997 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:10.997 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:10.997 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:10.997 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-09-21T05:40:10.997 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:11.104 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627778 2024-09-21T05:40:11.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:11.114 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:11.114 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:11.114 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:11.114 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:11.114 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-09-21T05:40:11.114 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:11.221 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:40:11.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:11.230 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:11.230 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:11.230 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:11.230 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:11.230 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-09-21T05:40:11.231 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:11.337 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628279 2024-09-21T05:40:11.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:11.346 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:11.346 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:11.346 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:11.346 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:11.346 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-09-21T05:40:11.346 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:11.453 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:40:11.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:11.461 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:11.462 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:11.462 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:11.462 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:11.462 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-09-21T05:40:11.462 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:11.569 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628280 2024-09-21T05:40:11.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:11.578 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:11.578 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:11.634 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:11.635 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:11.691 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 -- 172.21.15.22:0/1543222289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0cf7a0 msgr2=0x7f3f2c1626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 --2- 172.21.15.22:0/1543222289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0cf7a0 0x7f3f2c1626e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c005fd0 tx=0x7f3f1c02aa90 comp rx=0 tx=0).stop 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 -- 172.21.15.22:0/1543222289 shutdown_connections 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 --2- 172.21.15.22:0/1543222289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0cf7a0 0x7f3f2c1626e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 --2- 172.21.15.22:0/1543222289 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f2c0d2ed0 0x7f3f2c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 -- 172.21.15.22:0/1543222289 >> 172.21.15.22:0/1543222289 conn(0x7f3f2c08bfc0 msgr2=0x7f3f2c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 -- 172.21.15.22:0/1543222289 shutdown_connections 2024-09-21T05:40:11.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.846+0000 7f3f34169640 1 -- 172.21.15.22:0/1543222289 wait complete. 2024-09-21T05:40:11.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.848+0000 7f3f34169640 1 Processor -- start 2024-09-21T05:40:11.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.848+0000 7f3f34169640 1 -- start start 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.849+0000 7f3f34169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 0x7f3f2c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.849+0000 7f3f34169640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f2c0cf7a0 0x7f3f2c0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.849+0000 7f3f34169640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f2c0d2ed0 0x7f3f2c0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.849+0000 7f3f34169640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c162da0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.849+0000 7f3f34169640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c162c20 con 0x7f3f2c0cf7a0 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.849+0000 7f3f34169640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2c162f20 con 0x7f3f2c0d2ed0 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 0x7f3f2c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 0x7f3f2c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53770/0 (socket says 172.21.15.22:53770) 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 -- 172.21.15.22:0/2834522017 learned_addr learned my addr 172.21.15.22:0/2834522017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:11.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f32706640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f2c0d2ed0 0x7f3f2c0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31704640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f2c0cf7a0 0x7f3f2c0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 -- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f2c0d2ed0 msgr2=0x7f3f2c0b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f2c0d2ed0 0x7f3f2c0b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 -- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f2c0cf7a0 msgr2=0x7f3f2c0cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f2c0cf7a0 0x7f3f2c0cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 -- 172.21.15.22:0/2834522017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c0cb9e0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f31f05640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 0x7f3f2c156010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400e7c0 tx=0x7f3f2400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400ccc0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f24010430 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c0cbcc0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24010840 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.850+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c0cc280 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.851+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c1634b0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.852+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f240109e0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.853+0000 7f3f2a7fc640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.853+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3f2409b7f0 con 0x7f3f2c0b9b70 2024-09-21T05:40:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.853+0000 7f3f31704640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:11.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.853+0000 7f3f31704640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f18078b90 0x7f3f1807b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c06fc40 tx=0x7f3f20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:11.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:11.857+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f24063c30 con 0x7f3f2c0b9b70 2024-09-21T05:40:12.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.055+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3f2c162990 con 0x7f3f2c0b9b70 2024-09-21T05:40:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.055+0000 7f3f2a7fc640 1 -- 172.21.15.22:0/2834522017 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f3f24063530 con 0x7f3f2c0b9b70 2024-09-21T05:40:12.058 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f18078b90 msgr2=0x7f3f1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f18078b90 0x7f3f1807b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c06fc40 tx=0x7f3f20009040 comp rx=0 tx=0).stop 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 msgr2=0x7f3f2c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 0x7f3f2c156010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400e7c0 tx=0x7f3f2400ec80 comp rx=0 tx=0).stop 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 shutdown_connections 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3f18078b90 0x7f3f1807b0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f2c0d2ed0 0x7f3f2c0b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f2c0cf7a0 0x7f3f2c0cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 --2- 172.21.15.22:0/2834522017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f2c0b9b70 0x7f3f2c156010 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.057+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 >> 172.21.15.22:0/2834522017 conn(0x7f3f2c08bfc0 msgr2=0x7f3f2c0d49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:12.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.058+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 shutdown_connections 2024-09-21T05:40:12.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.058+0000 7f3f34169640 1 -- 172.21.15.22:0/2834522017 wait complete. 2024-09-21T05:40:12.071 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:40:12.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:12.071 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:12.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:40:12.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:12.126 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:40:12.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 -- 172.21.15.22:0/3087225757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00b9970 msgr2=0x7fbae00b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 --2- 172.21.15.22:0/3087225757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00b9970 0x7fbae00b3710 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbad0005fd0 tx=0x7fbad002aa10 comp rx=0 tx=0).stop 2024-09-21T05:40:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 -- 172.21.15.22:0/3087225757 shutdown_connections 2024-09-21T05:40:12.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 --2- 172.21.15.22:0/3087225757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbae00b5fa0 0x7fbae00b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 --2- 172.21.15.22:0/3087225757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00b9970 0x7fbae00b3710 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 -- 172.21.15.22:0/3087225757 >> 172.21.15.22:0/3087225757 conn(0x7fbae009d7f0 msgr2=0x7fbae00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 -- 172.21.15.22:0/3087225757 shutdown_connections 2024-09-21T05:40:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.311+0000 7fbae5838640 1 -- 172.21.15.22:0/3087225757 wait complete. 2024-09-21T05:40:12.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.313+0000 7fbae5838640 1 Processor -- start 2024-09-21T05:40:12.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.313+0000 7fbae5838640 1 -- start start 2024-09-21T05:40:12.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbae5838640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbae00b5fa0 0x7fbae01592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbae5838640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae00b9970 0x7fbae00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbae5838640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 0x7fbae00b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbae5838640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbae006f270 con 0x7fbae00ced80 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbae5838640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbae006f0f0 con 0x7fbae00b9970 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbae5838640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbae006f3f0 con 0x7fbae00b5fa0 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 0x7fbae00b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 0x7fbae00b4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53786/0 (socket says 172.21.15.22:53786) 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 -- 172.21.15.22:0/1063287479 learned_addr learned my addr 172.21.15.22:0/1063287479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadeffd640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbae00b5fa0 0x7fbae01592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbade7fc640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae00b9970 0x7fbae00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 -- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbae00b5fa0 msgr2=0x7fbae01592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbae00b5fa0 0x7fbae01592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 -- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae00b9970 msgr2=0x7fbae00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae00b9970 0x7fbae00b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.314+0000 7fbadf7fe640 1 -- 172.21.15.22:0/1063287479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae00ce900 con 0x7fbae00ced80 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.315+0000 7fbadf7fe640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 0x7fbae00b4310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c3c0 tx=0x7fbad800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.315+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8017070 con 0x7fbae00ced80 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.315+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbad80035f0 con 0x7fbae00ced80 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.315+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae01eb780 con 0x7fbae00ced80 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.315+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8003a40 con 0x7fbae00ced80 2024-09-21T05:40:12.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.315+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae01eba20 con 0x7fbae00ced80 2024-09-21T05:40:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.316+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae006f970 con 0x7fbae00ced80 2024-09-21T05:40:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.317+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbad801a070 con 0x7fbae00ced80 2024-09-21T05:40:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.317+0000 7fbac7fff640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbacc078b90 0x7fbacc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.318+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbad8099f80 con 0x7fbae00ced80 2024-09-21T05:40:12.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.318+0000 7fbadeffd640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbacc078b90 0x7fbacc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.318+0000 7fbadeffd640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbacc078b90 0x7fbacc07b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbad0035000 tx=0x7fbad003b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:12.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.322+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbae00b68f0 con 0x7fbae00ced80 2024-09-21T05:40:12.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.548+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbae00b49a0 con 0x7fbae00ced80 2024-09-21T05:40:12.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.549+0000 7fbac7fff640 1 -- 172.21.15.22:0/1063287479 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7fbad8061d90 con 0x7fbae00ced80 2024-09-21T05:40:12.552 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:12.552 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:12.552 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.551+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbacc078b90 msgr2=0x7fbacc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.551+0000 7fbae5838640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbacc078b90 0x7fbacc07b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbad0035000 tx=0x7fbad003b040 comp rx=0 tx=0).stop 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 msgr2=0x7fbae00b4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 0x7fbae00b4310 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbad800c3c0 tx=0x7fbad800c880 comp rx=0 tx=0).stop 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 shutdown_connections 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbae00ced80 0x7fbae00b4310 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbae00b9970 0x7fbae00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbacc078b90 0x7fbacc07b0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 --2- 172.21.15.22:0/1063287479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbae00b5fa0 0x7fbae01592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 >> 172.21.15.22:0/1063287479 conn(0x7fbae009d7f0 msgr2=0x7fbae00d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:12.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 shutdown_connections 2024-09-21T05:40:12.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.552+0000 7fbae5838640 1 -- 172.21.15.22:0/1063287479 wait complete. 2024-09-21T05:40:12.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-21T05:40:12.705 INFO:teuthology.orchestra.run.smithi022.stdout:{"message":"","return_code":0} 2024-09-21T05:40:12.705 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:40:12.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3707300953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b9970 msgr2=0x7fb1a40b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3707300953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b9970 0x7fb1a40b3710 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40d3de0 tx=0x7fb19402a980 comp rx=0 tx=0).stop 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3707300953 shutdown_connections 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3707300953 >> v2:172.21.15.134:3300/0 conn(0x7fb1a40b5fa0 0x7fb1a40b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3707300953 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b9970 0x7fb1a40b3710 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3707300953 >> 172.21.15.22:0/3707300953 conn(0x7fb1a409d7f0 msgr2=0x7fb1a40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3707300953 shutdown_connections 2024-09-21T05:40:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.820+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3707300953 wait complete. 2024-09-21T05:40:12.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.822+0000 7fb1ac15e640 1 Processor -- start 2024-09-21T05:40:12.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.822+0000 7fb1ac15e640 1 -- start start 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1ac15e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 0x7fb1a415da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1ac15e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a40b9970 0x7fb1a415ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1ac15e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1a40ced80 0x7fb1a40cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1ac15e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a406f270 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1ac15e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a406f0f0 con 0x7fb1a40b9970 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1ac15e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a406f3f0 con 0x7fb1a40ced80 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1a9efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 0x7fb1a415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1a9efa640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 0x7fb1a415da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53802/0 (socket says 172.21.15.22:53802) 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.823+0000 7fb1a9efa640 1 -- 172.21.15.22:0/3289701862 learned_addr learned my addr 172.21.15.22:0/3289701862 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a96f9640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a40b9970 0x7fb1a415ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1aa6fb640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1a40ced80 0x7fb1a40cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a9efa640 1 -- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1a40ced80 msgr2=0x7fb1a40cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a9efa640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1a40ced80 0x7fb1a40cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a9efa640 1 -- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a40b9970 msgr2=0x7fb1a415ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a9efa640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a40b9970 0x7fb1a415ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a9efa640 1 -- 172.21.15.22:0/3289701862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a41f8d80 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1aa6fb640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1a40ced80 0x7fb1a40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a96f9640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a40b9970 0x7fb1a415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a9efa640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 0x7fb1a415da50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb194009be0 tx=0x7fb194003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb194034040 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb194004e30 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41f9060 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1940045b0 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.824+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a41f9620 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.825+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a40cfc60 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.826+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb194037020 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.827+0000 7fb1a27fc640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb190078bc0 0x7fb19007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.827+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb1940bc410 con 0x7fb1a40b5fa0 2024-09-21T05:40:12.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.827+0000 7fb1a96f9640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb190078bc0 0x7fb19007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:12.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.828+0000 7fb1a96f9640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb190078bc0 0x7fb19007b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1a41ec880 tx=0x7fb198005d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:12.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:12.831+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1a40b68f0 con 0x7fb1a40b5fa0 2024-09-21T05:40:13.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.058+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1a40b66b0 con 0x7fb1a40b5fa0 2024-09-21T05:40:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.059+0000 7fb1a27fc640 1 -- 172.21.15.22:0/3289701862 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7fb1940847c0 con 0x7fb1a40b5fa0 2024-09-21T05:40:13.061 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:13.061 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:13.061 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb190078bc0 msgr2=0x7fb19007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb190078bc0 0x7fb19007b0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb1a41ec880 tx=0x7fb198005d10 comp rx=0 tx=0).stop 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 msgr2=0x7fb1a415da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 0x7fb1a415da50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb194009be0 tx=0x7fb194003f90 comp rx=0 tx=0).stop 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 shutdown_connections 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb190078bc0 0x7fb19007b0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1a40ced80 0x7fb1a40cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1a40b9970 0x7fb1a415ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 --2- 172.21.15.22:0/3289701862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1a40b5fa0 0x7fb1a415da50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 >> 172.21.15.22:0/3289701862 conn(0x7fb1a409d7f0 msgr2=0x7fb1a40d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 shutdown_connections 2024-09-21T05:40:13.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:13.061+0000 7fb1ac15e640 1 -- 172.21.15.22:0/3289701862 wait complete. 2024-09-21T05:40:13.074 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-09-21T05:40:13.206 INFO:teuthology.orchestra.run.smithi022.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-09-21T05:40:13.206 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-09-21T05:40:21.056 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:40:21.059 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:40:23.208 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-09-21T05:40:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.274+0000 7f225d7b8f40 1 Processor -- start 2024-09-21T05:40:23.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.274+0000 7f225d7b8f40 1 -- start start 2024-09-21T05:40:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.275+0000 7f225d7b8f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e63883d0 0x5577e63b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.275+0000 7f225d7b8f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e64e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.275+0000 7f225d7b8f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e638be60 0x5577e64e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.275+0000 7f225d7b8f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577e62d9d60 con 0x5577e6389ea0 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.275+0000 7f225d7b8f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577e6338430 con 0x5577e63883d0 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.275+0000 7f225d7b8f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5577e638b3d0 con 0x5577e638be60 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e64e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e64e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59530/0 (socket says 172.21.15.22:59530) 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 -- 172.21.15.22:0/2967416371 learned_addr learned my addr 172.21.15.22:0/2967416371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2257272640 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e63883d0 0x5577e63b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2257a73640 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e638be60 0x5577e64e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 -- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e638be60 msgr2=0x5577e64e9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e638be60 0x5577e64e9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 -- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e63883d0 msgr2=0x5577e63b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e63883d0 0x5577e63b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 -- 172.21.15.22:0/2967416371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e645e990 con 0x5577e6389ea0 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2257272640 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e63883d0 0x5577e63b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:23.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2256a71640 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e64e9720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f224000b790 tx=0x7f224000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bcb08874e19e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2255a6f640 1 -- 172.21.15.22:0/2967416371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224000ab40 con 0x5577e6389ea0 2024-09-21T05:40:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2255a6f640 1 -- 172.21.15.22:0/2967416371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f224000ace0 con 0x5577e6389ea0 2024-09-21T05:40:23.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.276+0000 7f2255a6f640 1 -- 172.21.15.22:0/2967416371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240015680 con 0x5577e6389ea0 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2967416371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 msgr2=0x5577e64e9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e64e9720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f224000b790 tx=0x7f224000bc50 comp rx=0 tx=0).stop 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2967416371 shutdown_connections 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e638be60 0x5577e64e9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e64e9720 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2967416371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e63883d0 0x5577e63b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2967416371 >> 172.21.15.22:0/2967416371 conn(0x5577e62aaac0 msgr2=0x5577e645d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2967416371 shutdown_connections 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.277+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2967416371 wait complete. 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.278+0000 7f225d7b8f40 1 Processor -- start 2024-09-21T05:40:23.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.278+0000 7f225d7b8f40 1 -- start start 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f225d7b8f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e63883d0 0x5577e63b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f225d7b8f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e6452110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f225d7b8f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e638be60 0x5577e6453550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f225d7b8f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577e645ec70 con 0x5577e6389ea0 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f225d7b8f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577e645e1b0 con 0x5577e638be60 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f225d7b8f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5577e64ebe70 con 0x5577e63883d0 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e6452110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e6452110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59544/0 (socket says 172.21.15.22:59544) 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 -- 172.21.15.22:0/2217730411 learned_addr learned my addr 172.21.15.22:0/2217730411 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2257272640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e63883d0 0x5577e63b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2257a73640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e638be60 0x5577e6453550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 -- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e63883d0 msgr2=0x5577e63b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577e63883d0 0x5577e63b6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 -- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e638be60 msgr2=0x5577e6453550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577e638be60 0x5577e6453550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 -- 172.21.15.22:0/2217730411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e6453a90 con 0x5577e6389ea0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2256a71640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e6452110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f224000a7a0 tx=0x7f22400089c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.280+0000 7f2254a6d640 1 -- 172.21.15.22:0/2217730411 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240004580 con 0x5577e6389ea0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.281+0000 7f2254a6d640 1 -- 172.21.15.22:0/2217730411 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2240008df0 con 0x5577e6389ea0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.281+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577e64fc7e0 con 0x5577e6389ea0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.281+0000 7f2254a6d640 1 -- 172.21.15.22:0/2217730411 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224001e420 con 0x5577e6389ea0 2024-09-21T05:40:23.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.281+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e6452a70 con 0x5577e6389ea0 2024-09-21T05:40:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.282+0000 7f2254a6d640 1 -- 172.21.15.22:0/2217730411 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f224001e5c0 con 0x5577e6389ea0 2024-09-21T05:40:23.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.283+0000 7f2254a6d640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2238077570 0x7f2238079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.283+0000 7f2254a6d640 1 -- 172.21.15.22:0/2217730411 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f224009b290 con 0x5577e6389ea0 2024-09-21T05:40:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.283+0000 7f2257272640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2238077570 0x7f2238079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.283+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x5577e6453dd0 0x5577e63cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.283+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5577e63b7ed0 con 0x5577e6453dd0 2024-09-21T05:40:23.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.283+0000 7f2257a73640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x5577e6453dd0 0x5577e63cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.284+0000 7f2257272640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2238077570 0x7f2238079a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5577e645cd40 tx=0x7f2248008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.284+0000 7f2257a73640 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x5577e6453dd0 0x5577e63cfbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.284+0000 7f2257a73640 1 -- 172.21.15.22:0/2217730411 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f225004db50 con 0x5577e6453dd0 2024-09-21T05:40:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.284+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x5577e6453dd0 msgr2=0x5577e63cfbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.284+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x5577e6453dd0 0x5577e63cfbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2238077570 msgr2=0x7f2238079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2238077570 0x7f2238079a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5577e645cd40 tx=0x7f2248008040 comp rx=0 tx=0).stop 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 msgr2=0x5577e6452110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 --2- 172.21.15.22:0/2217730411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577e6389ea0 0x5577e6452110 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f224000a7a0 tx=0x7f22400089c0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f2257a73640 1 -- 172.21.15.22:0/2217730411 reap_dead start 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 shutdown_connections 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 >> 172.21.15.22:0/2217730411 conn(0x5577e62aaac0 msgr2=0x5577e63b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 shutdown_connections 2024-09-21T05:40:23.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.285+0000 7f225d7b8f40 1 -- 172.21.15.22:0/2217730411 wait complete. 2024-09-21T05:40:23.290 INFO:teuthology.orchestra.run.smithi022.stdout:mytag 2024-09-21T05:40:23.290 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-09-21T05:40:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.347+0000 7f4784523f40 1 Processor -- start 2024-09-21T05:40:23.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.347+0000 7f4784523f40 1 -- start start 2024-09-21T05:40:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.348+0000 7f4784523f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f719e2c0 0x55f3f7236740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.348+0000 7f4784523f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f7243300 0x55f3f7170de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.348+0000 7f4784523f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 0x55f3f7171320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.348+0000 7f4784523f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f3f70bed60 con 0x55f3f7243c50 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.348+0000 7f4784523f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3f711d430 con 0x55f3f719e2c0 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.348+0000 7f4784523f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55f3f71b7a50 con 0x55f3f7243300 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 0x55f3f7171320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 0x55f3f7171320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59552/0 (socket says 172.21.15.22:59552) 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 -- 172.21.15.22:0/3294618364 learned_addr learned my addr 172.21.15.22:0/3294618364 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477d50b640 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f719e2c0 0x55f3f7236740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477cd0a640 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f7243300 0x55f3f7170de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 -- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f7243300 msgr2=0x55f3f7170de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f7243300 0x55f3f7170de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 -- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f719e2c0 msgr2=0x55f3f7236740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f719e2c0 0x55f3f7236740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 -- 172.21.15.22:0/3294618364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f7236c80 con 0x55f3f7243c50 2024-09-21T05:40:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f477dd0c640 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 0x55f3f7171320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f47780015b0 tx=0x7f4778001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c088cd85a8cd4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f476f7fe640 1 -- 172.21.15.22:0/3294618364 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778017070 con 0x55f3f7243c50 2024-09-21T05:40:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f476f7fe640 1 -- 172.21.15.22:0/3294618364 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47780027a0 con 0x55f3f7243c50 2024-09-21T05:40:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.349+0000 7f476f7fe640 1 -- 172.21.15.22:0/3294618364 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778002c30 con 0x55f3f7243c50 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 -- 172.21.15.22:0/3294618364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 msgr2=0x55f3f7171320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 0x55f3f7171320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f47780015b0 tx=0x7f4778001a70 comp rx=0 tx=0).stop 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 -- 172.21.15.22:0/3294618364 shutdown_connections 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243c50 0x55f3f7171320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f7243300 0x55f3f7170de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 --2- 172.21.15.22:0/3294618364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f719e2c0 0x55f3f7236740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 -- 172.21.15.22:0/3294618364 >> 172.21.15.22:0/3294618364 conn(0x55f3f708fac0 msgr2=0x55f3f716e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 -- 172.21.15.22:0/3294618364 shutdown_connections 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.350+0000 7f4784523f40 1 -- 172.21.15.22:0/3294618364 wait complete. 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.351+0000 7f4784523f40 1 Processor -- start 2024-09-21T05:40:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.351+0000 7f4784523f40 1 -- start start 2024-09-21T05:40:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f4784523f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f719e2c0 0x55f3f7233400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f4784523f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243300 0x55f3f716fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f4784523f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f7243c50 0x55f3f7170160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f4784523f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f3f716f780 con 0x55f3f7243300 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f4784523f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f3f716f250 con 0x55f3f7243c50 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f4784523f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55f3f71b5bb0 con 0x55f3f719e2c0 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f477cd0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243300 0x55f3f716fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f477cd0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243300 0x55f3f716fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59560/0 (socket says 172.21.15.22:59560) 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f477cd0a640 1 -- 172.21.15.22:0/1849882692 learned_addr learned my addr 172.21.15.22:0/1849882692 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f477d50b640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f719e2c0 0x55f3f7233400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.352+0000 7f477dd0c640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f7243c50 0x55f3f7170160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f477cd0a640 1 -- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f719e2c0 msgr2=0x55f3f7233400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f477cd0a640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f3f719e2c0 0x55f3f7233400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f477cd0a640 1 -- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f7243c50 msgr2=0x55f3f7170160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f477cd0a640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f3f7243c50 0x55f3f7170160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f477cd0a640 1 -- 172.21.15.22:0/1849882692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3f719c170 con 0x55f3f7243300 2024-09-21T05:40:23.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f477cd0a640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243300 0x55f3f716fc20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f477400a720 tx=0x7f477400abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f476e7fc640 1 -- 172.21.15.22:0/1849882692 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477401b070 con 0x55f3f7243300 2024-09-21T05:40:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f476e7fc640 1 -- 172.21.15.22:0/1849882692 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4774009d50 con 0x55f3f7243300 2024-09-21T05:40:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f3f71b3850 con 0x55f3f7243300 2024-09-21T05:40:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f476e7fc640 1 -- 172.21.15.22:0/1849882692 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4774016680 con 0x55f3f7243300 2024-09-21T05:40:23.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.353+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3f71b50d0 con 0x55f3f7243300 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.355+0000 7f476e7fc640 1 -- 172.21.15.22:0/1849882692 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f477401e070 con 0x55f3f7243300 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.355+0000 7f476e7fc640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4760077570 0x7f4760079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.355+0000 7f476e7fc640 1 -- 172.21.15.22:0/1849882692 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f477409a370 con 0x55f3f7243300 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f4784523f40 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f3f71b4050 0x55f3f71b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55f3f71b49a0 con 0x55f3f71b4050 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f477d50b640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4760077570 0x7f4760079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f477dd0c640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f3f71b4050 0x55f3f71b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f477dd0c640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f3f71b4050 0x55f3f71b4460 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f477d50b640 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4760077570 0x7f4760079a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4770055000 tx=0x7f4770003020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f477dd0c640 1 -- 172.21.15.22:0/1849882692 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f477800f070 con 0x55f3f71b4050 2024-09-21T05:40:23.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f3f71b4050 msgr2=0x55f3f71b4460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.356+0000 7f4784523f40 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f3f71b4050 0x55f3f71b4460 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4760077570 msgr2=0x7f4760079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4760077570 0x7f4760079a80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4770055000 tx=0x7f4770003020 comp rx=0 tx=0).stop 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243300 msgr2=0x55f3f716fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 --2- 172.21.15.22:0/1849882692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f3f7243300 0x55f3f716fc20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f477400a720 tx=0x7f477400abe0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f477dd0c640 1 -- 172.21.15.22:0/1849882692 reap_dead start 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 shutdown_connections 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 >> 172.21.15.22:0/1849882692 conn(0x55f3f708fac0 msgr2=0x55f3f716e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 shutdown_connections 2024-09-21T05:40:23.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.357+0000 7f4784523f40 1 -- 172.21.15.22:0/1849882692 wait complete. 2024-09-21T05:40:23.362 INFO:teuthology.orchestra.run.smithi022.stdout:mytag 2024-09-21T05:40:23.362 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-09-21T05:40:23.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.417+0000 7fc19cd09f40 1 Processor -- start 2024-09-21T05:40:23.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.417+0000 7fc19cd09f40 1 -- start start 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc19cd09f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 0x564ac3299fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc19cd09f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac329d300 0x564ac328de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc19cd09f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac32a46b0 0x564ac31ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc19cd09f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ac3118d60 con 0x564ac31f82c0 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc19cd09f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564ac3177430 con 0x564ac32a46b0 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc19cd09f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564ac3211840 con 0x564ac329d300 2024-09-21T05:40:23.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc1967c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 0x564ac3299fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc1967c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 0x564ac3299fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59584/0 (socket says 172.21.15.22:59584) 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc1967c3640 1 -- 172.21.15.22:0/3306952392 learned_addr learned my addr 172.21.15.22:0/3306952392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc196fc4640 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac32a46b0 0x564ac31ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.418+0000 7fc195fc2640 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac329d300 0x564ac328de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc1967c3640 1 -- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac329d300 msgr2=0x564ac328de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc1967c3640 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac329d300 0x564ac328de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc1967c3640 1 -- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac32a46b0 msgr2=0x564ac31ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc1967c3640 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac32a46b0 0x564ac31ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc1967c3640 1 -- 172.21.15.22:0/3306952392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac320f9c0 con 0x564ac31f82c0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc1967c3640 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 0x564ac3299fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c5d0 tx=0x7fc18800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdc5936b75aba4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc194fc0640 1 -- 172.21.15.22:0/3306952392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18800aae0 con 0x564ac31f82c0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc194fc0640 1 -- 172.21.15.22:0/3306952392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc18800ac80 con 0x564ac31f82c0 2024-09-21T05:40:23.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.419+0000 7fc194fc0640 1 -- 172.21.15.22:0/3306952392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1880156a0 con 0x564ac31f82c0 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 -- 172.21.15.22:0/3306952392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 msgr2=0x564ac3299fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 0x564ac3299fd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc18800c5d0 tx=0x7fc18800ca90 comp rx=0 tx=0).stop 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 -- 172.21.15.22:0/3306952392 shutdown_connections 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac32a46b0 0x564ac31ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac329d300 0x564ac328de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/3306952392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac31f82c0 0x564ac3299fd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 -- 172.21.15.22:0/3306952392 >> 172.21.15.22:0/3306952392 conn(0x564ac30e9ac0 msgr2=0x564ac3296fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 -- 172.21.15.22:0/3306952392 shutdown_connections 2024-09-21T05:40:23.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.420+0000 7fc19cd09f40 1 -- 172.21.15.22:0/3306952392 wait complete. 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.421+0000 7fc19cd09f40 1 Processor -- start 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.421+0000 7fc19cd09f40 1 -- start start 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc19cd09f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac31f82c0 0x564ac329f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc19cd09f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac329d300 0x564ac329bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc19cd09f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac32a46b0 0x564ac328de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc19cd09f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564ac31c9fb0 con 0x564ac329d300 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc19cd09f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564ac320fc60 con 0x564ac31f82c0 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc19cd09f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564ac320ff40 con 0x564ac32a46b0 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac329d300 0x564ac329bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac329d300 0x564ac329bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59596/0 (socket says 172.21.15.22:59596) 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 -- 172.21.15.22:0/2806151231 learned_addr learned my addr 172.21.15.22:0/2806151231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc1967c3640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac31f82c0 0x564ac329f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc196fc4640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac32a46b0 0x564ac328de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 -- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac32a46b0 msgr2=0x564ac328de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564ac32a46b0 0x564ac328de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 -- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac31f82c0 msgr2=0x564ac329f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564ac31f82c0 0x564ac329f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.422+0000 7fc195fc2640 1 -- 172.21.15.22:0/2806151231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ac3335f00 con 0x564ac329d300 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.423+0000 7fc195fc2640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac329d300 0x564ac329bf80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00de60 tx=0x7fc18c00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.423+0000 7fc1877fe640 1 -- 172.21.15.22:0/2806151231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c013070 con 0x564ac329d300 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.423+0000 7fc1877fe640 1 -- 172.21.15.22:0/2806151231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc18c006070 con 0x564ac329d300 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.423+0000 7fc1877fe640 1 -- 172.21.15.22:0/2806151231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00fb80 con 0x564ac329d300 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.423+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ac33361e0 con 0x564ac329d300 2024-09-21T05:40:23.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.423+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ac33367a0 con 0x564ac329d300 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.424+0000 7fc1877fe640 1 -- 172.21.15.22:0/2806151231 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc18c017410 con 0x564ac329d300 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.425+0000 7fc1877fe640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc178077570 0x7fc178079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.425+0000 7fc1877fe640 1 -- 172.21.15.22:0/2806151231 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc18c099e80 con 0x564ac329d300 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.425+0000 7fc1967c3640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc178077570 0x7fc178079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.425+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x564ac3294ed0 0x564ac32952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.425+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x564ac3342170 con 0x564ac3294ed0 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.425+0000 7fc196fc4640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x564ac3294ed0 0x564ac32952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc1967c3640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc178077570 0x7fc178079a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc18800dfd0 tx=0x7fc188017040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc196fc4640 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x564ac3294ed0 0x564ac32952e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc196fc4640 1 -- 172.21.15.22:0/2806151231 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fc19000b040 con 0x564ac3294ed0 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x564ac3294ed0 msgr2=0x564ac32952e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x564ac3294ed0 0x564ac32952e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc178077570 msgr2=0x7fc178079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc178077570 0x7fc178079a80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc18800dfd0 tx=0x7fc188017040 comp rx=0 tx=0).stop 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac329d300 msgr2=0x564ac329bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 --2- 172.21.15.22:0/2806151231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564ac329d300 0x564ac329bf80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00de60 tx=0x7fc18c00b670 comp rx=0 tx=0).stop 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc196fc4640 1 -- 172.21.15.22:0/2806151231 reap_dead start 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 shutdown_connections 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 >> 172.21.15.22:0/2806151231 conn(0x564ac30e9ac0 msgr2=0x564ac3296f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 shutdown_connections 2024-09-21T05:40:23.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.426+0000 7fc19cd09f40 1 -- 172.21.15.22:0/2806151231 wait complete. 2024-09-21T05:40:23.431 INFO:teuthology.orchestra.run.smithi022.stdout:mytag 2024-09-21T05:40:23.431 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-09-21T05:40:23.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.485+0000 7fa703ef6f40 1 Processor -- start 2024-09-21T05:40:23.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.485+0000 7fa703ef6f40 1 -- start start 2024-09-21T05:40:23.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa703ef6f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa703ef6f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 0x55c3e7cfd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa703ef6f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c2ae60 0x55c3e7cfbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa703ef6f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c3e7b78d60 con 0x55c3e7c28ea0 2024-09-21T05:40:23.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa703ef6f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3e7bd7430 con 0x55c3e7c273d0 2024-09-21T05:40:23.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa703ef6f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55c3e7c2a660 con 0x55c3e7c2ae60 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa6fd1af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 0x55c3e7cfd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa6fd1af640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 0x55c3e7cfd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59610/0 (socket says 172.21.15.22:59610) 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.486+0000 7fa6fd1af640 1 -- 172.21.15.22:0/3848635673 learned_addr learned my addr 172.21.15.22:0/3848635673 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd9b0640 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fe1b1640 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c2ae60 0x55c3e7cfbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd1af640 1 -- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c2ae60 msgr2=0x55c3e7cfbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd1af640 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c2ae60 0x55c3e7cfbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd1af640 1 -- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 msgr2=0x55c3e7cf96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd1af640 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd1af640 1 -- 172.21.15.22:0/3848635673 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e7c55840 con 0x55c3e7c28ea0 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6fd1af640 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 0x55c3e7cfd500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c5d0 tx=0x7fa6f400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcf8823c4aa2577 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6effff640 1 -- 172.21.15.22:0/3848635673 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400aae0 con 0x55c3e7c28ea0 2024-09-21T05:40:23.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6effff640 1 -- 172.21.15.22:0/3848635673 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f400ac80 con 0x55c3e7c28ea0 2024-09-21T05:40:23.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.487+0000 7fa6effff640 1 -- 172.21.15.22:0/3848635673 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40156a0 con 0x55c3e7c28ea0 2024-09-21T05:40:23.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 -- 172.21.15.22:0/3848635673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 msgr2=0x55c3e7cfd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 0x55c3e7cfd500 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c5d0 tx=0x7fa6f400ca90 comp rx=0 tx=0).stop 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 -- 172.21.15.22:0/3848635673 shutdown_connections 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c2ae60 0x55c3e7cfbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c28ea0 0x55c3e7cfd500 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/3848635673 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 -- 172.21.15.22:0/3848635673 >> 172.21.15.22:0/3848635673 conn(0x55c3e7b49ac0 msgr2=0x55c3e7c29430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 -- 172.21.15.22:0/3848635673 shutdown_connections 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.488+0000 7fa703ef6f40 1 -- 172.21.15.22:0/3848635673 wait complete. 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.489+0000 7fa703ef6f40 1 Processor -- start 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.489+0000 7fa703ef6f40 1 -- start start 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa703ef6f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa703ef6f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c28ea0 0x55c3e7c4ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa703ef6f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c2ae60 0x55c3e7c4b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa703ef6f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55c3e7c555a0 con 0x55c3e7c2ae60 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa703ef6f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55c3e7c56170 con 0x55c3e7c273d0 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa703ef6f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55c3e7cfb7d0 con 0x55c3e7c28ea0 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c2ae60 0x55c3e7c4b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c2ae60 0x55c3e7c4b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59618/0 (socket says 172.21.15.22:59618) 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 -- 172.21.15.22:0/900281595 learned_addr learned my addr 172.21.15.22:0/900281595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fd1af640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c28ea0 0x55c3e7c4ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fd9b0640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 -- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c28ea0 msgr2=0x55c3e7c4ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55c3e7c28ea0 0x55c3e7c4ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 -- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 msgr2=0x55c3e7cf6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55c3e7c273d0 0x55c3e7cf6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 -- 172.21.15.22:0/900281595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3e7cf7350 con 0x55c3e7c2ae60 2024-09-21T05:40:23.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.490+0000 7fa6fe1b1640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c2ae60 0x55c3e7c4b3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800dbd0 tx=0x7fa6f800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.491+0000 7fa6eeffd640 1 -- 172.21.15.22:0/900281595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80030d0 con 0x55c3e7c2ae60 2024-09-21T05:40:23.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.491+0000 7fa6eeffd640 1 -- 172.21.15.22:0/900281595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f8016070 con 0x55c3e7c2ae60 2024-09-21T05:40:23.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.491+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c3e7cf7920 con 0x55c3e7c2ae60 2024-09-21T05:40:23.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.491+0000 7fa6eeffd640 1 -- 172.21.15.22:0/900281595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f80116e0 con 0x55c3e7c2ae60 2024-09-21T05:40:23.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.491+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3e7c6cdd0 con 0x55c3e7c2ae60 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.492+0000 7fa6eeffd640 1 -- 172.21.15.22:0/900281595 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6f8011880 con 0x55c3e7c2ae60 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.493+0000 7fa6eeffd640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6e00775a0 0x7fa6e0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.493+0000 7fa6eeffd640 1 -- 172.21.15.22:0/900281595 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fa6f8099840 con 0x55c3e7c2ae60 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.493+0000 7fa6fd9b0640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6e00775a0 0x7fa6e0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.493+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55c3e7c6d800 0x55c3e7c6dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.493+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55c3e7c6e150 con 0x55c3e7c6d800 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.493+0000 7fa6fd1af640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55c3e7c6d800 0x55c3e7c6dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa6fd9b0640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6e00775a0 0x7fa6e0079ab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0056000 tx=0x7fa6f00033a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa6fd1af640 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55c3e7c6d800 0x55c3e7c6dc10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa6fd1af640 1 -- 172.21.15.22:0/900281595 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fa6f400aa60 con 0x55c3e7c6d800 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55c3e7c6d800 msgr2=0x55c3e7c6dc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55c3e7c6d800 0x55c3e7c6dc10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6e00775a0 msgr2=0x7fa6e0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6e00775a0 0x7fa6e0079ab0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0056000 tx=0x7fa6f00033a0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c2ae60 msgr2=0x55c3e7c4b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 --2- 172.21.15.22:0/900281595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55c3e7c2ae60 0x55c3e7c4b3b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800dbd0 tx=0x7fa6f800b5b0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa6fe1b1640 1 -- 172.21.15.22:0/900281595 reap_dead start 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 shutdown_connections 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 >> 172.21.15.22:0/900281595 conn(0x55c3e7b49ac0 msgr2=0x55c3e7cfc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 shutdown_connections 2024-09-21T05:40:23.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.494+0000 7fa703ef6f40 1 -- 172.21.15.22:0/900281595 wait complete. 2024-09-21T05:40:23.500 INFO:teuthology.orchestra.run.smithi022.stdout:mytag 2024-09-21T05:40:23.500 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-09-21T05:40:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.555+0000 7fdce3520f40 1 Processor -- start 2024-09-21T05:40:23.557 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.555+0000 7fdce3520f40 1 -- start start 2024-09-21T05:40:23.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdce3520f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 0x55d66ff75770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdce3520f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d66fee2af0 0x55d66feeae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdce3520f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2f00 0x55d66feeb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdce3520f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d66fdf0d60 con 0x7fdcc80028c0 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdce3520f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d66fe4f430 con 0x55d66fee2f00 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdce3520f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d66feebb80 con 0x55d66fee2af0 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 0x55d66ff75770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 0x55d66ff75770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59644/0 (socket says 172.21.15.22:59644) 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 -- 172.21.15.22:0/410037541 learned_addr learned my addr 172.21.15.22:0/410037541 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 -- 172.21.15.22:0/410037541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d66fee2af0 msgr2=0x55d66feeae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d66fee2af0 0x55d66feeae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 -- 172.21.15.22:0/410037541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2f00 msgr2=0x55d66feeb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2f00 0x55d66feeb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.556+0000 7fdcdc573640 1 -- 172.21.15.22:0/410037541 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66fee7660 con 0x7fdcc80028c0 2024-09-21T05:40:23.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdcdc573640 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 0x55d66ff75770 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400b810 tx=0x7fdcd400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db3d0d9d9daaf9b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdcdad70640 1 -- 172.21.15.22:0/410037541 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400be60 con 0x7fdcc80028c0 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdcdad70640 1 -- 172.21.15.22:0/410037541 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd400ad40 con 0x7fdcc80028c0 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdcdad70640 1 -- 172.21.15.22:0/410037541 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4016680 con 0x7fdcc80028c0 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 -- 172.21.15.22:0/410037541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 msgr2=0x55d66ff75770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 0x55d66ff75770 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400b810 tx=0x7fdcd400bc10 comp rx=0 tx=0).stop 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 -- 172.21.15.22:0/410037541 shutdown_connections 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2f00 0x55d66feeb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d66fee2af0 0x55d66feeae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 --2- 172.21.15.22:0/410037541 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdcc80028c0 0x55d66ff75770 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 -- 172.21.15.22:0/410037541 >> 172.21.15.22:0/410037541 conn(0x55d66fdc1ac0 msgr2=0x55d66fee8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 -- 172.21.15.22:0/410037541 shutdown_connections 2024-09-21T05:40:23.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.557+0000 7fdce3520f40 1 -- 172.21.15.22:0/410037541 wait complete. 2024-09-21T05:40:23.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.558+0000 7fdce3520f40 1 Processor -- start 2024-09-21T05:40:23.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.558+0000 7fdce3520f40 1 -- start start 2024-09-21T05:40:23.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2af0 0x55d66ff5b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d66fee2f00 0x55d6700100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc80028c0 0x55d670002850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d66fee7940 con 0x55d66fee2f00 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d66ff752e0 con 0x55d66fee2af0 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d66fee7c20 con 0x7fdcc80028c0 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d66fee2f00 0x55d6700100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d66fee2f00 0x55d6700100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59654/0 (socket says 172.21.15.22:59654) 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 -- 172.21.15.22:0/59861079 learned_addr learned my addr 172.21.15.22:0/59861079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdc573640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2af0 0x55d66ff5b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdcd74640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc80028c0 0x55d670002850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 -- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc80028c0 msgr2=0x55d670002850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdcc80028c0 0x55d670002850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 -- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2af0 msgr2=0x55d66ff5b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d66fee2af0 0x55d66ff5b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 -- 172.21.15.22:0/59861079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d66ff49630 con 0x55d66fee2f00 2024-09-21T05:40:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcdbd72640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d66fee2f00 0x55d6700100d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4026270 tx=0x7fdcc40569e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcd9d6e640 1 -- 172.21.15.22:0/59861079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4063030 con 0x55d66fee2f00 2024-09-21T05:40:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcd9d6e640 1 -- 172.21.15.22:0/59861079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc4004e30 con 0x55d66fee2f00 2024-09-21T05:40:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d66ff5b570 con 0x55d66fee2f00 2024-09-21T05:40:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.560+0000 7fdcd9d6e640 1 -- 172.21.15.22:0/59861079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc404dad0 con 0x55d66fee2f00 2024-09-21T05:40:23.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.561+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d66ff48b50 con 0x55d66fee2f00 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.562+0000 7fdcd9d6e640 1 -- 172.21.15.22:0/59861079 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdcc4066070 con 0x55d66fee2f00 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdcd9d6e640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdcbc077570 0x7fdcbc079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdcd9d6e640 1 -- 172.21.15.22:0/59861079 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fdcc40dd3e0 con 0x55d66fee2f00 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdcdc573640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdcbc077570 0x7fdcbc079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdce3520f40 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55d66ff73e90 0x55d66ff742a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55d66ff747e0 con 0x55d66ff73e90 2024-09-21T05:40:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdcdcd74640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55d66ff73e90 0x55d66ff742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdcdc573640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdcbc077570 0x7fdcbc079a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4004aa0 tx=0x7fdcd400e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.563+0000 7fdcdcd74640 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55d66ff73e90 0x55d66ff742a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdcdcd74640 1 -- 172.21.15.22:0/59861079 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fdccc003040 con 0x55d66ff73e90 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55d66ff73e90 msgr2=0x55d66ff742a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55d66ff73e90 0x55d66ff742a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdcbc077570 msgr2=0x7fdcbc079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdcbc077570 0x7fdcbc079a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4004aa0 tx=0x7fdcd400e040 comp rx=0 tx=0).stop 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d66fee2f00 msgr2=0x55d6700100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 --2- 172.21.15.22:0/59861079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d66fee2f00 0x55d6700100d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4026270 tx=0x7fdcc40569e0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdcdcd74640 1 -- 172.21.15.22:0/59861079 reap_dead start 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 shutdown_connections 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 >> 172.21.15.22:0/59861079 conn(0x55d66fdc1ac0 msgr2=0x55d66fee7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 shutdown_connections 2024-09-21T05:40:23.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.564+0000 7fdce3520f40 1 -- 172.21.15.22:0/59861079 wait complete. 2024-09-21T05:40:23.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:40:23.569 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:40:23.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:40:23.603 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:40:23.642 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:40:23.642 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:40:23.642 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:23.642 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:23.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:40:23.660 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:40:23.660 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:40:23.660 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:23.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:40:23.701 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:40:23.701 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:23.701 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:23.717 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:40:23.717 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:40:23.717 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:40:23.717 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:23.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:40:23.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60eb27e640 1 -- 172.21.15.22:0/1364125743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 msgr2=0x7f60cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60eb27e640 1 --2- 172.21.15.22:0/1364125743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60cc003e10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f60dc001740 tx=0x7f60cc005830 comp rx=0 tx=0).stop 2024-09-21T05:40:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60e981b640 1 -- 172.21.15.22:0/1364125743 reap_dead start 2024-09-21T05:40:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60eb27e640 1 -- 172.21.15.22:0/1364125743 shutdown_connections 2024-09-21T05:40:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60eb27e640 1 -- 172.21.15.22:0/1364125743 >> 172.21.15.22:0/1364125743 conn(0x7f60e4072ae0 msgr2=0x7f60e4072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60eb27e640 1 -- 172.21.15.22:0/1364125743 shutdown_connections 2024-09-21T05:40:23.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.874+0000 7f60eb27e640 1 -- 172.21.15.22:0/1364125743 wait complete. 2024-09-21T05:40:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.876+0000 7f60eb27e640 1 Processor -- start 2024-09-21T05:40:23.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.876+0000 7f60eb27e640 1 -- start start 2024-09-21T05:40:23.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60eb27e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60e41f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60eb27e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40b0fb0 0x7f60e41f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60eb27e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e41f2000 0x7f60e41f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60eb27e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40d1a60 con 0x7f60cc0039e0 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60eb27e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40d18e0 con 0x7f60e40b0fb0 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60eb27e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f60e40cf300 con 0x7f60e41f2000 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60e901a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60e41f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60e901a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60e41f63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59676/0 (socket says 172.21.15.22:59676) 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.877+0000 7f60e901a640 1 -- 172.21.15.22:0/616839182 learned_addr learned my addr 172.21.15.22:0/616839182 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e8819640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40b0fb0 0x7f60e41f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e981b640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e41f2000 0x7f60e41f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e901a640 1 -- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e41f2000 msgr2=0x7f60e41f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e901a640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e41f2000 0x7f60e41f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e901a640 1 -- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40b0fb0 msgr2=0x7f60e41f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:23.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e901a640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40b0fb0 0x7f60e41f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e901a640 1 -- 172.21.15.22:0/616839182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e40ab150 con 0x7f60cc0039e0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e901a640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60e41f63b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f60dc02a9f0 tx=0x7f60dc006220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc042070 con 0x7f60cc0039e0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60dc0037f0 con 0x7f60cc0039e0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e40ab430 con 0x7f60cc0039e0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60dc037040 con 0x7f60cc0039e0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.878+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e40ab9f0 con 0x7f60cc0039e0 2024-09-21T05:40:23.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.879+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e40cf890 con 0x7f60cc0039e0 2024-09-21T05:40:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.880+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f60dc045070 con 0x7f60cc0039e0 2024-09-21T05:40:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.881+0000 7f60e1ffb640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60d0078b90 0x7f60d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.881+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f60dc0ba8c0 con 0x7f60cc0039e0 2024-09-21T05:40:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.881+0000 7f60e8819640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60d0078b90 0x7f60d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:23.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.882+0000 7f60e8819640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60d0078b90 0x7f60d007b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60e40b14f0 tx=0x7f60d8005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:23.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:23.885+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60dc082820 con 0x7f60cc0039e0 2024-09-21T05:40:24.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.111+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60e40b7b10 con 0x7f60cc0039e0 2024-09-21T05:40:24.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.112+0000 7f60e1ffb640 1 -- 172.21.15.22:0/616839182 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f60dc087250 con 0x7f60cc0039e0 2024-09-21T05:40:24.114 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:24.114 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:24.115 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:24.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60d0078b90 msgr2=0x7f60d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60d0078b90 0x7f60d007b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60e40b14f0 tx=0x7f60d8005f90 comp rx=0 tx=0).stop 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 msgr2=0x7f60e41f63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60e41f63b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f60dc02a9f0 tx=0x7f60dc006220 comp rx=0 tx=0).stop 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 shutdown_connections 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60d0078b90 0x7f60d007b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e41f2000 0x7f60e41f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40b0fb0 0x7f60e41f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 --2- 172.21.15.22:0/616839182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60cc0039e0 0x7f60e41f63b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 >> 172.21.15.22:0/616839182 conn(0x7f60e4072ae0 msgr2=0x7f60e415b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 shutdown_connections 2024-09-21T05:40:24.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.114+0000 7f60eb27e640 1 -- 172.21.15.22:0/616839182 wait complete. 2024-09-21T05:40:24.128 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:24.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 -- 172.21.15.22:0/319368735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c01600b0 msgr2=0x7fe2c00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 --2- 172.21.15.22:0/319368735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c01600b0 0x7fe2c00ccbc0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00b7c50 tx=0x7fe2bc02a560 comp rx=0 tx=0).stop 2024-09-21T05:40:24.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 -- 172.21.15.22:0/319368735 shutdown_connections 2024-09-21T05:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 --2- 172.21.15.22:0/319368735 >> v2:172.21.15.134:3300/0 conn(0x7fe2c00cf320 0x7fe2c00d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 --2- 172.21.15.22:0/319368735 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c01600b0 0x7fe2c00ccbc0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 -- 172.21.15.22:0/319368735 >> 172.21.15.22:0/319368735 conn(0x7fe2c006d4f0 msgr2=0x7fe2c0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 -- 172.21.15.22:0/319368735 shutdown_connections 2024-09-21T05:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.243+0000 7fe2c8539640 1 -- 172.21.15.22:0/319368735 wait complete. 2024-09-21T05:40:24.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.245+0000 7fe2c8539640 1 Processor -- start 2024-09-21T05:40:24.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.245+0000 7fe2c8539640 1 -- start start 2024-09-21T05:40:24.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c8539640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c00b7cc0 0x7fe2c01f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c8539640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 0x7fe2c00aafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c8539640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c01600b0 0x7fe2c01503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c8539640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c00d13c0 con 0x7fe2c00cf320 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c8539640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c00d1240 con 0x7fe2c00b7cc0 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c8539640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c00d1540 con 0x7fe2c01600b0 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 0x7fe2c00aafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 0x7fe2c00aafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52248/0 (socket says 172.21.15.22:52248) 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 -- 172.21.15.22:0/184309931 learned_addr learned my addr 172.21.15.22:0/184309931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c62d5640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c00b7cc0 0x7fe2c01f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c6ad6640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c01600b0 0x7fe2c01503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 -- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c01600b0 msgr2=0x7fe2c01503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c01600b0 0x7fe2c01503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 -- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c00b7cc0 msgr2=0x7fe2c01f44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c00b7cc0 0x7fe2c01f44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.246+0000 7fe2c5ad4640 1 -- 172.21.15.22:0/184309931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c01f49f0 con 0x7fe2c00cf320 2024-09-21T05:40:24.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.247+0000 7fe2c5ad4640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 0x7fe2c00aafd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000b420 tx=0x7fe2b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.247+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0011830 con 0x7fe2c00cf320 2024-09-21T05:40:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.247+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b0016070 con 0x7fe2c00cf320 2024-09-21T05:40:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.247+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c00a9540 con 0x7fe2c00cf320 2024-09-21T05:40:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.247+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0011d60 con 0x7fe2c00cf320 2024-09-21T05:40:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.247+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c01f7930 con 0x7fe2c00cf320 2024-09-21T05:40:24.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.248+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c00d1a60 con 0x7fe2c00cf320 2024-09-21T05:40:24.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.249+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe2b001b460 con 0x7fe2c00cf320 2024-09-21T05:40:24.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.249+0000 7fe2b6ffd640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe2a4078b90 0x7fe2a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.249+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe2b009a740 con 0x7fe2c00cf320 2024-09-21T05:40:24.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.250+0000 7fe2c62d5640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe2a4078b90 0x7fe2a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.250+0000 7fe2c62d5640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe2a4078b90 0x7fe2a407b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc033000 tx=0x7fe2bc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:24.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.254+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2b0020020 con 0x7fe2c00cf320 2024-09-21T05:40:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.477+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2c00b7630 con 0x7fe2c00cf320 2024-09-21T05:40:24.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.477+0000 7fe2b6ffd640 1 -- 172.21.15.22:0/184309931 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7fe2c01f49f0 con 0x7fe2c00cf320 2024-09-21T05:40:24.480 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:24.480 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:24.481 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe2a4078b90 msgr2=0x7fe2a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe2a4078b90 0x7fe2a407b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc033000 tx=0x7fe2bc004730 comp rx=0 tx=0).stop 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 msgr2=0x7fe2c00aafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 0x7fe2c00aafd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000b420 tx=0x7fe2b000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 shutdown_connections 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe2a4078b90 0x7fe2a407b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2c01600b0 0x7fe2c01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2c00cf320 0x7fe2c00aafd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 --2- 172.21.15.22:0/184309931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2c00b7cc0 0x7fe2c01f44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 >> 172.21.15.22:0/184309931 conn(0x7fe2c006d4f0 msgr2=0x7fe2c015f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 shutdown_connections 2024-09-21T05:40:24.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.479+0000 7fe2c8539640 1 -- 172.21.15.22:0/184309931 wait complete. 2024-09-21T05:40:24.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 -- 172.21.15.22:0/1714804504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 msgr2=0x7f8e040d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 --2- 172.21.15.22:0/1714804504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040d4510 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8e0408e380 tx=0x7f8df802a530 comp rx=0 tx=0).stop 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 -- 172.21.15.22:0/1714804504 shutdown_connections 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 --2- 172.21.15.22:0/1714804504 >> v2:172.21.15.134:3300/0 conn(0x7f8e040d4a50 0x7f8e040b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 --2- 172.21.15.22:0/1714804504 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040d4510 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 -- 172.21.15.22:0/1714804504 >> 172.21.15.22:0/1714804504 conn(0x7f8e0408bfc0 msgr2=0x7f8e0409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 -- 172.21.15.22:0/1714804504 shutdown_connections 2024-09-21T05:40:24.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.608+0000 7f8e0a913640 1 -- 172.21.15.22:0/1714804504 wait complete. 2024-09-21T05:40:24.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.610+0000 7f8e0a913640 1 Processor -- start 2024-09-21T05:40:24.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.610+0000 7f8e0a913640 1 -- start start 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e0a913640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e0a913640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e040d0ce0 0x7f8e0415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e0a913640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e040d4a50 0x7f8e040cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e0a913640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e040b4a00 con 0x7f8e040b9be0 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e0a913640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e040b4880 con 0x7f8e040d0ce0 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e0a913640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8e040b4b80 con 0x7f8e040d4a50 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52260/0 (socket says 172.21.15.22:52260) 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e03fff640 1 -- 172.21.15.22:0/981779903 learned_addr learned my addr 172.21.15.22:0/981779903 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e08eb0640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e040d4a50 0x7f8e040cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e037fe640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e040d0ce0 0x7f8e0415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.611+0000 7f8e03fff640 1 -- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e040d4a50 msgr2=0x7f8e040cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e03fff640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e040d4a50 0x7f8e040cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e03fff640 1 -- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e040d0ce0 msgr2=0x7f8e0415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e03fff640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e040d0ce0 0x7f8e0415ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e03fff640 1 -- 172.21.15.22:0/981779903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0415f4c0 con 0x7f8e040b9be0 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e037fe640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e040d0ce0 0x7f8e0415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e03fff640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040cdd10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8df400c3f0 tx=0x7f8df400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4017070 con 0x7f8e040b9be0 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8df40035f0 con 0x7f8e040b9be0 2024-09-21T05:40:24.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e040b25d0 con 0x7f8e040b9be0 2024-09-21T05:40:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4003a40 con 0x7f8e040b9be0 2024-09-21T05:40:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.612+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e040b2b50 con 0x7f8e040b9be0 2024-09-21T05:40:24.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.613+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e040b50a0 con 0x7f8e040b9be0 2024-09-21T05:40:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.614+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8df401a070 con 0x7f8e040b9be0 2024-09-21T05:40:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.615+0000 7f8e00ff9640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8df0078bc0 0x7f8df007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.615+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8df4099fa0 con 0x7f8e040b9be0 2024-09-21T05:40:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.615+0000 7f8e037fe640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8df0078bc0 0x7f8df007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.615+0000 7f8e037fe640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8df0078bc0 0x7f8df007b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8df8034000 tx=0x7f8df8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:24.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.619+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e040d4e60 con 0x7f8e040b9be0 2024-09-21T05:40:24.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.806+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e041e7640 con 0x7f8e040b9be0 2024-09-21T05:40:24.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.807+0000 7f8e00ff9640 1 -- 172.21.15.22:0/981779903 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16717 (secure 0 0 0) 0x7f8df4062500 con 0x7f8e040b9be0 2024-09-21T05:40:24.810 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:24.810 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":32,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:04.576394+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:01.033773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:01.908775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:24.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8df0078bc0 msgr2=0x7f8df007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8df0078bc0 0x7f8df007b0d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8df8034000 tx=0x7f8df8004730 comp rx=0 tx=0).stop 2024-09-21T05:40:24.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 msgr2=0x7f8e040cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040cdd10 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8df400c3f0 tx=0x7f8df400c8b0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 shutdown_connections 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8df0078bc0 0x7f8df007b0d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e040d4a50 0x7f8e040cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e040d0ce0 0x7f8e0415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 --2- 172.21.15.22:0/981779903 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e040b9be0 0x7f8e040cdd10 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 >> 172.21.15.22:0/981779903 conn(0x7f8e0408bfc0 msgr2=0x7f8e04161d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 shutdown_connections 2024-09-21T05:40:24.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.809+0000 7f8e0a913640 1 -- 172.21.15.22:0/981779903 wait complete. 2024-09-21T05:40:24.824 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:40:24.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:24.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 -- 172.21.15.22:0/1057875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c06e0a0 msgr2=0x7f151c0b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 --2- 172.21.15.22:0/1057875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c06e0a0 0x7f151c0b97e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f150c0098e0 tx=0x7f150c02a410 comp rx=0 tx=0).stop 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 -- 172.21.15.22:0/1057875 shutdown_connections 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 --2- 172.21.15.22:0/1057875 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151c06f6e0 0x7f151c06e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 --2- 172.21.15.22:0/1057875 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c06e0a0 0x7f151c0b97e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 -- 172.21.15.22:0/1057875 >> 172.21.15.22:0/1057875 conn(0x7f151c09d840 msgr2=0x7f151c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 -- 172.21.15.22:0/1057875 shutdown_connections 2024-09-21T05:40:24.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.938+0000 7f1522225640 1 -- 172.21.15.22:0/1057875 wait complete. 2024-09-21T05:40:24.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.940+0000 7f1522225640 1 Processor -- start 2024-09-21T05:40:24.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.940+0000 7f1522225640 1 -- start start 2024-09-21T05:40:24.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f1522225640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f151c06e0a0 0x7f151c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f1522225640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151c06f6e0 0x7f151c151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f1522225640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 0x7f151c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f1522225640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f151c0cfcd0 con 0x7f151c0d0e50 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f1522225640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f151c0cfb50 con 0x7f151c06e0a0 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f1522225640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f151c0cfe50 con 0x7f151c06f6e0 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f151bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 0x7f151c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f151bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 0x7f151c151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52274/0 (socket says 172.21.15.22:52274) 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f151bfff640 1 -- 172.21.15.22:0/482350059 learned_addr learned my addr 172.21.15.22:0/482350059 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.941+0000 7f151b7fe640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f151c06e0a0 0x7f151c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151affd640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151c06f6e0 0x7f151c151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151bfff640 1 -- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151c06f6e0 msgr2=0x7f151c151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151bfff640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151c06f6e0 0x7f151c151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151bfff640 1 -- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f151c06e0a0 msgr2=0x7f151c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151bfff640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f151c06e0a0 0x7f151c150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:24.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151bfff640 1 -- 172.21.15.22:0/482350059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151c1609e0 con 0x7f151c0d0e50 2024-09-21T05:40:24.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f151bfff640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 0x7f151c151650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f151400b9e0 tx=0x7f151400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:24.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1514003b70 con 0x7f151c0d0e50 2024-09-21T05:40:24.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1514003e00 con 0x7f151c0d0e50 2024-09-21T05:40:24.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151c160700 con 0x7f151c0d0e50 2024-09-21T05:40:24.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151400cb80 con 0x7f151c0d0e50 2024-09-21T05:40:24.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.942+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151c14ef00 con 0x7f151c0d0e50 2024-09-21T05:40:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.943+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151c0d0200 con 0x7f151c0d0e50 2024-09-21T05:40:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.944+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1514004000 con 0x7f151c0d0e50 2024-09-21T05:40:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.945+0000 7f1503fff640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1508078b90 0x7f150807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.945+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f151409ab80 con 0x7f151c0d0e50 2024-09-21T05:40:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.945+0000 7f151b7fe640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1508078b90 0x7f150807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:24.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.945+0000 7f151b7fe640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1508078b90 0x7f150807b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f150c034000 tx=0x7f150c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:24.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:24.949+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1514062960 con 0x7f151c0d0e50 2024-09-21T05:40:25.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.175+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f151c0b7500 con 0x7f151c0d0e50 2024-09-21T05:40:25.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.176+0000 7f1503fff640 1 -- 172.21.15.22:0/482350059 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7f1514067390 con 0x7f151c0d0e50 2024-09-21T05:40:25.179 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:25.179 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:25.180 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:25.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1508078b90 msgr2=0x7f150807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1508078b90 0x7f150807b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f150c034000 tx=0x7f150c004730 comp rx=0 tx=0).stop 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 msgr2=0x7f151c151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 0x7f151c151650 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f151400b9e0 tx=0x7f151400bea0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 shutdown_connections 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f151c0d0e50 0x7f151c151650 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1508078b90 0x7f150807b0a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f151c06f6e0 0x7f151c151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 --2- 172.21.15.22:0/482350059 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f151c06e0a0 0x7f151c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 >> 172.21.15.22:0/482350059 conn(0x7f151c09d840 msgr2=0x7f151c0d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 shutdown_connections 2024-09-21T05:40:25.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.178+0000 7f1522225640 1 -- 172.21.15.22:0/482350059 wait complete. 2024-09-21T05:40:25.193 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:25.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 -- 172.21.15.22:0/3254427831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9380d2630 msgr2=0x7fe93815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/3254427831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9380d2630 0x7fe93815d130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe938074e30 tx=0x7fe92c02a5f0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 -- 172.21.15.22:0/3254427831 shutdown_connections 2024-09-21T05:40:25.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/3254427831 >> v2:172.21.15.134:3300/0 conn(0x7fe93815d670 0x7fe9380b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/3254427831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9380d2630 0x7fe93815d130 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 -- 172.21.15.22:0/3254427831 >> 172.21.15.22:0/3254427831 conn(0x7fe93806d550 msgr2=0x7fe938071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:25.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 -- 172.21.15.22:0/3254427831 shutdown_connections 2024-09-21T05:40:25.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.307+0000 7fe93fbc3640 1 -- 172.21.15.22:0/3254427831 wait complete. 2024-09-21T05:40:25.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.309+0000 7fe93fbc3640 1 Processor -- start 2024-09-21T05:40:25.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.309+0000 7fe93fbc3640 1 -- start start 2024-09-21T05:40:25.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93fbc3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9381570e0 0x7fe93814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93fbc3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93815d670 0x7fe938154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93fbc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 0x7fe938155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93fbc3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380b7d60 con 0x7fe93814f340 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93fbc3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380b7be0 con 0x7fe9381570e0 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93fbc3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9380b7ee0 con 0x7fe93815d670 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93e160640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 0x7fe938155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93e160640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 0x7fe938155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52302/0 (socket says 172.21.15.22:52302) 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93e160640 1 -- 172.21.15.22:0/2630159891 learned_addr learned my addr 172.21.15.22:0/2630159891 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93d15e640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93815d670 0x7fe938154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.310+0000 7fe93d95f640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9381570e0 0x7fe93814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93e160640 1 -- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93815d670 msgr2=0x7fe938154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93e160640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93815d670 0x7fe938154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93e160640 1 -- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9381570e0 msgr2=0x7fe93814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93e160640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9381570e0 0x7fe93814ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93e160640 1 -- 172.21.15.22:0/2630159891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93814f0f0 con 0x7fe93814f340 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93e160640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 0x7fe938155320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe928005d00 tx=0x7fe92800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928017070 con 0x7fe93814f340 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe928005330 con 0x7fe93814f340 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9380b2d20 con 0x7fe93814f340 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9280047a0 con 0x7fe93814f340 2024-09-21T05:40:25.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.311+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9380b3c90 con 0x7fe93814f340 2024-09-21T05:40:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.312+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9380b2a90 con 0x7fe93814f340 2024-09-21T05:40:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.313+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe928004940 con 0x7fe93814f340 2024-09-21T05:40:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.313+0000 7fe9367fc640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe924078b90 0x7fe92407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.313+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fe92809adc0 con 0x7fe93814f340 2024-09-21T05:40:25.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.314+0000 7fe93d95f640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe924078b90 0x7fe92407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.314+0000 7fe93d95f640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe924078b90 0x7fe92407b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9300012e0 tx=0x7fe930009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:25.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.318+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe928062bb0 con 0x7fe93814f340 2024-09-21T05:40:25.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.547+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe93815ff20 con 0x7fe93814f340 2024-09-21T05:40:25.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.547+0000 7fe9367fc640 1 -- 172.21.15.22:0/2630159891 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5226 (secure 0 0 0) 0x7fe93815ff20 con 0x7fe93814f340 2024-09-21T05:40:25.551 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:25.551 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":17,"btime":"2024-09-21T05:40:06:588181+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:02.941475+0000","modified":"2024-09-21T05:40:05.976698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1454316694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1454316694},{"type":"v1","addr":"172.21.15.22:6834","nonce":1454316694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-09-21T05:40:25.551 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 17 2024-09-21T05:40:25.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe924078b90 msgr2=0x7fe92407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe924078b90 0x7fe92407b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe9300012e0 tx=0x7fe930009040 comp rx=0 tx=0).stop 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 msgr2=0x7fe938155320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 0x7fe938155320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe928005d00 tx=0x7fe92800dae0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 shutdown_connections 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe93814f340 0x7fe938155320 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe924078b90 0x7fe92407b0a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe93815d670 0x7fe938154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 --2- 172.21.15.22:0/2630159891 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9381570e0 0x7fe93814ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 >> 172.21.15.22:0/2630159891 conn(0x7fe93806d550 msgr2=0x7fe93815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 shutdown_connections 2024-09-21T05:40:25.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.550+0000 7fe93fbc3640 1 -- 172.21.15.22:0/2630159891 wait complete. 2024-09-21T05:40:25.565 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 -- 172.21.15.22:0/100592442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c161f50 msgr2=0x7fb44c164370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 --2- 172.21.15.22:0/100592442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c161f50 0x7fb44c164370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb44c072850 tx=0x7fb44402aa50 comp rx=0 tx=0).stop 2024-09-21T05:40:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 -- 172.21.15.22:0/100592442 shutdown_connections 2024-09-21T05:40:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 --2- 172.21.15.22:0/100592442 >> v2:172.21.15.134:3300/0 conn(0x7fb44c0b6100 0x7fb44c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 --2- 172.21.15.22:0/100592442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c161f50 0x7fb44c164370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 --2- 172.21.15.22:0/100592442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb44c0b6680 0x7fb44c0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 -- 172.21.15.22:0/100592442 >> 172.21.15.22:0/100592442 conn(0x7fb44c0d12a0 msgr2=0x7fb44c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:25.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 -- 172.21.15.22:0/100592442 shutdown_connections 2024-09-21T05:40:25.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.678+0000 7fb450a15640 1 -- 172.21.15.22:0/100592442 wait complete. 2024-09-21T05:40:25.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.680+0000 7fb450a15640 1 Processor -- start 2024-09-21T05:40:25.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.681+0000 7fb450a15640 1 -- start start 2024-09-21T05:40:25.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb450a15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 0x7fb44c1604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb450a15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb44c0b6680 0x7fb44c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb450a15640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb44c161f50 0x7fb44c1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb450a15640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c0b20d0 con 0x7fb44c0b6100 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb450a15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb44c0b19a0 con 0x7fb44c0b6680 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb450a15640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb44c1648b0 con 0x7fb44c161f50 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 0x7fb44c1604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 0x7fb44c1604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52314/0 (socket says 172.21.15.22:52314) 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 -- 172.21.15.22:0/2915255858 learned_addr learned my addr 172.21.15.22:0/2915255858 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44ad9d640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb44c161f50 0x7fb44c1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb449d9b640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb44c0b6680 0x7fb44c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 -- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb44c161f50 msgr2=0x7fb44c1f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb44c161f50 0x7fb44c1f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 -- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb44c0b6680 msgr2=0x7fb44c15cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb44c0b6680 0x7fb44c15cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 -- 172.21.15.22:0/2915255858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44c0cb550 con 0x7fb44c0b6100 2024-09-21T05:40:25.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.682+0000 7fb44a59c640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 0x7fb44c1604e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00d750 tx=0x7fb43c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:25.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.683+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c017070 con 0x7fb44c0b6100 2024-09-21T05:40:25.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.683+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb43c009ac0 con 0x7fb44c0b6100 2024-09-21T05:40:25.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.683+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb44c15d070 con 0x7fb44c0b6100 2024-09-21T05:40:25.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.683+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c005020 con 0x7fb44c0b6100 2024-09-21T05:40:25.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.683+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44c0cdaf0 con 0x7fb44c0b6100 2024-09-21T05:40:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.684+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb44c0cd340 con 0x7fb44c0b6100 2024-09-21T05:40:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.685+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb43c0051c0 con 0x7fb44c0b6100 2024-09-21T05:40:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.685+0000 7fb442ffd640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb42c078bc0 0x7fb42c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.685+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb43c09ad30 con 0x7fb44c0b6100 2024-09-21T05:40:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.685+0000 7fb449d9b640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb42c078bc0 0x7fb42c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:25.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.686+0000 7fb449d9b640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb42c078bc0 0x7fb42c07b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0b5b50 tx=0x7fb434005d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:25.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.690+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb44c0cb550 con 0x7fb44c0b6100 2024-09-21T05:40:25.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.878+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb44c0b0de0 con 0x7fb44c0b6100 2024-09-21T05:40:25.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.879+0000 7fb442ffd640 1 -- 172.21.15.22:0/2915255858 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16717 (secure 0 0 0) 0x7fb43c063200 con 0x7fb44c0b6100 2024-09-21T05:40:25.882 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:25.883 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":32,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:04.576394+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:01.033773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:01.908775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb42c078bc0 msgr2=0x7fb42c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb42c078bc0 0x7fb42c07b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0b5b50 tx=0x7fb434005d70 comp rx=0 tx=0).stop 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 msgr2=0x7fb44c1604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 0x7fb44c1604e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb43c00d750 tx=0x7fb43c00dc10 comp rx=0 tx=0).stop 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 shutdown_connections 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb42c078bc0 0x7fb42c07b0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb44c161f50 0x7fb44c1f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb44c0b6680 0x7fb44c15cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 --2- 172.21.15.22:0/2915255858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb44c0b6100 0x7fb44c1604e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 >> 172.21.15.22:0/2915255858 conn(0x7fb44c0d12a0 msgr2=0x7fb44c06cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 shutdown_connections 2024-09-21T05:40:25.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:25.881+0000 7fb450a15640 1 -- 172.21.15.22:0/2915255858 wait complete. 2024-09-21T05:40:25.896 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 -- 172.21.15.22:0/1346329179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b9b60 msgr2=0x7fd0880d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 --2- 172.21.15.22:0/1346329179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b9b60 0x7fd0880d4d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd0780098e0 tx=0x7fd07802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 -- 172.21.15.22:0/1346329179 shutdown_connections 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 --2- 172.21.15.22:0/1346329179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880b66b0 0x7fd08806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 --2- 172.21.15.22:0/1346329179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b9b60 0x7fd0880d4d80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 -- 172.21.15.22:0/1346329179 >> 172.21.15.22:0/1346329179 conn(0x7fd08809d7f0 msgr2=0x7fd0880d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 -- 172.21.15.22:0/1346329179 shutdown_connections 2024-09-21T05:40:26.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.010+0000 7fd08dcec640 1 -- 172.21.15.22:0/1346329179 wait complete. 2024-09-21T05:40:26.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.012+0000 7fd08dcec640 1 Processor -- start 2024-09-21T05:40:26.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.012+0000 7fd08dcec640 1 -- start start 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.013+0000 7fd08dcec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 0x7fd0880c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd08dcec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880cf540 0x7fd0880b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd08dcec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b8380 0x7fd0881f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd08dcec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd08806f630 con 0x7fd0880b9b60 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd08dcec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd08806f4b0 con 0x7fd0880b8380 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd08dcec640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd08806f7b0 con 0x7fd0880cf540 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 0x7fd0880c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 0x7fd0880c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52316/0 (socket says 172.21.15.22:52316) 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 -- 172.21.15.22:0/3795056770 learned_addr learned my addr 172.21.15.22:0/3795056770 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd086ffd640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880cf540 0x7fd0880b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd087fff640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b8380 0x7fd0881f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 -- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880cf540 msgr2=0x7fd0880b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880cf540 0x7fd0880b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 -- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b8380 msgr2=0x7fd0881f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b8380 0x7fd0881f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 -- 172.21.15.22:0/3795056770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0880b3e40 con 0x7fd0880b9b60 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd087fff640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b8380 0x7fd0881f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd0877fe640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 0x7fd0880c9590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd078009a00 tx=0x7fd0780049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.014+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078031950 con 0x7fd0880b9b60 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.015+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd078031c00 con 0x7fd0880b9b60 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.015+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780330d0 con 0x7fd0880b9b60 2024-09-21T05:40:26.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.015+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0880b4120 con 0x7fd0880b9b60 2024-09-21T05:40:26.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.015+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0880b46e0 con 0x7fd0880b9b60 2024-09-21T05:40:26.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.015+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08806fcd0 con 0x7fd0880b9b60 2024-09-21T05:40:26.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.016+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd078033270 con 0x7fd0880b9b60 2024-09-21T05:40:26.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.017+0000 7fd06ffff640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd074078b80 0x7fd07407b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.017+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fd0780bbff0 con 0x7fd0880b9b60 2024-09-21T05:40:26.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.017+0000 7fd086ffd640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd074078b80 0x7fd07407b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.018+0000 7fd086ffd640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd074078b80 0x7fd07407b090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd0880b9230 tx=0x7fd07c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:26.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.021+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd07803f030 con 0x7fd0880b9b60 2024-09-21T05:40:26.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.247+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd0880d3fb0 con 0x7fd0880b9b60 2024-09-21T05:40:26.251 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:40:26.248+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:40:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.249+0000 7fd06ffff640 1 -- 172.21.15.22:0/3795056770 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fd078084430 con 0x7fd0880b9b60 2024-09-21T05:40:26.252 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:40:26.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd074078b80 msgr2=0x7fd07407b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd074078b80 0x7fd07407b090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd0880b9230 tx=0x7fd07c009040 comp rx=0 tx=0).stop 2024-09-21T05:40:26.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 msgr2=0x7fd0880c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 0x7fd0880c9590 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd078009a00 tx=0x7fd0780049d0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 shutdown_connections 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0880b8380 0x7fd0881f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd074078b80 0x7fd07407b090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0880cf540 0x7fd0880b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 --2- 172.21.15.22:0/3795056770 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0880b9b60 0x7fd0880c9590 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 >> 172.21.15.22:0/3795056770 conn(0x7fd08809d7f0 msgr2=0x7fd0880d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 shutdown_connections 2024-09-21T05:40:26.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.253+0000 7fd08dcec640 1 -- 172.21.15.22:0/3795056770 wait complete. 2024-09-21T05:40:26.260 INFO:tasks.ceph.mds.a.smithi022.stderr: -7302> 2024-09-21T05:40:08.506+0000 7f4d5997e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:40:26.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:40:26.322 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:40:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 -- 172.21.15.22:0/2517333042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 msgr2=0x7f128c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/2517333042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f128c004c10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f12a4070050 tx=0x7f128c005830 comp rx=0 tx=0).stop 2024-09-21T05:40:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 -- 172.21.15.22:0/2517333042 shutdown_connections 2024-09-21T05:40:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/2517333042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f128c004c10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 -- 172.21.15.22:0/2517333042 >> 172.21.15.22:0/2517333042 conn(0x7f12a409d7f0 msgr2=0x7f12a40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:26.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 -- 172.21.15.22:0/2517333042 shutdown_connections 2024-09-21T05:40:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.388+0000 7f12aa1a1640 1 -- 172.21.15.22:0/2517333042 wait complete. 2024-09-21T05:40:26.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.389+0000 7f12aa1a1640 1 Processor -- start 2024-09-21T05:40:26.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.390+0000 7f12aa1a1640 1 -- start start 2024-09-21T05:40:26.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12aa1a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f12a41f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12aa1a1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a41f9870 0x7f12a41f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12aa1a1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a40c8330 0x7f12a40c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12aa1a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40b6120 con 0x7f128c004800 2024-09-21T05:40:26.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12aa1a1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12a40b5fa0 con 0x7f12a40c8330 2024-09-21T05:40:26.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12aa1a1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12a40b62a0 con 0x7f12a41f9870 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f12a41f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f12a41f8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52342/0 (socket says 172.21.15.22:52342) 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 -- 172.21.15.22:0/4259267227 learned_addr learned my addr 172.21.15.22:0/4259267227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a2ffd640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a41f9870 0x7f12a41f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a3fff640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a40c8330 0x7f12a40c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 -- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a41f9870 msgr2=0x7f12a41f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a41f9870 0x7f12a41f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 -- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a40c8330 msgr2=0x7f12a40c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a40c8330 0x7f12a40c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.391+0000 7f12a37fe640 1 -- 172.21.15.22:0/4259267227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a40c9340 con 0x7f128c004800 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.392+0000 7f12a37fe640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f12a41f8cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f129400f350 tx=0x7f129400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.392+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294015070 con 0x7f128c004800 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.392+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1294011900 con 0x7f128c004800 2024-09-21T05:40:26.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.392+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294011cf0 con 0x7f128c004800 2024-09-21T05:40:26.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.392+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a40c9620 con 0x7f128c004800 2024-09-21T05:40:26.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.392+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a40c8c80 con 0x7f128c004800 2024-09-21T05:40:26.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.393+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a40b67b0 con 0x7f128c004800 2024-09-21T05:40:26.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.394+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12940103c0 con 0x7f128c004800 2024-09-21T05:40:26.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.394+0000 7f128bfff640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1290078bf0 0x7f129007b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:26.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.394+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7f129409b950 con 0x7f128c004800 2024-09-21T05:40:26.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.394+0000 7f12a2ffd640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1290078bf0 0x7f129007b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:26.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.395+0000 7f12a2ffd640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1290078bf0 0x7f129007b100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1298035000 tx=0x7f12980046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:26.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.399+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1294063e80 con 0x7f128c004800 2024-09-21T05:40:26.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:26.627+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f12a40c97f0 con 0x7f128c004800 2024-09-21T05:40:27.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.251+0000 7f128bfff640 1 -- 172.21.15.22:0/4259267227 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f1294063780 con 0x7f128c004800 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1290078bf0 msgr2=0x7f129007b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1290078bf0 0x7f129007b100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1298035000 tx=0x7f12980046c0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 msgr2=0x7f12a41f8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f12a41f8cd0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f129400f350 tx=0x7f129400f810 comp rx=0 tx=0).stop 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 shutdown_connections 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12a40c8330 0x7f12a40c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1290078bf0 0x7f129007b100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12a41f9870 0x7f12a41f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.253+0000 7f12aa1a1640 1 --2- 172.21.15.22:0/4259267227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f128c004800 0x7f12a41f8cd0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.254+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 >> 172.21.15.22:0/4259267227 conn(0x7f12a409d7f0 msgr2=0x7f12a41eaec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:27.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.254+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 shutdown_connections 2024-09-21T05:40:27.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.254+0000 7f12aa1a1640 1 -- 172.21.15.22:0/4259267227 wait complete. 2024-09-21T05:40:27.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.392+0000 7fa987d94640 1 -- 172.21.15.22:0/4094167594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 msgr2=0x7fa9800d1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.392+0000 7fa987d94640 1 --2- 172.21.15.22:0/4094167594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9800d1ce0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa9800cd490 tx=0x7fa97c02a5f0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.392+0000 7fa987d94640 1 -- 172.21.15.22:0/4094167594 shutdown_connections 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.392+0000 7fa987d94640 1 --2- 172.21.15.22:0/4094167594 >> v2:172.21.15.134:3300/0 conn(0x7fa980071090 0x7fa9800714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.392+0000 7fa987d94640 1 --2- 172.21.15.22:0/4094167594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9800d1ce0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.392+0000 7fa987d94640 1 -- 172.21.15.22:0/4094167594 >> 172.21.15.22:0/4094167594 conn(0x7fa9800ced50 msgr2=0x7fa9800cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.393+0000 7fa987d94640 1 -- 172.21.15.22:0/4094167594 shutdown_connections 2024-09-21T05:40:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.393+0000 7fa987d94640 1 -- 172.21.15.22:0/4094167594 wait complete. 2024-09-21T05:40:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.394+0000 7fa987d94640 1 Processor -- start 2024-09-21T05:40:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.394+0000 7fa987d94640 1 -- start start 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa987d94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa980071090 0x7fa9800cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa987d94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9801f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa987d94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa98015cc30 0x7fa9801f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa987d94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9800d01a0 con 0x7fa98015a310 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa987d94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9800d0020 con 0x7fa980071090 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa987d94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9800d0320 con 0x7fa98015cc30 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9801f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9801f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52374/0 (socket says 172.21.15.22:52374) 2024-09-21T05:40:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 -- 172.21.15.22:0/3926538534 learned_addr learned my addr 172.21.15.22:0/3926538534 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa985b30640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa980071090 0x7fa9800cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa986331640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa98015cc30 0x7fa9801f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 -- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa98015cc30 msgr2=0x7fa9801f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa98015cc30 0x7fa9801f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 -- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa980071090 msgr2=0x7fa9800cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa980071090 0x7fa9800cc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 -- 172.21.15.22:0/3926538534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9801533a0 con 0x7fa98015a310 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa98532f640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9801f7490 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa97000c3a0 tx=0x7fa97000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.396+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970017070 con 0x7fa98015a310 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.397+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa980153680 con 0x7fa98015a310 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.397+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9700053c0 con 0x7fa98015a310 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.397+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa970003850 con 0x7fa98015a310 2024-09-21T05:40:27.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.397+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9801f6c90 con 0x7fa98015a310 2024-09-21T05:40:27.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.398+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa980153110 con 0x7fa98015a310 2024-09-21T05:40:27.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.398+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa97001a070 con 0x7fa98015a310 2024-09-21T05:40:27.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.399+0000 7fa9767fc640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa964078b90 0x7fa96407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:27.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.400+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7fa97009a100 con 0x7fa98015a310 2024-09-21T05:40:27.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.400+0000 7fa985b30640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa964078b90 0x7fa96407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:27.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.401+0000 7fa985b30640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa964078b90 0x7fa96407b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97c033000 tx=0x7fa97c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:27.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.404+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa970061e10 con 0x7fa98015a310 2024-09-21T05:40:27.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:27.597+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9800714a0 con 0x7fa98015a310 2024-09-21T05:40:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.255+0000 7fa9767fc640 1 -- 172.21.15.22:0/3926538534 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7fa970066840 con 0x7fa98015a310 2024-09-21T05:40:28.259 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:40:28.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.260+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa964078b90 msgr2=0x7fa96407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.260+0000 7fa987d94640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa964078b90 0x7fa96407b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97c033000 tx=0x7fa97c004730 comp rx=0 tx=0).stop 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.260+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 msgr2=0x7fa9801f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.260+0000 7fa987d94640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9801f7490 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa97000c3a0 tx=0x7fa97000c860 comp rx=0 tx=0).stop 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 shutdown_connections 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa964078b90 0x7fa96407b0a0 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa97c033000 tx=0x7fa97c004730 comp rx=0 tx=0).stop 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa98015cc30 0x7fa9801f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa98015a310 0x7fa9801f7490 secure :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa97000c3a0 tx=0x7fa97000c860 comp rx=0 tx=0).stop 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 --2- 172.21.15.22:0/3926538534 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa980071090 0x7fa9800cc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:28.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 >> 172.21.15.22:0/3926538534 conn(0x7fa9800ced50 msgr2=0x7fa98015e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 shutdown_connections 2024-09-21T05:40:28.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.261+0000 7fa987d94640 1 -- 172.21.15.22:0/3926538534 wait complete. 2024-09-21T05:40:28.278 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 -- 172.21.15.22:0/2409852312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 msgr2=0x7f394806d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 --2- 172.21.15.22:0/2409852312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394806d1f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3948073730 tx=0x7f393002a560 comp rx=0 tx=0).stop 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 -- 172.21.15.22:0/2409852312 shutdown_connections 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 --2- 172.21.15.22:0/2409852312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394806d1f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 --2- 172.21.15.22:0/2409852312 >> v2:172.21.15.134:3300/0 conn(0x7f39480b38f0 0x7f39480b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 -- 172.21.15.22:0/2409852312 >> 172.21.15.22:0/2409852312 conn(0x7f3948074190 msgr2=0x7f39480b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 -- 172.21.15.22:0/2409852312 shutdown_connections 2024-09-21T05:40:28.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.401+0000 7f394cb89640 1 -- 172.21.15.22:0/2409852312 wait complete. 2024-09-21T05:40:28.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.403+0000 7f394cb89640 1 Processor -- start 2024-09-21T05:40:28.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.403+0000 7f394cb89640 1 -- start start 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394814f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39480b38f0 0x7f394809b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3948160500 0x7f394815dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39480b5f20 con 0x7f394806cde0 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39480b51a0 con 0x7f39480b38f0 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39480b34d0 con 0x7f3948160500 2024-09-21T05:40:28.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394814f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394814f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52398/0 (socket says 172.21.15.22:52398) 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 -- 172.21.15.22:0/2399971471 learned_addr learned my addr 172.21.15.22:0/2399971471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f3946d9d640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3948160500 0x7f394815dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f3945d9b640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39480b38f0 0x7f394809b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 -- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3948160500 msgr2=0x7f394815dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3948160500 0x7f394815dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 -- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39480b38f0 msgr2=0x7f394809b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39480b38f0 0x7f394809b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 -- 172.21.15.22:0/2399971471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394809bd90 con 0x7f394806cde0 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f3946d9d640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3948160500 0x7f394815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:28.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f3945d9b640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39480b38f0 0x7f394809b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394659c640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394814f890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f393800ea20 tx=0x7f393800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393800cde0 con 0x7f394806cde0 2024-09-21T05:40:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3938011560 con 0x7f394806cde0 2024-09-21T05:40:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394809c070 con 0x7f394806cde0 2024-09-21T05:40:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938011990 con 0x7f394806cde0 2024-09-21T05:40:28.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.405+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394809da90 con 0x7f394806cde0 2024-09-21T05:40:28.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.406+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394806d1f0 con 0x7f394806cde0 2024-09-21T05:40:28.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.407+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3938011b30 con 0x7f394806cde0 2024-09-21T05:40:28.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.408+0000 7f393effd640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3928078b90 0x7f392807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:28.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.408+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7f393809b890 con 0x7f394806cde0 2024-09-21T05:40:28.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.408+0000 7f3945d9b640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3928078b90 0x7f392807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:28.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.409+0000 7f3945d9b640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3928078b90 0x7f392807b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3930033000 tx=0x7f3930004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:28.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.413+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3938016030 con 0x7f394806cde0 2024-09-21T05:40:28.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:28.601+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3948074ac0 con 0x7f394806cde0 2024-09-21T05:40:29.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.262+0000 7f393effd640 1 -- 172.21.15.22:0/2399971471 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7f3938011e20 con 0x7f394806cde0 2024-09-21T05:40:29.265 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:40:29.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3928078b90 msgr2=0x7f392807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3928078b90 0x7f392807b0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3930033000 tx=0x7f3930004730 comp rx=0 tx=0).stop 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 msgr2=0x7f394814f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394814f890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f393800ea20 tx=0x7f393800eee0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 shutdown_connections 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3928078b90 0x7f392807b0a0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3930033000 tx=0x7f3930004730 comp rx=0 tx=0).stop 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3948160500 0x7f394815dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f39480b38f0 0x7f394809b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 --2- 172.21.15.22:0/2399971471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f394806cde0 0x7f394814f890 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.265+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 >> 172.21.15.22:0/2399971471 conn(0x7f3948074190 msgr2=0x7f39480b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:29.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.266+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 shutdown_connections 2024-09-21T05:40:29.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.266+0000 7f394cb89640 1 -- 172.21.15.22:0/2399971471 wait complete. 2024-09-21T05:40:29.284 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.402+0000 7f818a2c5640 1 -- 172.21.15.22:0/1830864166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 msgr2=0x7f81840d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.402+0000 7f818a2c5640 1 --2- 172.21.15.22:0/1830864166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840d0d80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8184073f50 tx=0x7f817802a560 comp rx=0 tx=0).stop 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.402+0000 7f818a2c5640 1 -- 172.21.15.22:0/1830864166 shutdown_connections 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.402+0000 7f818a2c5640 1 --2- 172.21.15.22:0/1830864166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840d0d80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.402+0000 7f818a2c5640 1 --2- 172.21.15.22:0/1830864166 >> v2:172.21.15.134:3300/0 conn(0x7f81840b51c0 0x7f818415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.402+0000 7f818a2c5640 1 -- 172.21.15.22:0/1830864166 >> 172.21.15.22:0/1830864166 conn(0x7f8184072ae0 msgr2=0x7f8184072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.403+0000 7f818a2c5640 1 -- 172.21.15.22:0/1830864166 shutdown_connections 2024-09-21T05:40:29.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.403+0000 7f818a2c5640 1 -- 172.21.15.22:0/1830864166 wait complete. 2024-09-21T05:40:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.404+0000 7f818a2c5640 1 Processor -- start 2024-09-21T05:40:29.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.404+0000 7f818a2c5640 1 -- start start 2024-09-21T05:40:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f818a2c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81840b51c0 0x7f81840ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f818a2c5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f818a2c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81840d12c0 0x7f81840b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f818a2c5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f81840d19f0 con 0x7f81840ccbd0 2024-09-21T05:40:29.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f818a2c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8184160a60 con 0x7f81840b51c0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f818a2c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f81841512e0 con 0x7f81840d12c0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840a9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52414/0 (socket says 172.21.15.22:52414) 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 -- 172.21.15.22:0/305725337 learned_addr learned my addr 172.21.15.22:0/305725337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f8183fff640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81840b51c0 0x7f81840ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f8188862640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81840d12c0 0x7f81840b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 -- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81840d12c0 msgr2=0x7f81840b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81840d12c0 0x7f81840b6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 -- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81840b51c0 msgr2=0x7f81840ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81840b51c0 0x7f81840ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 -- 172.21.15.22:0/305725337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81840ae2d0 con 0x7f81840ccbd0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.406+0000 7f81837fe640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840a9c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8178034040 tx=0x7f81780039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.407+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817803e030 con 0x7f81840ccbd0 2024-09-21T05:40:29.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.407+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81780047a0 con 0x7f81840ccbd0 2024-09-21T05:40:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.407+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8184150240 con 0x7f81840ccbd0 2024-09-21T05:40:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.407+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8178035040 con 0x7f81840ccbd0 2024-09-21T05:40:29.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.407+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8184150800 con 0x7f81840ccbd0 2024-09-21T05:40:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.408+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8184160f90 con 0x7f81840ccbd0 2024-09-21T05:40:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.408+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8178041030 con 0x7f81840ccbd0 2024-09-21T05:40:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.409+0000 7f8180ff9640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8170078b90 0x7f817007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.409+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f81780bb3b0 con 0x7f81840ccbd0 2024-09-21T05:40:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.409+0000 7f8183fff640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8170078b90 0x7f817007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.410+0000 7f8183fff640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8170078b90 0x7f817007b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f817400b810 tx=0x7f8174008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:29.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.413+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8178083a60 con 0x7f81840ccbd0 2024-09-21T05:40:29.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.600+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f81840690b0 con 0x7f81840ccbd0 2024-09-21T05:40:29.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.604+0000 7f8180ff9640 1 -- 172.21.15.22:0/305725337 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f8178083360 con 0x7f81840ccbd0 2024-09-21T05:40:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8170078b90 msgr2=0x7f817007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8170078b90 0x7f817007b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f817400b810 tx=0x7f8174008040 comp rx=0 tx=0).stop 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 msgr2=0x7f81840a9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840a9c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8178034040 tx=0x7f81780039e0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 shutdown_connections 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8170078b90 0x7f817007b0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f81840d12c0 0x7f81840b6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f81840ccbd0 0x7f81840a9c70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 --2- 172.21.15.22:0/305725337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f81840b51c0 0x7f81840ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 >> 172.21.15.22:0/305725337 conn(0x7f8184072ae0 msgr2=0x7f81840b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 shutdown_connections 2024-09-21T05:40:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.606+0000 7f818a2c5640 1 -- 172.21.15.22:0/305725337 wait complete. 2024-09-21T05:40:29.620 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-09-21T05:40:29.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 -- 172.21.15.22:0/1586496050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdf8005300 msgr2=0x7fbdf800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 --2- 172.21.15.22:0/1586496050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdf8005300 0x7fbdf800b9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800bbe0 tx=0x7fbdf8005830 comp rx=0 tx=0).stop 2024-09-21T05:40:29.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 -- 172.21.15.22:0/1586496050 shutdown_connections 2024-09-21T05:40:29.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 --2- 172.21.15.22:0/1586496050 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbdf8005300 0x7fbdf800b9c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 -- 172.21.15.22:0/1586496050 >> 172.21.15.22:0/1586496050 conn(0x7fbe10072ae0 msgr2=0x7fbe10072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:29.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 -- 172.21.15.22:0/1586496050 shutdown_connections 2024-09-21T05:40:29.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.734+0000 7fbe17d01640 1 -- 172.21.15.22:0/1586496050 wait complete. 2024-09-21T05:40:29.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.736+0000 7fbe17d01640 1 Processor -- start 2024-09-21T05:40:29.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.736+0000 7fbe17d01640 1 -- start start 2024-09-21T05:40:29.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe17d01640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf8005300 0x7fbe1014dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe17d01640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1006f590 0x7fbe1014e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe17d01640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 0x7fbe1014efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe17d01640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1015ca70 con 0x7fbe1014eba0 2024-09-21T05:40:29.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe17d01640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1015c8f0 con 0x7fbdf8005300 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe17d01640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1015cbf0 con 0x7fbe1006f590 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 0x7fbe1014efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 0x7fbe1014efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52428/0 (socket says 172.21.15.22:52428) 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 -- 172.21.15.22:0/951169985 learned_addr learned my addr 172.21.15.22:0/951169985 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe15a9d640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf8005300 0x7fbe1014dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1529c640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1006f590 0x7fbe1014e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 -- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1006f590 msgr2=0x7fbe1014e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1006f590 0x7fbe1014e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 -- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf8005300 msgr2=0x7fbe1014dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf8005300 0x7fbe1014dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.737+0000 7fbe1629e640 1 -- 172.21.15.22:0/951169985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe10151a60 con 0x7fbe1014eba0 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.738+0000 7fbe1629e640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 0x7fbe1014efb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800bd00 tx=0x7fbe08031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:29.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.738+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe08034040 con 0x7fbe1014eba0 2024-09-21T05:40:29.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.738+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe0803c070 con 0x7fbe1014eba0 2024-09-21T05:40:29.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.738+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1006e4f0 con 0x7fbe1014eba0 2024-09-21T05:40:29.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.738+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800b7b0 con 0x7fbe1014eba0 2024-09-21T05:40:29.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.738+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1006eab0 con 0x7fbe1014eba0 2024-09-21T05:40:29.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.739+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe100d1000 con 0x7fbe1014eba0 2024-09-21T05:40:29.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.740+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe0802e420 con 0x7fbe1014eba0 2024-09-21T05:40:29.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.740+0000 7fbe0e7fc640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdfc078bc0 0x7fbdfc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:29.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.740+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fbe080bb530 con 0x7fbe1014eba0 2024-09-21T05:40:29.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.740+0000 7fbe15a9d640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdfc078bc0 0x7fbdfc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:29.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.741+0000 7fbe15a9d640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdfc078bc0 0x7fbdfc07b0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe00006fd0 tx=0x7fbe00006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:29.745+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbe10151a60 con 0x7fbe1014eba0 2024-09-21T05:40:30.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.017+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fbe100b3940 con 0x7fbe1014eba0 2024-09-21T05:40:30.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.263+0000 7fbe0e7fc640 1 -- 172.21.15.22:0/951169985 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v79) ==== 142+0+0 (secure 0 0 0) 0x7fbe08031e20 con 0x7fbe1014eba0 2024-09-21T05:40:30.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdfc078bc0 msgr2=0x7fbdfc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdfc078bc0 0x7fbdfc07b0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe00006fd0 tx=0x7fbe00006b20 comp rx=0 tx=0).stop 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 msgr2=0x7fbe1014efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 0x7fbe1014efb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800bd00 tx=0x7fbe08031c30 comp rx=0 tx=0).stop 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 shutdown_connections 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe1014eba0 0x7fbe1014efb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdfc078bc0 0x7fbdfc07b0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1006f590 0x7fbe1014e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 --2- 172.21.15.22:0/951169985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbdf8005300 0x7fbe1014dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 >> 172.21.15.22:0/951169985 conn(0x7fbe10072ae0 msgr2=0x7fbe100b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 shutdown_connections 2024-09-21T05:40:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.265+0000 7fbe17d01640 1 -- 172.21.15.22:0/951169985 wait complete. 2024-09-21T05:40:30.280 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-21T05:40:30.280 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-09-21T05:40:30.280 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:40:30.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 -- 172.21.15.22:0/1535782214 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a2815c5e0 msgr2=0x7f2a281609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 --2- 172.21.15.22:0/1535782214 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a2815c5e0 0x7f2a281609c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2a10005fd0 tx=0x7f2a1002a540 comp rx=0 tx=0).stop 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 -- 172.21.15.22:0/1535782214 shutdown_connections 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 --2- 172.21.15.22:0/1535782214 >> v2:172.21.15.134:3300/0 conn(0x7f2a2806cde0 0x7f2a2806d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 --2- 172.21.15.22:0/1535782214 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a2815c5e0 0x7f2a281609c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 -- 172.21.15.22:0/1535782214 >> 172.21.15.22:0/1535782214 conn(0x7f2a280d1020 msgr2=0x7f2a280d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 -- 172.21.15.22:0/1535782214 shutdown_connections 2024-09-21T05:40:30.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.395+0000 7f2a2759e640 1 -- 172.21.15.22:0/1535782214 wait complete. 2024-09-21T05:40:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 Processor -- start 2024-09-21T05:40:30.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 -- start start 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a2806cde0 0x7f2a280b6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a280b00d0 0x7f2a280b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 0x7f2a28157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2815d390 con 0x7f2a2815c5e0 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2815d210 con 0x7f2a2806cde0 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.398+0000 7f2a2759e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2a2815d510 con 0x7f2a280b00d0 2024-09-21T05:40:30.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 0x7f2a28157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 0x7f2a28157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52438/0 (socket says 172.21.15.22:52438) 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 -- 172.21.15.22:0/3452821035 learned_addr learned my addr 172.21.15.22:0/3452821035 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a2659c640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a2806cde0 0x7f2a280b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a1ffff640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a280b00d0 0x7f2a280b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 -- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a280b00d0 msgr2=0x7f2a280b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a280b00d0 0x7f2a280b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 -- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a2806cde0 msgr2=0x7f2a280b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a2806cde0 0x7f2a280b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 -- 172.21.15.22:0/3452821035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a28155f70 con 0x7f2a2815c5e0 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a26d9d640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 0x7f2a28157b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000ec20 tx=0x7f2a2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:30.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20003b70 con 0x7f2a2815c5e0 2024-09-21T05:40:30.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a20003e00 con 0x7f2a2815c5e0 2024-09-21T05:40:30.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a28156250 con 0x7f2a2815c5e0 2024-09-21T05:40:30.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20004280 con 0x7f2a2815c5e0 2024-09-21T05:40:30.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.399+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a28156810 con 0x7f2a2815c5e0 2024-09-21T05:40:30.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.400+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a280770e0 con 0x7f2a2815c5e0 2024-09-21T05:40:30.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.401+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2a20004420 con 0x7f2a2815c5e0 2024-09-21T05:40:30.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.402+0000 7f2a1effd640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2a08078b90 0x7f2a0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.402+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f2a2009b690 con 0x7f2a2815c5e0 2024-09-21T05:40:30.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.402+0000 7f2a2659c640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2a08078b90 0x7f2a0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.403+0000 7f2a2659c640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2a08078b90 0x7f2a0807b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2a180017e0 tx=0x7f2a18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:30.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.407+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a280770e0 con 0x7f2a2815c5e0 2024-09-21T05:40:30.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.594+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a2806d1f0 con 0x7f2a2815c5e0 2024-09-21T05:40:30.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.594+0000 7f2a1effd640 1 -- 172.21.15.22:0/3452821035 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13020 (secure 0 0 0) 0x7f2a200638a0 con 0x7f2a2815c5e0 2024-09-21T05:40:30.598 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:30.598 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:29.258097+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6834/1454316694":"2024-09-22T05:40:26.250029+0000","172.21.15.22:6832/1454316694":"2024-09-22T05:40:26.250029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2a08078b90 msgr2=0x7f2a0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2a08078b90 0x7f2a0807b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2a180017e0 tx=0x7f2a18009040 comp rx=0 tx=0).stop 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 msgr2=0x7f2a28157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 0x7f2a28157b90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2a2000ec20 tx=0x7f2a2000c6a0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 shutdown_connections 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2a2815c5e0 0x7f2a28157b90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2a08078b90 0x7f2a0807b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2a280b00d0 0x7f2a280b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 --2- 172.21.15.22:0/3452821035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2a2806cde0 0x7f2a280b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.596+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 >> 172.21.15.22:0/3452821035 conn(0x7f2a280d1020 msgr2=0x7f2a280af580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.597+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 shutdown_connections 2024-09-21T05:40:30.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.597+0000 7f2a2759e640 1 -- 172.21.15.22:0/3452821035 wait complete. 2024-09-21T05:40:30.611 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:40:30.611 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:40:30.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 -- 172.21.15.22:0/389882592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 msgr2=0x7f31340cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 --2- 172.21.15.22:0/389882592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 0x7f31340cf060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f312c005fd0 tx=0x7f312c02a940 comp rx=0 tx=0).stop 2024-09-21T05:40:30.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 -- 172.21.15.22:0/389882592 shutdown_connections 2024-09-21T05:40:30.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 --2- 172.21.15.22:0/389882592 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 0x7f31340cf060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 --2- 172.21.15.22:0/389882592 >> v2:172.21.15.134:3300/0 conn(0x7f3134161020 0x7f31340ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 -- 172.21.15.22:0/389882592 >> 172.21.15.22:0/389882592 conn(0x7f313406d4f0 msgr2=0x7f3134071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:30.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 -- 172.21.15.22:0/389882592 shutdown_connections 2024-09-21T05:40:30.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.728+0000 7f313a84d640 1 -- 172.21.15.22:0/389882592 wait complete. 2024-09-21T05:40:30.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 Processor -- start 2024-09-21T05:40:30.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 -- start start 2024-09-21T05:40:30.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 0x7f313414c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134156f10 0x7f313414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 0x7f31340b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f31340d08c0 con 0x7f3134161020 2024-09-21T05:40:30.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f31340d0740 con 0x7f3134156f10 2024-09-21T05:40:30.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f313a84d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f31340d0a40 con 0x7f31340cec50 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f3138dea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 0x7f31340b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f3138dea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 0x7f31340b74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52450/0 (socket says 172.21.15.22:52450) 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f3138dea640 1 -- 172.21.15.22:0/1980342143 learned_addr learned my addr 172.21.15.22:0/1980342143 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f3133fff640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 0x7f313414c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.731+0000 7f312bfff640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134156f10 0x7f313414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f3138dea640 1 -- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 msgr2=0x7f313414c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f3138dea640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 0x7f313414c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f3138dea640 1 -- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134156f10 msgr2=0x7f313414f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f3138dea640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134156f10 0x7f313414f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f3138dea640 1 -- 172.21.15.22:0/1980342143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313414c580 con 0x7f3134161020 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f3138dea640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 0x7f31340b74d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f312c02af30 tx=0x7f312c031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c03e070 con 0x7f3134161020 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f312c004540 con 0x7f3134161020 2024-09-21T05:40:30.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31341502a0 con 0x7f3134161020 2024-09-21T05:40:30.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c030430 con 0x7f3134161020 2024-09-21T05:40:30.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.732+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3134160280 con 0x7f3134161020 2024-09-21T05:40:30.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.733+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31340d0f60 con 0x7f3134161020 2024-09-21T05:40:30.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.734+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f312c03a440 con 0x7f3134161020 2024-09-21T05:40:30.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.734+0000 7f31317fa640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3114078bc0 0x7f311407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:30.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.734+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f312c0bc330 con 0x7f3134161020 2024-09-21T05:40:30.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.735+0000 7f3133fff640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3114078bc0 0x7f311407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:30.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.735+0000 7f3133fff640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3114078bc0 0x7f311407b0d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31240095f0 tx=0x7f3124005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:30.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.740+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f312c0844b0 con 0x7f3134161020 2024-09-21T05:40:30.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.926+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3134155200 con 0x7f3134161020 2024-09-21T05:40:30.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.926+0000 7f31317fa640 1 -- 172.21.15.22:0/1980342143 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f312c088ee0 con 0x7f3134161020 2024-09-21T05:40:30.929 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.928+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3114078bc0 msgr2=0x7f311407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.928+0000 7f313a84d640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3114078bc0 0x7f311407b0d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f31240095f0 tx=0x7f3124005f90 comp rx=0 tx=0).stop 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.928+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 msgr2=0x7f31340b74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.928+0000 7f313a84d640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 0x7f31340b74d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f312c02af30 tx=0x7f312c031ea0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.928+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 shutdown_connections 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.928+0000 7f313a84d640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3134161020 0x7f31340b74d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.929+0000 7f313a84d640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3134156f10 0x7f313414f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.929+0000 7f313a84d640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3114078bc0 0x7f311407b0d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.929+0000 7f313a84d640 1 --2- 172.21.15.22:0/1980342143 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f31340cec50 0x7f313414c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.929+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 >> 172.21.15.22:0/1980342143 conn(0x7f313406d4f0 msgr2=0x7f3134155720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.929+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 shutdown_connections 2024-09-21T05:40:30.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:30.929+0000 7f313a84d640 1 -- 172.21.15.22:0/1980342143 wait complete. 2024-09-21T05:40:30.943 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 -- 172.21.15.22:0/2723092974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 msgr2=0x7f21100b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 --2- 172.21.15.22:0/2723092974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21100b4520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2104005fd0 tx=0x7f210402a670 comp rx=0 tx=0).stop 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 -- 172.21.15.22:0/2723092974 shutdown_connections 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 --2- 172.21.15.22:0/2723092974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21100b4520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 -- 172.21.15.22:0/2723092974 >> 172.21.15.22:0/2723092974 conn(0x7f211006def0 msgr2=0x7f211006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 -- 172.21.15.22:0/2723092974 shutdown_connections 2024-09-21T05:40:31.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.056+0000 7f2117ec5640 1 -- 172.21.15.22:0/2723092974 wait complete. 2024-09-21T05:40:31.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.058+0000 7f2117ec5640 1 Processor -- start 2024-09-21T05:40:31.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.058+0000 7f2117ec5640 1 -- start start 2024-09-21T05:40:31.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.059+0000 7f2117ec5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2110074830 0x7f2110142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:31.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21101f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:31.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2110155a20 0x7f21101516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:31.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f211006d170 con 0x7f21100b4110 2024-09-21T05:40:31.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f211006cff0 con 0x7f2110074830 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f211006d2f0 con 0x7f2110155a20 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21101f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21101f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52478/0 (socket says 172.21.15.22:52478) 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 -- 172.21.15.22:0/3071434233 learned_addr learned my addr 172.21.15.22:0/3071434233 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2116462640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2110155a20 0x7f21101516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115c61640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2110074830 0x7f2110142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 -- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2110155a20 msgr2=0x7f21101516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2110155a20 0x7f21101516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 -- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2110074830 msgr2=0x7f2110142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2110074830 0x7f2110142510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 -- 172.21.15.22:0/3071434233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21101f6240 con 0x7f21100b4110 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2116462640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2110155a20 0x7f21101516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115c61640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2110074830 0x7f2110142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:31.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2115460640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21101f5780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2104034040 tx=0x7f2104003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:31.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210403e030 con 0x7f21100b4110 2024-09-21T05:40:31.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2104003e00 con 0x7f21100b4110 2024-09-21T05:40:31.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104035040 con 0x7f21100b4110 2024-09-21T05:40:31.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21101f6520 con 0x7f21100b4110 2024-09-21T05:40:31.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.060+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2110142a50 con 0x7f21100b4110 2024-09-21T05:40:31.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.061+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21100b4520 con 0x7f21100b4110 2024-09-21T05:40:31.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.062+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2104041030 con 0x7f21100b4110 2024-09-21T05:40:31.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.063+0000 7f210e7fc640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20fc078bc0 0x7f20fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:31.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.063+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f21040bb2a0 con 0x7f21100b4110 2024-09-21T05:40:31.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.063+0000 7f2115c61640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20fc078bc0 0x7f20fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:31.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.064+0000 7f2115c61640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20fc078bc0 0x7f20fc07b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2100007fd0 tx=0x7f2100009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:31.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.067+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2104083a80 con 0x7f21100b4110 2024-09-21T05:40:31.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:31.329+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f21101f5fd0 con 0x7f21100b4110 2024-09-21T05:40:32.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.270+0000 7f210e7fc640 1 -- 172.21.15.22:0/3071434233 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v81) ==== 152+0+0 (secure 0 0 0) 0x7f2104083380 con 0x7f21100b4110 2024-09-21T05:40:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20fc078bc0 msgr2=0x7f20fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20fc078bc0 0x7f20fc07b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2100007fd0 tx=0x7f2100009040 comp rx=0 tx=0).stop 2024-09-21T05:40:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 msgr2=0x7f21101f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21101f5780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2104034040 tx=0x7f2104003910 comp rx=0 tx=0).stop 2024-09-21T05:40:32.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 shutdown_connections 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f20fc078bc0 0x7f20fc07b0d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2110155a20 0x7f21101516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21100b4110 0x7f21101f5780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 --2- 172.21.15.22:0/3071434233 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2110074830 0x7f2110142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 >> 172.21.15.22:0/3071434233 conn(0x7f211006def0 msgr2=0x7f211015ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 shutdown_connections 2024-09-21T05:40:32.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.273+0000 7f2117ec5640 1 -- 172.21.15.22:0/3071434233 wait complete. 2024-09-21T05:40:32.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:40:32.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 -- 172.21.15.22:0/3027723624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 msgr2=0x7f51a0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 --2- 172.21.15.22:0/3027723624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51a0003ec0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c640 tx=0x7f51a0005830 comp rx=0 tx=0).stop 2024-09-21T05:40:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 -- 172.21.15.22:0/3027723624 shutdown_connections 2024-09-21T05:40:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 --2- 172.21.15.22:0/3027723624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51a0003ec0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 -- 172.21.15.22:0/3027723624 >> 172.21.15.22:0/3027723624 conn(0x7f51b4072ae0 msgr2=0x7f51b4072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 -- 172.21.15.22:0/3027723624 shutdown_connections 2024-09-21T05:40:32.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.412+0000 7f51b8990640 1 -- 172.21.15.22:0/3027723624 wait complete. 2024-09-21T05:40:32.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.414+0000 7f51b8990640 1 Processor -- start 2024-09-21T05:40:32.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.414+0000 7f51b8990640 1 -- start start 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b8990640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51b40b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b8990640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51b41f6010 0x7f51b4152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b8990640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51b41f6420 0x7f51b4152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b8990640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51b40d1a60 con 0x7f51a0003a50 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b8990640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51b40d18e0 con 0x7f51b41f6420 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b8990640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51b40d1100 con 0x7f51b41f6010 2024-09-21T05:40:32.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51b40b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51b40b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52496/0 (socket says 172.21.15.22:52496) 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.415+0000 7f51b259c640 1 -- 172.21.15.22:0/1973157354 learned_addr learned my addr 172.21.15.22:0/1973157354 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51b41f6010 0x7f51b4152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b2d9d640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51b41f6420 0x7f51b4152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b259c640 1 -- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51b41f6010 msgr2=0x7f51b4152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b259c640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51b41f6010 0x7f51b4152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b259c640 1 -- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51b41f6420 msgr2=0x7f51b4152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b259c640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51b41f6420 0x7f51b4152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b259c640 1 -- 172.21.15.22:0/1973157354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b4154880 con 0x7f51a0003a50 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b2d9d640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51b41f6420 0x7f51b4152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:32.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51b41f6010 0x7f51b4152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b259c640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51b40b0d80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f51a4000f80 tx=0x7f51a4001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a402bbb0 con 0x7f51a0003a50 2024-09-21T05:40:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51a403a070 con 0x7f51a0003a50 2024-09-21T05:40:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a4001250 con 0x7f51a0003a50 2024-09-21T05:40:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b4154b60 con 0x7f51a0003a50 2024-09-21T05:40:32.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.416+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b4155120 con 0x7f51a0003a50 2024-09-21T05:40:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.417+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b415c9d0 con 0x7f51a0003a50 2024-09-21T05:40:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.418+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51a403c070 con 0x7f51a0003a50 2024-09-21T05:40:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.419+0000 7f51aaffd640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5194078d60 0x7f519407b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.419+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f51a40baa30 con 0x7f51a0003a50 2024-09-21T05:40:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.419+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5194078d60 0x7f519407b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.419+0000 7f51b1d9b640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5194078d60 0x7f519407b270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f519c005d00 tx=0x7f519c005ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:32.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.423+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51a4083170 con 0x7f51a0003a50 2024-09-21T05:40:32.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.613+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f51b40b27c0 con 0x7f51a0003a50 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.614+0000 7f51aaffd640 1 -- 172.21.15.22:0/1973157354 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7f51a4082a70 con 0x7f51a0003a50 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:40:32.616 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:40:32.617 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:40:32.618 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5194078d60 msgr2=0x7f519407b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5194078d60 0x7f519407b270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f519c005d00 tx=0x7f519c005ef0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 msgr2=0x7f51b40b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51b40b0d80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f51a4000f80 tx=0x7f51a4001b20 comp rx=0 tx=0).stop 2024-09-21T05:40:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 shutdown_connections 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51b41f6420 0x7f51b4152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5194078d60 0x7f519407b270 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51b41f6010 0x7f51b4152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 --2- 172.21.15.22:0/1973157354 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51a0003a50 0x7f51b40b0d80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 >> 172.21.15.22:0/1973157354 conn(0x7f51b4072ae0 msgr2=0x7f51b40b6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 shutdown_connections 2024-09-21T05:40:32.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.616+0000 7f51b8990640 1 -- 172.21.15.22:0/1973157354 wait complete. 2024-09-21T05:40:32.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:40:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 -- 172.21.15.22:0/1812159154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980d2ed0 msgr2=0x7f3d980d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/1812159154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980d2ed0 0x7f3d980d1a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009820 tx=0x7f3d8802a510 comp rx=0 tx=0).stop 2024-09-21T05:40:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 -- 172.21.15.22:0/1812159154 shutdown_connections 2024-09-21T05:40:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/1812159154 >> v2:172.21.15.134:3300/0 conn(0x7f3d980cf7a0 0x7f3d981626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/1812159154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980d2ed0 0x7f3d980d1a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 -- 172.21.15.22:0/1812159154 >> 172.21.15.22:0/1812159154 conn(0x7f3d9808bfc0 msgr2=0x7f3d9809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 -- 172.21.15.22:0/1812159154 shutdown_connections 2024-09-21T05:40:32.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.747+0000 7f3d9e28b640 1 -- 172.21.15.22:0/1812159154 wait complete. 2024-09-21T05:40:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.749+0000 7f3d9e28b640 1 Processor -- start 2024-09-21T05:40:32.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.749+0000 7f3d9e28b640 1 -- start start 2024-09-21T05:40:32.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.750+0000 7f3d9e28b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 0x7f3d981f9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.750+0000 7f3d9e28b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980cf7a0 0x7f3d981567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.750+0000 7f3d9e28b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d980d2ed0 0x7f3d9815e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.750+0000 7f3d9e28b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98162da0 con 0x7f3d980b9b70 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.750+0000 7f3d9e28b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98162c20 con 0x7f3d980d2ed0 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.750+0000 7f3d9e28b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3d98162f20 con 0x7f3d980cf7a0 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 0x7f3d981f9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 0x7f3d981f9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52508/0 (socket says 172.21.15.22:52508) 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 -- 172.21.15.22:0/348363194 learned_addr learned my addr 172.21.15.22:0/348363194 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d977fe640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980cf7a0 0x7f3d981567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d9c828640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d980d2ed0 0x7f3d9815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 -- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980cf7a0 msgr2=0x7f3d981567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980cf7a0 0x7f3d981567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 -- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d980d2ed0 msgr2=0x7f3d9815e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d980d2ed0 0x7f3d9815e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 -- 172.21.15.22:0/348363194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d980b1c80 con 0x7f3d980b9b70 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d97fff640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 0x7f3d981f9dc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009940 tx=0x7f3d880049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031950 con 0x7f3d980b9b70 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d88031be0 con 0x7f3d980b9b70 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d980b1f60 con 0x7f3d980b9b70 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88005040 con 0x7f3d980b9b70 2024-09-21T05:40:32.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.751+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d980b13d0 con 0x7f3d980b9b70 2024-09-21T05:40:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.752+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d981634b0 con 0x7f3d980b9b70 2024-09-21T05:40:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.753+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d88005200 con 0x7f3d980b9b70 2024-09-21T05:40:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.754+0000 7f3d94ff9640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d84078bc0 0x7f3d8407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.754+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f3d880bbac0 con 0x7f3d980b9b70 2024-09-21T05:40:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.754+0000 7f3d977fe640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d84078bc0 0x7f3d8407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:32.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.755+0000 7f3d977fe640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d84078bc0 0x7f3d8407b0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3d9815e320 tx=0x7f3d8c007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:32.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.758+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d8804a030 con 0x7f3d980b9b70 2024-09-21T05:40:32.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.948+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3d980b4020 con 0x7f3d980b9b70 2024-09-21T05:40:32.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.952+0000 7f3d94ff9640 1 -- 172.21.15.22:0/348363194 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f3d8803f030 con 0x7f3d980b9b70 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d84078bc0 msgr2=0x7f3d8407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d84078bc0 0x7f3d8407b0d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3d9815e320 tx=0x7f3d8c007e90 comp rx=0 tx=0).stop 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 msgr2=0x7f3d981f9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 0x7f3d981f9dc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009940 tx=0x7f3d880049d0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 shutdown_connections 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d980d2ed0 0x7f3d9815e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d84078bc0 0x7f3d8407b0d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d980cf7a0 0x7f3d981567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 --2- 172.21.15.22:0/348363194 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d980b9b70 0x7f3d981f9dc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.954+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 >> 172.21.15.22:0/348363194 conn(0x7f3d9808bfc0 msgr2=0x7f3d980ce4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:32.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.955+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 shutdown_connections 2024-09-21T05:40:32.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:32.955+0000 7f3d9e28b640 1 -- 172.21.15.22:0/348363194 wait complete. 2024-09-21T05:40:32.968 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:40:32.968 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:40:33.002 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:40:33.048 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:40:33.048 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:33.048 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:33.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:40:33.064 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:40:33.064 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:33.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:40:33.082 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:33.082 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:33.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:40:33.120 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:40:33.120 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:33.140 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:40:33.140 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:40:33.141 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:33.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 -- 172.21.15.22:0/2136200260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 msgr2=0x7f70e00ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 --2- 172.21.15.22:0/2136200260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e00ceb50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009830 tx=0x7f70d402a510 comp rx=0 tx=0).stop 2024-09-21T05:40:33.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 -- 172.21.15.22:0/2136200260 shutdown_connections 2024-09-21T05:40:33.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 --2- 172.21.15.22:0/2136200260 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e00ceb50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 --2- 172.21.15.22:0/2136200260 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70e00d1d40 0x7f70e00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 -- 172.21.15.22:0/2136200260 >> 172.21.15.22:0/2136200260 conn(0x7f70e00b3f10 msgr2=0x7f70e00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:33.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 -- 172.21.15.22:0/2136200260 shutdown_connections 2024-09-21T05:40:33.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.276+0000 7f70e534a640 1 -- 172.21.15.22:0/2136200260 wait complete. 2024-09-21T05:40:33.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.278+0000 7f70e534a640 1 Processor -- start 2024-09-21T05:40:33.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.278+0000 7f70e534a640 1 -- start start 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70e534a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70e00ccba0 0x7f70e014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70e534a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 0x7f70e014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70e534a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e0156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70e534a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f70e00d12d0 con 0x7f70e00d1d40 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70e534a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f70e00d1150 con 0x7f70e015c6f0 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70e534a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f70e00d1450 con 0x7f70e00ccba0 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70de7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 0x7f70e014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70de7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 0x7f70e014a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52512/0 (socket says 172.21.15.22:52512) 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70de7fc640 1 -- 172.21.15.22:0/3046753616 learned_addr learned my addr 172.21.15.22:0/3046753616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:33.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70deffd640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70e00ccba0 0x7f70e014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.279+0000 7f70df7fe640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e0156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70de7fc640 1 -- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70e00ccba0 msgr2=0x7f70e014b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70de7fc640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70e00ccba0 0x7f70e014b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70de7fc640 1 -- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 msgr2=0x7f70e0156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70de7fc640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e0156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70de7fc640 1 -- 172.21.15.22:0/3046753616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e01578a0 con 0x7f70e00d1d40 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70df7fe640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e0156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70de7fc640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 0x7f70e014a6d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000c00 tx=0x7f70d4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4031d60 con 0x7f70e00d1d40 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70d4033e10 con 0x7f70e00d1d40 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e0157b80 con 0x7f70e00d1d40 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d4033310 con 0x7f70e00d1d40 2024-09-21T05:40:33.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.280+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e0158140 con 0x7f70e00d1d40 2024-09-21T05:40:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.281+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e00d1970 con 0x7f70e00d1d40 2024-09-21T05:40:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.282+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f70d40334b0 con 0x7f70e00d1d40 2024-09-21T05:40:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.283+0000 7f70c77fe640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70cc078bc0 0x7f70cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.283+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f70d40bbdb0 con 0x7f70e00d1d40 2024-09-21T05:40:33.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.283+0000 7f70deffd640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70cc078bc0 0x7f70cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.283+0000 7f70deffd640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70cc078bc0 0x7f70cc07b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009b30 tx=0x7f70d0005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:33.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.287+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f70d403f030 con 0x7f70e00d1d40 2024-09-21T05:40:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.512+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70e0050ff0 con 0x7f70e00d1d40 2024-09-21T05:40:33.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.512+0000 7f70c77fe640 1 -- 172.21.15.22:0/3046753616 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3911 (secure 0 0 0) 0x7f70d40844f0 con 0x7f70e00d1d40 2024-09-21T05:40:33.515 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:33.515 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":19,"btime":"2024-09-21T05:40:27:251711+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/143481885","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":143481885},{"type":"v1","addr":"172.21.15.134:6837","nonce":143481885}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4801,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3980542773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3980542773},{"type":"v1","addr":"172.21.15.134:6836","nonce":3980542773}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2207324628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2207324628},{"type":"v1","addr":"172.21.15.22:6835","nonce":2207324628}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5214,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/572465870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":572465870},{"type":"v1","addr":"172.21.15.22:6834","nonce":572465870}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-21T05:40:33.515 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 19 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70cc078bc0 msgr2=0x7f70cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70cc078bc0 0x7f70cc07b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009b30 tx=0x7f70d0005f90 comp rx=0 tx=0).stop 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 msgr2=0x7f70e014a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 0x7f70e014a6d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f70d4000c00 tx=0x7f70d4033c20 comp rx=0 tx=0).stop 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 shutdown_connections 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70e015c6f0 0x7f70e0156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70e00d1d40 0x7f70e014a6d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70cc078bc0 0x7f70cc07b0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 --2- 172.21.15.22:0/3046753616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f70e00ccba0 0x7f70e014b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 >> 172.21.15.22:0/3046753616 conn(0x7f70e00b3f10 msgr2=0x7f70e00b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:33.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 shutdown_connections 2024-09-21T05:40:33.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.515+0000 7f70e534a640 1 -- 172.21.15.22:0/3046753616 wait complete. 2024-09-21T05:40:33.528 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:40:33.528 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:40:33.529 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:40:33.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:40:33.529 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:40:33.529 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:40:33.529 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:40:33.529 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:40:33.529 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:40:33.529 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:40:33.530 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:40:33.530 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:40:33.530 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:40:33.530 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:40:33.530 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:40:33.530 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:40:33.539 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:40:33.539 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:40:33.541 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:40:33.541 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:40:33.542 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:40:33.542 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:40:33.543 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:40:33.543 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:40:33.583 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:40:33.583 DEBUG:teuthology.parallel:result is None 2024-09-21T05:40:33.584 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:40:33.584 DEBUG:teuthology.parallel:result is None 2024-09-21T05:40:33.585 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:40:33.585 DEBUG:teuthology.parallel:result is None 2024-09-21T05:40:33.586 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:40:33.586 DEBUG:teuthology.parallel:result is None 2024-09-21T05:40:33.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:40:33.672 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:40:33.673 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:40:33.677 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:40:33.678 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:40:33.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.706+0000 7f5378ad3640 1 -- 172.21.15.22:0/3496079712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740ccbd0 msgr2=0x7f53740d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.706+0000 7f5378ad3640 1 --2- 172.21.15.22:0/3496079712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740ccbd0 0x7f53740d0d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5374073f50 tx=0x7f536802a5f0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.707+0000 7f5378ad3640 1 -- 172.21.15.22:0/3496079712 shutdown_connections 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.707+0000 7f5378ad3640 1 --2- 172.21.15.22:0/3496079712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740ccbd0 0x7f53740d0d80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.707+0000 7f5378ad3640 1 --2- 172.21.15.22:0/3496079712 >> v2:172.21.15.134:3300/0 conn(0x7f53740b51c0 0x7f537415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.707+0000 7f5378ad3640 1 -- 172.21.15.22:0/3496079712 >> 172.21.15.22:0/3496079712 conn(0x7f5374072ae0 msgr2=0x7f5374072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.707+0000 7f5378ad3640 1 -- 172.21.15.22:0/3496079712 shutdown_connections 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.707+0000 7f5378ad3640 1 -- 172.21.15.22:0/3496079712 wait complete. 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.709+0000 7f5378ad3640 1 Processor -- start 2024-09-21T05:40:33.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.709+0000 7f5378ad3640 1 -- start start 2024-09-21T05:40:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5378ad3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 0x7f53740b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5378ad3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53740ccbd0 0x7f537415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5378ad3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53740d12c0 0x7f53741f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5378ad3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53740d19f0 con 0x7f53740b51c0 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5378ad3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5374160a60 con 0x7f53740ccbd0 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5378ad3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53741f7360 con 0x7f53740d12c0 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f537259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 0x7f53740b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f537259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 0x7f53740b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:52542/0 (socket says 172.21.15.22:52542) 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f537259c640 1 -- 172.21.15.22:0/1442351037 learned_addr learned my addr 172.21.15.22:0/1442351037 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5372d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53740d12c0 0x7f53741f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.710+0000 7f5371d9b640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53740ccbd0 0x7f537415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f537259c640 1 -- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53740d12c0 msgr2=0x7f53741f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f537259c640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53740d12c0 0x7f53741f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f537259c640 1 -- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53740ccbd0 msgr2=0x7f537415e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f537259c640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53740ccbd0 0x7f537415e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f537259c640 1 -- 172.21.15.22:0/1442351037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537415d400 con 0x7f53740b51c0 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f537259c640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 0x7f53740b0e50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f536400e790 tx=0x7f536400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536400eea0 con 0x7f53740b51c0 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f536400cdb0 con 0x7f53740b51c0 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364010690 con 0x7f53740b51c0 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537415d6e0 con 0x7f53740b51c0 2024-09-21T05:40:33.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.711+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537415dca0 con 0x7f53740b51c0 2024-09-21T05:40:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.712+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5374160f10 con 0x7f53740b51c0 2024-09-21T05:40:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.713+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5364010830 con 0x7f53740b51c0 2024-09-21T05:40:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.714+0000 7f5362ffd640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5354078b90 0x7f535407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.714+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f536409b480 con 0x7f53740b51c0 2024-09-21T05:40:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.714+0000 7f5371d9b640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5354078b90 0x7f535407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:33.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.714+0000 7f5371d9b640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5354078b90 0x7f535407b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5368033000 tx=0x7f5368004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.718+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53640634d0 con 0x7f53740b51c0 2024-09-21T05:40:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:33.915+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f53740b3c60 con 0x7f53740b51c0 2024-09-21T05:40:34.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.283+0000 7f5362ffd640 1 -- 172.21.15.22:0/1442351037 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v37) ==== 97+0+0 (secure 0 0 0) 0x7f5364067f00 con 0x7f53740b51c0 2024-09-21T05:40:34.286 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:40:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5354078b90 msgr2=0x7f535407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5354078b90 0x7f535407b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5368033000 tx=0x7f5368004730 comp rx=0 tx=0).stop 2024-09-21T05:40:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 msgr2=0x7f53740b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 0x7f53740b0e50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f536400e790 tx=0x7f536400ec50 comp rx=0 tx=0).stop 2024-09-21T05:40:34.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 shutdown_connections 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5354078b90 0x7f535407b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53740d12c0 0x7f53741f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53740ccbd0 0x7f537415e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 --2- 172.21.15.22:0/1442351037 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53740b51c0 0x7f53740b0e50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 >> 172.21.15.22:0/1442351037 conn(0x7f5374072ae0 msgr2=0x7f53740b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 shutdown_connections 2024-09-21T05:40:34.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.286+0000 7f5378ad3640 1 -- 172.21.15.22:0/1442351037 wait complete. 2024-09-21T05:40:34.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:40:34.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f66944f9640 1 -- 172.21.15.22:0/3350342342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6678004d40 msgr2=0x7f6678005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f66944f9640 1 --2- 172.21.15.22:0/3350342342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6678004d40 0x7f6678005210 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b5ec0 tx=0x7f6678005830 comp rx=0 tx=0).stop 2024-09-21T05:40:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f6692a96640 1 -- 172.21.15.22:0/3350342342 reap_dead start 2024-09-21T05:40:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f66944f9640 1 -- 172.21.15.22:0/3350342342 shutdown_connections 2024-09-21T05:40:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f66944f9640 1 -- 172.21.15.22:0/3350342342 >> 172.21.15.22:0/3350342342 conn(0x7f668c08bfc0 msgr2=0x7f668c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f66944f9640 1 -- 172.21.15.22:0/3350342342 shutdown_connections 2024-09-21T05:40:34.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.417+0000 7f66944f9640 1 -- 172.21.15.22:0/3350342342 wait complete. 2024-09-21T05:40:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.419+0000 7f66944f9640 1 Processor -- start 2024-09-21T05:40:34.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.419+0000 7f66944f9640 1 -- start start 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f66944f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6678004d40 0x7f668c0cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f66944f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 0x7f668c0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f66944f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f668c1e19b0 0x7f668c1e1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f66944f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0b9020 con 0x7f668c0cd4a0 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f66944f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f668c0b4230 con 0x7f6678004d40 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f66944f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f668c0b91a0 con 0x7f668c1e19b0 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f6691a94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 0x7f668c0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f6691a94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 0x7f668c0cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59454/0 (socket says 172.21.15.22:59454) 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.420+0000 7f6691a94640 1 -- 172.21.15.22:0/1553292835 learned_addr learned my addr 172.21.15.22:0/1553292835 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:34.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6692a96640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f668c1e19b0 0x7f668c1e1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6692295640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6678004d40 0x7f668c0cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6691a94640 1 -- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f668c1e19b0 msgr2=0x7f668c1e1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6691a94640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f668c1e19b0 0x7f668c1e1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6691a94640 1 -- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6678004d40 msgr2=0x7f668c0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6691a94640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6678004d40 0x7f668c0cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6691a94640 1 -- 172.21.15.22:0/1553292835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f668c0cc700 con 0x7f668c0cd4a0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6691a94640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 0x7f668c0cd8b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f667c011b70 tx=0x7f667c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c01e070 con 0x7f668c0cd4a0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f667c00e7c0 con 0x7f668c0cd4a0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c00ebf0 con 0x7f668c0cd4a0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c0b34c0 con 0x7f668c0cd4a0 2024-09-21T05:40:34.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.421+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c0b1670 con 0x7f668c0cd4a0 2024-09-21T05:40:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.422+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f668c0b9730 con 0x7f668c0cd4a0 2024-09-21T05:40:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.423+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f667c021070 con 0x7f668c0cd4a0 2024-09-21T05:40:34.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.424+0000 7f6682ffd640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.424+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f667c09b970 con 0x7f668c0cd4a0 2024-09-21T05:40:34.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.424+0000 7f6692295640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.425+0000 7f6692295640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6688000c00 tx=0x7f6688006bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:34.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.429+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f667c063ab0 con 0x7f668c0cd4a0 2024-09-21T05:40:34.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.689+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f668c0d10e0 con 0x7f668c0cd4a0 2024-09-21T05:40:34.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.689+0000 7f6682ffd640 1 -- 172.21.15.22:0/1553292835 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f667c0684e0 con 0x7f668c0cd4a0 2024-09-21T05:40:34.691 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:40:34.693 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:40:34.694 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.696 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:40:34.697 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow r", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:40:34.703 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:40:34.704 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:40:34.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 msgr2=0x7f667007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6688000c00 tx=0x7f6688006bc0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 msgr2=0x7f668c0cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 0x7f668c0cd8b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f667c011b70 tx=0x7f667c00f790 comp rx=0 tx=0).stop 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 shutdown_connections 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6670078b90 0x7f667007b0a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f668c1e19b0 0x7f668c1e1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f668c0cd4a0 0x7f668c0cd8b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 --2- 172.21.15.22:0/1553292835 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6678004d40 0x7f668c0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 >> 172.21.15.22:0/1553292835 conn(0x7f668c08bfc0 msgr2=0x7f668c1e4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 shutdown_connections 2024-09-21T05:40:34.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.691+0000 7f66944f9640 1 -- 172.21.15.22:0/1553292835 wait complete. 2024-09-21T05:40:34.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:40:34.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5caca41640 1 -- 172.21.15.22:0/3150353144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 msgr2=0x7f5c90004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5caca41640 1 --2- 172.21.15.22:0/3150353144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5c90004c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c260 tx=0x7f5c90005830 comp rx=0 tx=0).stop 2024-09-21T05:40:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5ca6d9d640 1 -- 172.21.15.22:0/3150353144 reap_dead start 2024-09-21T05:40:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5caca41640 1 -- 172.21.15.22:0/3150353144 shutdown_connections 2024-09-21T05:40:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5caca41640 1 -- 172.21.15.22:0/3150353144 >> 172.21.15.22:0/3150353144 conn(0x7f5ca809d7f0 msgr2=0x7f5ca80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5caca41640 1 -- 172.21.15.22:0/3150353144 shutdown_connections 2024-09-21T05:40:34.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.822+0000 7f5caca41640 1 -- 172.21.15.22:0/3150353144 wait complete. 2024-09-21T05:40:34.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.824+0000 7f5caca41640 1 Processor -- start 2024-09-21T05:40:34.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.824+0000 7f5caca41640 1 -- start start 2024-09-21T05:40:34.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5caca41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5ca80cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5caca41640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ca81f9970 0x7f5ca81f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5caca41640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ca81f5c20 0x7f5ca80b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5caca41640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca80b3710 con 0x7f5c90004800 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5caca41640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca80d5010 con 0x7f5ca81f9970 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5caca41640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca80b3890 con 0x7f5ca81f5c20 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5ca659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5ca80cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5ca659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5ca80cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59480/0 (socket says 172.21.15.22:59480) 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.825+0000 7f5ca659c640 1 -- 172.21.15.22:0/2434043747 learned_addr learned my addr 172.21.15.22:0/2434043747 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca6d9d640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ca81f5c20 0x7f5ca80b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca5d9b640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ca81f9970 0x7f5ca81f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca659c640 1 -- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ca81f5c20 msgr2=0x7f5ca80b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca659c640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ca81f5c20 0x7f5ca80b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca659c640 1 -- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ca81f9970 msgr2=0x7f5ca81f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:34.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca659c640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ca81f9970 0x7f5ca81f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca659c640 1 -- 172.21.15.22:0/2434043747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca81f8eb0 con 0x7f5c90004800 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5ca659c640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5ca80cdde0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c98011c40 tx=0x7f5c9800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c98003d80 con 0x7f5c90004800 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c98004050 con 0x7f5c90004800 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c980044a0 con 0x7f5c90004800 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca81f6450 con 0x7f5c90004800 2024-09-21T05:40:34.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.826+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca80a1830 con 0x7f5c90004800 2024-09-21T05:40:34.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.827+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca80a1a90 con 0x7f5c90004800 2024-09-21T05:40:34.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.828+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c98004640 con 0x7f5c90004800 2024-09-21T05:40:34.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.829+0000 7f5c96ffd640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5c88078b90 0x7f5c8807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:34.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.829+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7f5c9809bdd0 con 0x7f5c90004800 2024-09-21T05:40:34.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.829+0000 7f5ca5d9b640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5c88078b90 0x7f5c8807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:34.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.829+0000 7f5ca5d9b640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5c88078b90 0x7f5c8807b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c260 tx=0x7f5c9c038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:34.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:34.833+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c98063fa0 con 0x7f5c90004800 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.023+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5ca80d2a00 con 0x7f5c90004800 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.024+0000 7f5c96ffd640 1 -- 172.21.15.22:0/2434043747 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) ==== 84+0+65 (secure 0 0 0) 0x7f5c980689d0 con 0x7f5c90004800 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:40:35.026 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:40:35.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:35.027 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:40:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5c88078b90 msgr2=0x7f5c8807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5c88078b90 0x7f5c8807b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c260 tx=0x7f5c9c038040 comp rx=0 tx=0).stop 2024-09-21T05:40:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 msgr2=0x7f5ca80cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5ca80cdde0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c98011c40 tx=0x7f5c9800f790 comp rx=0 tx=0).stop 2024-09-21T05:40:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 shutdown_connections 2024-09-21T05:40:35.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5c88078b90 0x7f5c8807b0a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:35.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5ca81f5c20 0x7f5ca80b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:35.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ca81f9970 0x7f5ca81f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:35.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 --2- 172.21.15.22:0/2434043747 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5c90004800 0x7f5ca80cdde0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:35.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 >> 172.21.15.22:0/2434043747 conn(0x7f5ca809d7f0 msgr2=0x7f5ca815e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:35.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 shutdown_connections 2024-09-21T05:40:35.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.026+0000 7f5caca41640 1 -- 172.21.15.22:0/2434043747 wait complete. 2024-09-21T05:40:35.039 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:40:35.040 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:40:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.158+0000 7fc9f7782640 1 -- 172.21.15.22:0/2405932331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9d8004d40 msgr2=0x7fc9d8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.159+0000 7fc9f7782640 1 --2- 172.21.15.22:0/2405932331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9d8004d40 0x7fc9d8005150 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00d1ca0 tx=0x7fc9d8005830 comp rx=0 tx=0).stop 2024-09-21T05:40:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.159+0000 7fc9f5d1f640 1 -- 172.21.15.22:0/2405932331 reap_dead start 2024-09-21T05:40:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.159+0000 7fc9f7782640 1 -- 172.21.15.22:0/2405932331 shutdown_connections 2024-09-21T05:40:35.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.159+0000 7fc9f7782640 1 -- 172.21.15.22:0/2405932331 >> 172.21.15.22:0/2405932331 conn(0x7fc9f009d800 msgr2=0x7fc9f008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.159+0000 7fc9f7782640 1 -- 172.21.15.22:0/2405932331 shutdown_connections 2024-09-21T05:40:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.159+0000 7fc9f7782640 1 -- 172.21.15.22:0/2405932331 wait complete. 2024-09-21T05:40:35.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.161+0000 7fc9f7782640 1 Processor -- start 2024-09-21T05:40:35.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.161+0000 7fc9f7782640 1 -- start start 2024-09-21T05:40:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f7782640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d8004d40 0x7fc9f01e4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f7782640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 0x7fc9f00b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:35.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f7782640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9f01d48c0 0x7fc9f01f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f7782640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00d15c0 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f7782640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f00d1440 con 0x7fc9d8004d40 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f7782640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f00d35c0 con 0x7fc9f01d48c0 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 0x7fc9f00b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 0x7fc9f00b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59494/0 (socket says 172.21.15.22:59494) 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 -- 172.21.15.22:0/1360413042 learned_addr learned my addr 172.21.15.22:0/1360413042 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f551e640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d8004d40 0x7fc9f01e4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f5d1f640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9f01d48c0 0x7fc9f01f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 -- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9f01d48c0 msgr2=0x7fc9f01f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9f01d48c0 0x7fc9f01f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:35.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 -- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d8004d40 msgr2=0x7fc9f01e4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d8004d40 0x7fc9f01e4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.162+0000 7fc9f4d1d640 1 -- 172.21.15.22:0/1360413042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f01f9b30 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.163+0000 7fc9f4d1d640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 0x7fc9f00b6940 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4011890 tx=0x7fc9e4011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.163+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e401e070 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.163+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9e400fdb0 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.163+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f01f8fe0 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.163+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4019690 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.163+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01f9570 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.164+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f00d3080 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.165+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9e4021070 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.165+0000 7fc9edffb640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc9dc078bc0 0x7fc9dc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.166+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4857+0+0 (secure 0 0 0) 0x7fc9e409b320 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.166+0000 7fc9f551e640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc9dc078bc0 0x7fc9dc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:35.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.166+0000 7fc9f551e640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc9dc078bc0 0x7fc9dc07b0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00016b0 tx=0x7fc9e0005040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:35.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.170+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc9e4063460 con 0x7fc9f01f8bd0 2024-09-21T05:40:35.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:35.360+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc9f01f55e0 con 0x7fc9f01f8bd0 2024-09-21T05:40:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.296+0000 7fc9edffb640 1 -- 172.21.15.22:0/1360413042 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) ==== 138+0+0 (secure 0 0 0) 0x7fc9e4067e90 con 0x7fc9f01f8bd0 2024-09-21T05:40:36.298 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:40:36.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc9dc078bc0 msgr2=0x7fc9dc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:36.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc9dc078bc0 0x7fc9dc07b0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00016b0 tx=0x7fc9e0005040 comp rx=0 tx=0).stop 2024-09-21T05:40:36.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 msgr2=0x7fc9f00b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 0x7fc9f00b6940 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4011890 tx=0x7fc9e4011c90 comp rx=0 tx=0).stop 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 shutdown_connections 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc9dc078bc0 0x7fc9dc07b0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc9f01d48c0 0x7fc9f01f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc9f01f8bd0 0x7fc9f00b6940 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 --2- 172.21.15.22:0/1360413042 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc9d8004d40 0x7fc9f01e4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 >> 172.21.15.22:0/1360413042 conn(0x7fc9f009d800 msgr2=0x7fc9f006f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 shutdown_connections 2024-09-21T05:40:36.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.299+0000 7fc9f7782640 1 -- 172.21.15.22:0/1360413042 wait complete. 2024-09-21T05:40:36.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 -- 172.21.15.22:0/1605241406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 msgr2=0x7fd0000b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 --2- 172.21.15.22:0/1605241406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0000b5680 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009830 tx=0x7fcff002a510 comp rx=0 tx=0).stop 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 -- 172.21.15.22:0/1605241406 shutdown_connections 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 --2- 172.21.15.22:0/1605241406 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd000142270 0x7fd000073e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 --2- 172.21.15.22:0/1605241406 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0000b5680 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 -- 172.21.15.22:0/1605241406 >> 172.21.15.22:0/1605241406 conn(0x7fd0000d15f0 msgr2=0x7fd000073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 -- 172.21.15.22:0/1605241406 shutdown_connections 2024-09-21T05:40:36.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.448+0000 7fd00528b640 1 -- 172.21.15.22:0/1605241406 wait complete. 2024-09-21T05:40:36.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.450+0000 7fd00528b640 1 Processor -- start 2024-09-21T05:40:36.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.450+0000 7fd00528b640 1 -- start start 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fd00528b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000142270 0x7fd00013ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fd00528b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd000142680 0x7fd00014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fd00528b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0001f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fd00528b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000d06f0 con 0x7fd000144270 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fd00528b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0000d0570 con 0x7fd000142270 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fd00528b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd0000d0870 con 0x7fd000142680 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0001f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0001f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59510/0 (socket says 172.21.15.22:59510) 2024-09-21T05:40:36.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 -- 172.21.15.22:0/3855817628 learned_addr learned my addr 172.21.15.22:0/3855817628 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcff67fc640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd000142680 0x7fd00014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcffeffd640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000142270 0x7fd00013ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 -- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd000142680 msgr2=0x7fd00014d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd000142680 0x7fd00014d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 -- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000142270 msgr2=0x7fd00013ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000142270 0x7fd00013ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 -- 172.21.15.22:0/3855817628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00013feb0 con 0x7fd000144270 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.452+0000 7fcfff7fe640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0001f6db0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcff800ca80 tx=0x7fcff800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.453+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8016020 con 0x7fd000144270 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.453+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcff800acc0 con 0x7fd000144270 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.453+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8012680 con 0x7fd000144270 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.453+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd000140190 con 0x7fd000144270 2024-09-21T05:40:36.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.453+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd000140750 con 0x7fd000144270 2024-09-21T05:40:36.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.454+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd000074550 con 0x7fd000144270 2024-09-21T05:40:36.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.455+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcff8019020 con 0x7fd000144270 2024-09-21T05:40:36.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.455+0000 7fcff7fff640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcfe0078b90 0x7fcfe007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:36.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.455+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5227+0+0 (secure 0 0 0) 0x7fcff809b390 con 0x7fd000144270 2024-09-21T05:40:36.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.455+0000 7fcffeffd640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcfe0078b90 0x7fcfe007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:36.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.456+0000 7fcffeffd640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcfe0078b90 0x7fcfe007b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcff0034000 tx=0x7fcff0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:36.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.459+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcff8020020 con 0x7fd000144270 2024-09-21T05:40:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:36.649+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd00014ddf0 con 0x7fd000144270 2024-09-21T05:40:37.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.304+0000 7fcff7fff640 1 -- 172.21.15.22:0/3855817628 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) ==== 170+0+0 (secure 0 0 0) 0x7fcff8063440 con 0x7fd000144270 2024-09-21T05:40:37.306 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:40:37.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.308+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcfe0078b90 msgr2=0x7fcfe007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.308+0000 7fd00528b640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcfe0078b90 0x7fcfe007b0a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcff0034000 tx=0x7fcff0004730 comp rx=0 tx=0).stop 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.308+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 msgr2=0x7fd0001f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.308+0000 7fd00528b640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0001f6db0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcff800ca80 tx=0x7fcff800cf40 comp rx=0 tx=0).stop 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.309+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 shutdown_connections 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.309+0000 7fd00528b640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd000144270 0x7fd0001f6db0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.309+0000 7fd00528b640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcfe0078b90 0x7fcfe007b0a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.309+0000 7fd00528b640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd000142680 0x7fd00014d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.309+0000 7fd00528b640 1 --2- 172.21.15.22:0/3855817628 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd000142270 0x7fd00013ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.309+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 >> 172.21.15.22:0/3855817628 conn(0x7fd0000d15f0 msgr2=0x7fd000068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:37.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.310+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 shutdown_connections 2024-09-21T05:40:37.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.311+0000 7fd00528b640 1 -- 172.21.15.22:0/3855817628 wait complete. 2024-09-21T05:40:37.340 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 -- 172.21.15.22:0/1224304763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 msgr2=0x7fd2200b9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 --2- 172.21.15.22:0/1224304763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2200b9be0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd22008e380 tx=0x7fd20802a5f0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 -- 172.21.15.22:0/1224304763 shutdown_connections 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 --2- 172.21.15.22:0/1224304763 >> v2:172.21.15.134:3300/0 conn(0x7fd2200d4c20 0x7fd2200b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 --2- 172.21.15.22:0/1224304763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2200b9be0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 -- 172.21.15.22:0/1224304763 >> 172.21.15.22:0/1224304763 conn(0x7fd22008bfc0 msgr2=0x7fd22009d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 -- 172.21.15.22:0/1224304763 shutdown_connections 2024-09-21T05:40:37.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.464+0000 7fd226c25640 1 -- 172.21.15.22:0/1224304763 wait complete. 2024-09-21T05:40:37.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.466+0000 7fd226c25640 1 Processor -- start 2024-09-21T05:40:37.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.466+0000 7fd226c25640 1 -- start start 2024-09-21T05:40:37.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.467+0000 7fd226c25640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2200cf0b0 0x7fd2200ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:37.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.467+0000 7fd226c25640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2201ea950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:37.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.467+0000 7fd226c25640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2200d11c0 0x7fd2200d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:37.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.467+0000 7fd226c25640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2200b4a00 con 0x7fd2200d0ce0 2024-09-21T05:40:37.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.467+0000 7fd226c25640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2200b4880 con 0x7fd2200d11c0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.467+0000 7fd226c25640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2200b4b80 con 0x7fd2200cf0b0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2201ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2201ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59538/0 (socket says 172.21.15.22:59538) 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 -- 172.21.15.22:0/3531993918 learned_addr learned my addr 172.21.15.22:0/3531993918 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd2249c1640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2200cf0b0 0x7fd2200ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd2251c2640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2200d11c0 0x7fd2200d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 -- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2200cf0b0 msgr2=0x7fd2200ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2200cf0b0 0x7fd2200ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 -- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2200d11c0 msgr2=0x7fd2200d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2200d11c0 0x7fd2200d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 -- 172.21.15.22:0/3531993918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2200d1b10 con 0x7fd2200d0ce0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21ffff640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2201ea950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c7a0 tx=0x7fd21000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210017070 con 0x7fd2200d0ce0 2024-09-21T05:40:37.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd210005070 con 0x7fd2200d0ce0 2024-09-21T05:40:37.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2200a12e0 con 0x7fd2200d0ce0 2024-09-21T05:40:37.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100054a0 con 0x7fd2200d0ce0 2024-09-21T05:40:37.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.468+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2200a1860 con 0x7fd2200d0ce0 2024-09-21T05:40:37.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.470+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2200b5860 con 0x7fd2200d0ce0 2024-09-21T05:40:37.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.471+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd21001a070 con 0x7fd2200d0ce0 2024-09-21T05:40:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.472+0000 7fd21d7fa640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd200078b90 0x7fd20007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.472+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5609+0+0 (secure 0 0 0) 0x7fd21009a850 con 0x7fd2200d0ce0 2024-09-21T05:40:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.472+0000 7fd2249c1640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd200078b90 0x7fd20007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:37.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.473+0000 7fd2249c1640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd200078b90 0x7fd20007b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd208033000 tx=0x7fd208004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:37.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.476+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd210021020 con 0x7fd2200d0ce0 2024-09-21T05:40:37.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:37.701+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd2200b6320 con 0x7fd2200d0ce0 2024-09-21T05:40:38.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.316+0000 7fd21d7fa640 1 -- 172.21.15.22:0/3531993918 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7fd2200d1b10 con 0x7fd2200d0ce0 2024-09-21T05:40:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 6 and data pool 7 2024-09-21T05:40:38.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.323+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd200078b90 msgr2=0x7fd20007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.323+0000 7fd226c25640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd200078b90 0x7fd20007b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd208033000 tx=0x7fd208004730 comp rx=0 tx=0).stop 2024-09-21T05:40:38.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.323+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 msgr2=0x7fd2201ea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.323+0000 7fd226c25640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2201ea950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd21000c7a0 tx=0x7fd21000cc60 comp rx=0 tx=0).stop 2024-09-21T05:40:38.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.326+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 shutdown_connections 2024-09-21T05:40:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.326+0000 7fd226c25640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2200d11c0 0x7fd2200d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.326+0000 7fd226c25640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2200d0ce0 0x7fd2201ea950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.326+0000 7fd226c25640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd200078b90 0x7fd20007b0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.326+0000 7fd226c25640 1 --2- 172.21.15.22:0/3531993918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2200cf0b0 0x7fd2200ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.326+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 >> 172.21.15.22:0/3531993918 conn(0x7fd22008bfc0 msgr2=0x7fd22006f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:38.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.328+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 shutdown_connections 2024-09-21T05:40:38.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.329+0000 7fd226c25640 1 -- 172.21.15.22:0/3531993918 wait complete. 2024-09-21T05:40:38.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:38.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 -- 172.21.15.22:0/2819286725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad48003a50 msgr2=0x7fad48003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 --2- 172.21.15.22:0/2819286725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad48003a50 0x7fad48003ec0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad44006fc0 tx=0x7fad48005830 comp rx=0 tx=0).stop 2024-09-21T05:40:38.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 -- 172.21.15.22:0/2819286725 shutdown_connections 2024-09-21T05:40:38.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 --2- 172.21.15.22:0/2819286725 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad48003a50 0x7fad48003ec0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 -- 172.21.15.22:0/2819286725 >> 172.21.15.22:0/2819286725 conn(0x7fad5c08bfc0 msgr2=0x7fad5c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:38.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 -- 172.21.15.22:0/2819286725 shutdown_connections 2024-09-21T05:40:38.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.472+0000 7fad62fad640 1 -- 172.21.15.22:0/2819286725 wait complete. 2024-09-21T05:40:38.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.474+0000 7fad62fad640 1 Processor -- start 2024-09-21T05:40:38.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.474+0000 7fad62fad640 1 -- start start 2024-09-21T05:40:38.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad62fad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad48003a50 0x7fad5c09ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad62fad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 0x7fad5c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad62fad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ca1f0 0x7fad5c0a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad62fad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0cfd70 con 0x7fad5c1f65e0 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad62fad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c0cfbf0 con 0x7fad5c0ca1f0 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad62fad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c0cfef0 con 0x7fad48003a50 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad5bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 0x7fad5c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad5bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 0x7fad5c0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59556/0 (socket says 172.21.15.22:59556) 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad5bfff640 1 -- 172.21.15.22:0/757525565 learned_addr learned my addr 172.21.15.22:0/757525565 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad60d49640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad48003a50 0x7fad5c09ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.475+0000 7fad6154a640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ca1f0 0x7fad5c0a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad5bfff640 1 -- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad48003a50 msgr2=0x7fad5c09ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad5bfff640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad48003a50 0x7fad5c09ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad5bfff640 1 -- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ca1f0 msgr2=0x7fad5c0a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad5bfff640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ca1f0 0x7fad5c0a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad5bfff640 1 -- 172.21.15.22:0/757525565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c09efc0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad5bfff640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 0x7fad5c0c9cb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00faf0 tx=0x7fad4c00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c011440 con 0x7fad5c1f65e0 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad4c0115e0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4c011a30 con 0x7fad5c1f65e0 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c09f6e0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.476+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c09f400 con 0x7fad5c1f65e0 2024-09-21T05:40:38.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.477+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c0b9da0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.478+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fad4c011bd0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.478+0000 7fad597fa640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad3c078bc0 0x7fad3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.479+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fad4c09bac0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.479+0000 7fad60d49640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad3c078bc0 0x7fad3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.480+0000 7fad60d49640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad3c078bc0 0x7fad3c07b0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad44033000 tx=0x7fad44001b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:38.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.483+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad5c09f6e0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.672+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad5c0b4020 con 0x7fad5c1f65e0 2024-09-21T05:40:38.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.673+0000 7fad597fa640 1 -- 172.21.15.22:0/757525565 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16674 (secure 0 0 0) 0x7fad4c0638b0 con 0x7fad5c1f65e0 2024-09-21T05:40:38.676 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:38.676 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:37.702282+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:35.364998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:36.653978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.675+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad3c078bc0 msgr2=0x7fad3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.675+0000 7fad62fad640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad3c078bc0 0x7fad3c07b0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad44033000 tx=0x7fad44001b20 comp rx=0 tx=0).stop 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.675+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 msgr2=0x7fad5c0c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.675+0000 7fad62fad640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 0x7fad5c0c9cb0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad4c00faf0 tx=0x7fad4c00d2e0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 shutdown_connections 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad5c0ca1f0 0x7fad5c0a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad5c1f65e0 0x7fad5c0c9cb0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fad3c078bc0 0x7fad3c07b0d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 --2- 172.21.15.22:0/757525565 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad48003a50 0x7fad5c09ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 >> 172.21.15.22:0/757525565 conn(0x7fad5c08bfc0 msgr2=0x7fad5c0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:38.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 shutdown_connections 2024-09-21T05:40:38.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.676+0000 7fad62fad640 1 -- 172.21.15.22:0/757525565 wait complete. 2024-09-21T05:40:38.689 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 -- 172.21.15.22:0/2982774283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640b51c0 msgr2=0x7fd76415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 --2- 172.21.15.22:0/2982774283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640b51c0 0x7fd76415c6e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd760009830 tx=0x7fd76002a510 comp rx=0 tx=0).stop 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 -- 172.21.15.22:0/2982774283 shutdown_connections 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 --2- 172.21.15.22:0/2982774283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7640ccbd0 0x7fd7640d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 --2- 172.21.15.22:0/2982774283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640b51c0 0x7fd76415c6e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 -- 172.21.15.22:0/2982774283 >> 172.21.15.22:0/2982774283 conn(0x7fd764072ae0 msgr2=0x7fd764072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 -- 172.21.15.22:0/2982774283 shutdown_connections 2024-09-21T05:40:38.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.803+0000 7fd76bd62640 1 -- 172.21.15.22:0/2982774283 wait complete. 2024-09-21T05:40:38.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.805+0000 7fd76bd62640 1 Processor -- start 2024-09-21T05:40:38.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.805+0000 7fd76bd62640 1 -- start start 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.806+0000 7fd76bd62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7640b51c0 0x7fd764159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.806+0000 7fd76bd62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7640ccbd0 0x7fd7640a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.806+0000 7fd76bd62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 0x7fd76414c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.806+0000 7fd76bd62640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd764160a60 con 0x7fd7640d12c0 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.806+0000 7fd76bd62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd7640d1a20 con 0x7fd7640ccbd0 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.806+0000 7fd76bd62640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd764160be0 con 0x7fd7640b51c0 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 0x7fd76414c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 0x7fd76414c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59578/0 (socket says 172.21.15.22:59578) 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 -- 172.21.15.22:0/207085386 learned_addr learned my addr 172.21.15.22:0/207085386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd769afe640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7640b51c0 0x7fd764159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd7692fd640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7640ccbd0 0x7fd7640a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 -- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7640b51c0 msgr2=0x7fd764159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7640b51c0 0x7fd764159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 -- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7640ccbd0 msgr2=0x7fd7640a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7640ccbd0 0x7fd7640a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 -- 172.21.15.22:0/207085386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7640a99e0 con 0x7fd7640d12c0 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76a2ff640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 0x7fd76414c740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00cb10 tx=0x7fd75c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c016020 con 0x7fd7640d12c0 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd75c00ad60 con 0x7fd7640d12c0 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76414cc80 con 0x7fd7640d12c0 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75c012680 con 0x7fd7640d12c0 2024-09-21T05:40:38.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.807+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76414d470 con 0x7fd7640d12c0 2024-09-21T05:40:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.808+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7640a9bb0 con 0x7fd7640d12c0 2024-09-21T05:40:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.809+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd75c019020 con 0x7fd7640d12c0 2024-09-21T05:40:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.810+0000 7fd75a7fc640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd748078b90 0x7fd74807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:38.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.810+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7fd75c09b350 con 0x7fd7640d12c0 2024-09-21T05:40:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.811+0000 7fd769afe640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd748078b90 0x7fd74807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:38.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.811+0000 7fd769afe640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd748078b90 0x7fd74807b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd760034000 tx=0x7fd760004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:38.815+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd75c020020 con 0x7fd7640d12c0 2024-09-21T05:40:39.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.003+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd76414cf00 con 0x7fd7640d12c0 2024-09-21T05:40:39.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.004+0000 7fd75a7fc640 1 -- 172.21.15.22:0/207085386 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16674 (secure 0 0 0) 0x7fd75c063820 con 0x7fd7640d12c0 2024-09-21T05:40:39.007 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:39.007 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":42,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:37.702282+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:35.364998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:36.653978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd748078b90 msgr2=0x7fd74807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd748078b90 0x7fd74807b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd760034000 tx=0x7fd760004730 comp rx=0 tx=0).stop 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 msgr2=0x7fd76414c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 0x7fd76414c740 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00cb10 tx=0x7fd75c00a060 comp rx=0 tx=0).stop 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 shutdown_connections 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd7640d12c0 0x7fd76414c740 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd7640ccbd0 0x7fd7640a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd748078b90 0x7fd74807b0a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:39.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 --2- 172.21.15.22:0/207085386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd7640b51c0 0x7fd764159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:39.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 >> 172.21.15.22:0/207085386 conn(0x7fd764072ae0 msgr2=0x7fd7640b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:39.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 shutdown_connections 2024-09-21T05:40:39.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.006+0000 7fd76bd62640 1 -- 172.21.15.22:0/207085386 wait complete. 2024-09-21T05:40:39.020 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa88a64640 1 -- 172.21.15.22:0/1348129323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 msgr2=0x7ffa70005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa88a64640 1 --2- 172.21.15.22:0/1348129323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa70005190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009b50 tx=0x7ffa70005830 comp rx=0 tx=0).stop 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa82d9d640 1 -- 172.21.15.22:0/1348129323 reap_dead start 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa88a64640 1 -- 172.21.15.22:0/1348129323 shutdown_connections 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa88a64640 1 -- 172.21.15.22:0/1348129323 >> 172.21.15.22:0/1348129323 conn(0x7ffa840b3f10 msgr2=0x7ffa840b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa88a64640 1 -- 172.21.15.22:0/1348129323 shutdown_connections 2024-09-21T05:40:39.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.138+0000 7ffa88a64640 1 -- 172.21.15.22:0/1348129323 wait complete. 2024-09-21T05:40:39.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.140+0000 7ffa88a64640 1 Processor -- start 2024-09-21T05:40:39.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.140+0000 7ffa88a64640 1 -- start start 2024-09-21T05:40:39.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa88a64640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa84158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:39.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa88a64640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa841574a0 0x7ffa841470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:39.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa88a64640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa841475f0 0x7ffa84147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:39.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa88a64640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840d1520 con 0x7ffa70004d40 2024-09-21T05:40:39.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa88a64640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840d2a40 con 0x7ffa841475f0 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa88a64640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840d16a0 con 0x7ffa841574a0 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa8259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa84158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa8259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa84158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59598/0 (socket says 172.21.15.22:59598) 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa8259c640 1 -- 172.21.15.22:0/3624867386 learned_addr learned my addr 172.21.15.22:0/3624867386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa81d9b640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa841574a0 0x7ffa841470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa82d9d640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa841475f0 0x7ffa84147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa8259c640 1 -- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa841574a0 msgr2=0x7ffa841470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.141+0000 7ffa8259c640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa841574a0 0x7ffa841470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa8259c640 1 -- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa841475f0 msgr2=0x7ffa84147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa8259c640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa841475f0 0x7ffa84147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa8259c640 1 -- 172.21.15.22:0/3624867386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa84157e60 con 0x7ffa70004d40 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa8259c640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa84158f00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa74001950 tx=0x7ffa74001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74015070 con 0x7ffa70004d40 2024-09-21T05:40:39.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa7400ebb0 con 0x7ffa70004d40 2024-09-21T05:40:39.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa84158140 con 0x7ffa70004d40 2024-09-21T05:40:39.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74011460 con 0x7ffa70004d40 2024-09-21T05:40:39.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.142+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa84158700 con 0x7ffa70004d40 2024-09-21T05:40:39.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.143+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa840d1b90 con 0x7ffa70004d40 2024-09-21T05:40:39.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.144+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa74018070 con 0x7ffa70004d40 2024-09-21T05:40:39.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.144+0000 7ffa7affd640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffa64078b90 0x7ffa6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:39.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.144+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5709+0+0 (secure 0 0 0) 0x7ffa7409b740 con 0x7ffa70004d40 2024-09-21T05:40:39.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.144+0000 7ffa81d9b640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffa64078b90 0x7ffa6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:39.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.145+0000 7ffa81d9b640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffa64078b90 0x7ffa6407b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa840cc8e0 tx=0x7ffa6c001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:39.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.149+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa74063530 con 0x7ffa70004d40 2024-09-21T05:40:39.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:39.380+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffa840cd5e0 con 0x7ffa70004d40 2024-09-21T05:40:40.299 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-21T05:40:40.297+0000 7fc2c8aab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:40:40.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.327+0000 7ffa7affd640 1 -- 172.21.15.22:0/3624867386 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7ffa74067f60 con 0x7ffa70004d40 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.330+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffa64078b90 msgr2=0x7ffa6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.330+0000 7ffa88a64640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffa64078b90 0x7ffa6407b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa840cc8e0 tx=0x7ffa6c001e40 comp rx=0 tx=0).stop 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.330+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 msgr2=0x7ffa84158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.330+0000 7ffa88a64640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa84158f00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ffa74001950 tx=0x7ffa74001e10 comp rx=0 tx=0).stop 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 shutdown_connections 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffa841475f0 0x7ffa84147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffa64078b90 0x7ffa6407b0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:40.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffa841574a0 0x7ffa841470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:40.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 --2- 172.21.15.22:0/3624867386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffa70004d40 0x7ffa84158f00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:40.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 >> 172.21.15.22:0/3624867386 conn(0x7ffa840b3f10 msgr2=0x7ffa840b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:40.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 shutdown_connections 2024-09-21T05:40:40.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.331+0000 7ffa88a64640 1 -- 172.21.15.22:0/3624867386 wait complete. 2024-09-21T05:40:40.347 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:40:40.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:40:40.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e23c3640 1 -- 172.21.15.22:0/1066055293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04c4004d40 msgr2=0x7f04c4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:40.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1066055293 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04c4004d40 0x7f04c4005210 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005fd0 tx=0x7f04c4005830 comp rx=0 tx=0).stop 2024-09-21T05:40:40.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e0960640 1 -- 172.21.15.22:0/1066055293 reap_dead start 2024-09-21T05:40:40.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e23c3640 1 -- 172.21.15.22:0/1066055293 shutdown_connections 2024-09-21T05:40:40.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e23c3640 1 -- 172.21.15.22:0/1066055293 >> 172.21.15.22:0/1066055293 conn(0x7f04dc0d1210 msgr2=0x7f04dc073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:40.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e23c3640 1 -- 172.21.15.22:0/1066055293 shutdown_connections 2024-09-21T05:40:40.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.473+0000 7f04e23c3640 1 -- 172.21.15.22:0/1066055293 wait complete. 2024-09-21T05:40:40.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.475+0000 7f04e23c3640 1 Processor -- start 2024-09-21T05:40:40.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.475+0000 7f04e23c3640 1 -- start start 2024-09-21T05:40:40.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.476+0000 7f04e23c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04c4004d40 0x7f04dc0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.476+0000 7f04e23c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04dc0aaef0 0x7f04dc1f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.476+0000 7f04e23c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 0x7f04dc1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.476+0000 7f04e23c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc15fe00 con 0x7f04dc1f70b0 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.476+0000 7f04e23c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc15fc80 con 0x7f04dc0aaef0 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.476+0000 7f04e23c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc15ff80 con 0x7f04c4004d40 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 0x7f04dc1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 0x7f04dc1f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59612/0 (socket says 172.21.15.22:59612) 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 -- 172.21.15.22:0/1216573198 learned_addr learned my addr 172.21.15.22:0/1216573198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04db7fe640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04dc0aaef0 0x7f04dc1f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04dbfff640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04c4004d40 0x7f04dc0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 -- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04c4004d40 msgr2=0x7f04dc0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04c4004d40 0x7f04dc0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 -- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04dc0aaef0 msgr2=0x7f04dc1f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:40.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04dc0aaef0 0x7f04dc1f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 -- 172.21.15.22:0/1216573198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc0cc9a0 con 0x7f04dc1f70b0 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e0960640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 0x7f04dc1f74c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005fd0 tx=0x7f04d4033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4042070 con 0x7f04dc1f70b0 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04d4003970 con 0x7f04dc1f70b0 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc0ab620 con 0x7f04dc1f70b0 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4038040 con 0x7f04dc1f70b0 2024-09-21T05:40:40.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.477+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc0abbe0 con 0x7f04dc1f70b0 2024-09-21T05:40:40.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.478+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc0b43f0 con 0x7f04dc1f70b0 2024-09-21T05:40:40.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.479+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04d4004030 con 0x7f04dc1f70b0 2024-09-21T05:40:40.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.480+0000 7f04d8ff9640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04c8078b90 0x7f04c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:40.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.480+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f04d40bb2d0 con 0x7f04dc1f70b0 2024-09-21T05:40:40.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.480+0000 7f04dbfff640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04c8078b90 0x7f04c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:40.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.481+0000 7f04dbfff640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04c8078b90 0x7f04c807b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000c00 tx=0x7f04cc006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:40.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.484+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04d4083040 con 0x7f04dc1f70b0 2024-09-21T05:40:40.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:40.713+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f04dc0ab3b0 con 0x7f04dc1f70b0 2024-09-21T05:40:41.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.335+0000 7f04d8ff9640 1 -- 172.21.15.22:0/1216573198 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f04d4087a70 con 0x7f04dc1f70b0 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04c8078b90 msgr2=0x7f04c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04c8078b90 0x7f04c807b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000c00 tx=0x7f04cc006b70 comp rx=0 tx=0).stop 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 msgr2=0x7f04dc1f74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 0x7f04dc1f74c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005fd0 tx=0x7f04d4033bf0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 shutdown_connections 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04dc1f70b0 0x7f04dc1f74c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04dc0aaef0 0x7f04dc1f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04c8078b90 0x7f04c807b0a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 --2- 172.21.15.22:0/1216573198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04c4004d40 0x7f04dc0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 >> 172.21.15.22:0/1216573198 conn(0x7f04dc0d1210 msgr2=0x7f04dc15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 shutdown_connections 2024-09-21T05:40:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.338+0000 7f04e23c3640 1 -- 172.21.15.22:0/1216573198 wait complete. 2024-09-21T05:40:41.354 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:40:41.354 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 -- 172.21.15.22:0/3053172962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4041600b0 msgr2=0x7fb4040ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 --2- 172.21.15.22:0/3053172962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4041600b0 0x7fb4040ccbc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb4040b7c50 tx=0x7fb3ec02a5f0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 -- 172.21.15.22:0/3053172962 shutdown_connections 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 --2- 172.21.15.22:0/3053172962 >> v2:172.21.15.134:3300/0 conn(0x7fb4040cf320 0x7fb4040d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 --2- 172.21.15.22:0/3053172962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4041600b0 0x7fb4040ccbc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 -- 172.21.15.22:0/3053172962 >> 172.21.15.22:0/3053172962 conn(0x7fb40406d4f0 msgr2=0x7fb404071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 -- 172.21.15.22:0/3053172962 shutdown_connections 2024-09-21T05:40:41.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.474+0000 7fb40b04d640 1 -- 172.21.15.22:0/3053172962 wait complete. 2024-09-21T05:40:41.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.476+0000 7fb40b04d640 1 Processor -- start 2024-09-21T05:40:41.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.476+0000 7fb40b04d640 1 -- start start 2024-09-21T05:40:41.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb40b04d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4040b7cc0 0x7fb40414a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb40b04d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 0x7fb4040b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb40b04d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4041600b0 0x7fb4040cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb40b04d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040d13c0 con 0x7fb4040cf320 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb40b04d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040d1240 con 0x7fb4041600b0 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb40b04d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040d1540 con 0x7fb4040b7cc0 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 0x7fb4040b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 0x7fb4040b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59622/0 (socket says 172.21.15.22:59622) 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 -- 172.21.15.22:0/767758479 learned_addr learned my addr 172.21.15.22:0/767758479 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb408de9640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4040b7cc0 0x7fb40414a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb4095ea640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4041600b0 0x7fb4040cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 -- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4040b7cc0 msgr2=0x7fb40414a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4040b7cc0 0x7fb40414a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 -- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4041600b0 msgr2=0x7fb4040cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4041600b0 0x7fb4040cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 -- 172.21.15.22:0/767758479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040b2dd0 con 0x7fb4040cf320 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb408de9640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4040b7cc0 0x7fb40414a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:41.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb4095ea640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4041600b0 0x7fb4040cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:41.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb403fff640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 0x7fb4040b4740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c7d0 tx=0x7fb3f400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:41.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.478+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4017070 con 0x7fb4040cf320 2024-09-21T05:40:41.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.479+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3f4005070 con 0x7fb4040cf320 2024-09-21T05:40:41.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.479+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4040b30b0 con 0x7fb4040cf320 2024-09-21T05:40:41.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.479+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40054a0 con 0x7fb4040cf320 2024-09-21T05:40:41.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.479+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4040caf20 con 0x7fb4040cf320 2024-09-21T05:40:41.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.480+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4040d1a60 con 0x7fb4040cf320 2024-09-21T05:40:41.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.480+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb3f401a070 con 0x7fb4040cf320 2024-09-21T05:40:41.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.481+0000 7fb4017fa640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb3e4078b90 0x7fb3e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.481+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb3f409a370 con 0x7fb4040cf320 2024-09-21T05:40:41.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.482+0000 7fb408de9640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb3e4078b90 0x7fb3e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.482+0000 7fb408de9640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb3e4078b90 0x7fb3e407b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec033000 tx=0x7fb3ec004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:41.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.486+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb3f4062840 con 0x7fb4040cf320 2024-09-21T05:40:41.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.711+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4040b2b60 con 0x7fb4040cf320 2024-09-21T05:40:41.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.711+0000 7fb4017fa640 1 -- 172.21.15.22:0/767758479 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fb3f4062140 con 0x7fb4040cf320 2024-09-21T05:40:41.714 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:41.714 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:40:41.714 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:40:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb3e4078b90 msgr2=0x7fb3e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb3e4078b90 0x7fb3e407b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec033000 tx=0x7fb3ec004730 comp rx=0 tx=0).stop 2024-09-21T05:40:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 msgr2=0x7fb4040b4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 0x7fb4040b4740 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c7d0 tx=0x7fb3f400cc90 comp rx=0 tx=0).stop 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 shutdown_connections 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4041600b0 0x7fb4040cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4040cf320 0x7fb4040b4740 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb3e4078b90 0x7fb3e407b0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 --2- 172.21.15.22:0/767758479 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4040b7cc0 0x7fb40414a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 >> 172.21.15.22:0/767758479 conn(0x7fb40406d4f0 msgr2=0x7fb40415f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 shutdown_connections 2024-09-21T05:40:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.714+0000 7fb40b04d640 1 -- 172.21.15.22:0/767758479 wait complete. 2024-09-21T05:40:41.728 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:40:41.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c93de6640 1 -- 172.21.15.22:0/2776485266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 msgr2=0x7f7c78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c93de6640 1 --2- 172.21.15.22:0/2776485266 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c78004800 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009b50 tx=0x7f7c78005830 comp rx=0 tx=0).stop 2024-09-21T05:40:41.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c92383640 1 -- 172.21.15.22:0/2776485266 reap_dead start 2024-09-21T05:40:41.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c93de6640 1 -- 172.21.15.22:0/2776485266 shutdown_connections 2024-09-21T05:40:41.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c93de6640 1 -- 172.21.15.22:0/2776485266 >> 172.21.15.22:0/2776485266 conn(0x7f7c8c09d840 msgr2=0x7f7c8c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:41.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c93de6640 1 -- 172.21.15.22:0/2776485266 shutdown_connections 2024-09-21T05:40:41.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.844+0000 7f7c93de6640 1 -- 172.21.15.22:0/2776485266 wait complete. 2024-09-21T05:40:41.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.846+0000 7f7c93de6640 1 Processor -- start 2024-09-21T05:40:41.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.846+0000 7f7c93de6640 1 -- start start 2024-09-21T05:40:41.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c93de6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c8c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c93de6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c8c15d4a0 0x7f7c8c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c93de6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c0cbe20 0x7f7c8c0cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c93de6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c0cfb10 con 0x7f7c780041c0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c93de6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c06f9d0 con 0x7f7c8c0cbe20 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c93de6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8c0cfc90 con 0x7f7c8c15d4a0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c8c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c8c0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59646/0 (socket says 172.21.15.22:59646) 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 -- 172.21.15.22:0/1543775682 learned_addr learned my addr 172.21.15.22:0/1543775682 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c92383640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c0cbe20 0x7f7c8c0cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91381640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c8c15d4a0 0x7f7c8c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 -- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c8c15d4a0 msgr2=0x7f7c8c0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c8c15d4a0 0x7f7c8c0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 -- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c0cbe20 msgr2=0x7f7c8c0cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c0cbe20 0x7f7c8c0cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.847+0000 7f7c91b82640 1 -- 172.21.15.22:0/1543775682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c0cc770 con 0x7f7c780041c0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.848+0000 7f7c91b82640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c8c0ce0e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006430 tx=0x7f7c880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.848+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88015070 con 0x7f7c780041c0 2024-09-21T05:40:41.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.848+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c88011900 con 0x7f7c780041c0 2024-09-21T05:40:41.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.848+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c15dcb0 con 0x7f7c780041c0 2024-09-21T05:40:41.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.848+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88011d50 con 0x7f7c780041c0 2024-09-21T05:40:41.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.848+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c15e270 con 0x7f7c780041c0 2024-09-21T05:40:41.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.849+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8c0d00d0 con 0x7f7c780041c0 2024-09-21T05:40:41.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.850+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7c88018070 con 0x7f7c780041c0 2024-09-21T05:40:41.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.851+0000 7f7c827fc640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c70078b90 0x7f7c7007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:41.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.851+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7c8809bc00 con 0x7f7c780041c0 2024-09-21T05:40:41.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.851+0000 7f7c91381640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c70078b90 0x7f7c7007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:41.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.852+0000 7f7c91381640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c70078b90 0x7f7c7007b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c035000 tx=0x7f7c7c0046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:41.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:41.855+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c88063b10 con 0x7f7c780041c0 2024-09-21T05:40:42.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.043+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c8c06ea20 con 0x7f7c780041c0 2024-09-21T05:40:42.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.044+0000 7f7c827fc640 1 -- 172.21.15.22:0/1543775682 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16661 (secure 0 0 0) 0x7f7c88068540 con 0x7f7c780041c0 2024-09-21T05:40:42.047 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:42.047 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":43,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:40:39.317721+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:35.364998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:36.653978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:40:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.046+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c70078b90 msgr2=0x7f7c7007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.046+0000 7f7c93de6640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c70078b90 0x7f7c7007b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c035000 tx=0x7f7c7c0046c0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.046+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 msgr2=0x7f7c8c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.046+0000 7f7c93de6640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c8c0ce0e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c88006430 tx=0x7f7c880039c0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.046+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 shutdown_connections 2024-09-21T05:40:42.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.046+0000 7f7c93de6640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7c8c0cbe20 0x7f7c8c0cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.047+0000 7f7c93de6640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7c70078b90 0x7f7c7007b0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.047+0000 7f7c93de6640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7c8c15d4a0 0x7f7c8c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.047+0000 7f7c93de6640 1 --2- 172.21.15.22:0/1543775682 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7c780041c0 0x7f7c8c0ce0e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.047+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 >> 172.21.15.22:0/1543775682 conn(0x7f7c8c09d840 msgr2=0x7f7c8c0b8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:42.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.047+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 shutdown_connections 2024-09-21T05:40:42.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.047+0000 7f7c93de6640 1 -- 172.21.15.22:0/1543775682 wait complete. 2024-09-21T05:40:42.060 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:40:42.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:40:42.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.175+0000 7f5dbae63640 1 -- 172.21.15.22:0/3185807463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da40039e0 msgr2=0x7f5da4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.175+0000 7f5dbae63640 1 --2- 172.21.15.22:0/3185807463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da40039e0 0x7f5da4003e10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5da8001740 tx=0x7f5da4005830 comp rx=0 tx=0).stop 2024-09-21T05:40:42.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.175+0000 7f5db9400640 1 -- 172.21.15.22:0/3185807463 reap_dead start 2024-09-21T05:40:42.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.176+0000 7f5dbae63640 1 -- 172.21.15.22:0/3185807463 shutdown_connections 2024-09-21T05:40:42.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.176+0000 7f5dbae63640 1 -- 172.21.15.22:0/3185807463 >> 172.21.15.22:0/3185807463 conn(0x7f5db40b3f10 msgr2=0x7f5db40b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:42.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.176+0000 7f5dbae63640 1 -- 172.21.15.22:0/3185807463 shutdown_connections 2024-09-21T05:40:42.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.176+0000 7f5dbae63640 1 -- 172.21.15.22:0/3185807463 wait complete. 2024-09-21T05:40:42.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.177+0000 7f5dbae63640 1 Processor -- start 2024-09-21T05:40:42.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.178+0000 7f5dbae63640 1 -- start start 2024-09-21T05:40:42.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5dbae63640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da40039e0 0x7f5db4159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5dbae63640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5db414a6b0 0x7f5db415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5dbae63640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 0x7f5db40ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5dbae63640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40d0fe0 con 0x7f5db40ca1b0 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5dbae63640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5db40d0e60 con 0x7f5da40039e0 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5dbae63640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5db40d1160 con 0x7f5db414a6b0 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5db9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 0x7f5db40ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5db9400640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 0x7f5db40ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59656/0 (socket says 172.21.15.22:59656) 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5db9400640 1 -- 172.21.15.22:0/4193996670 learned_addr learned my addr 172.21.15.22:0/4193996670 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5db3fff640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5db414a6b0 0x7f5db415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.179+0000 7f5db8bff640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da40039e0 0x7f5db4159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db9400640 1 -- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5db414a6b0 msgr2=0x7f5db415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db9400640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5db414a6b0 0x7f5db415db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db9400640 1 -- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da40039e0 msgr2=0x7f5db4159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db9400640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da40039e0 0x7f5db4159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db9400640 1 -- 172.21.15.22:0/4193996670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db415ca90 con 0x7f5db40ca1b0 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db3fff640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5db414a6b0 0x7f5db415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db9400640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 0x7f5db40ca5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5da4004780 tx=0x7f5dac00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac00cc20 con 0x7f5db40ca1b0 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db415cd70 con 0x7f5db40ca1b0 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dac017070 con 0x7f5db40ca1b0 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0116c0 con 0x7f5db40ca1b0 2024-09-21T05:40:42.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.180+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db415d330 con 0x7f5db40ca1b0 2024-09-21T05:40:42.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.181+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db40d16f0 con 0x7f5db40ca1b0 2024-09-21T05:40:42.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.182+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5dac011860 con 0x7f5db40ca1b0 2024-09-21T05:40:42.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.182+0000 7f5db17fa640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d98078bc0 0x7f5d9807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.182+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f5dac09b230 con 0x7f5db40ca1b0 2024-09-21T05:40:42.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.183+0000 7f5db8bff640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d98078bc0 0x7f5d9807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.183+0000 7f5db8bff640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d98078bc0 0x7f5d9807b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5da8001710 tx=0x7f5da8001270 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:42.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.187+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5dac063680 con 0x7f5db40ca1b0 2024-09-21T05:40:42.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.370+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5db415c1a0 con 0x7f5d98078bc0 2024-09-21T05:40:42.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.375+0000 7f5db17fa640 1 -- 172.21.15.22:0/4193996670 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f5db415c1a0 con 0x7f5d98078bc0 2024-09-21T05:40:42.380 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:42.381 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:40:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d98078bc0 msgr2=0x7f5d9807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d98078bc0 0x7f5d9807b0d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5da8001710 tx=0x7f5da8001270 comp rx=0 tx=0).stop 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 msgr2=0x7f5db40ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 0x7f5db40ca5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5da4004780 tx=0x7f5dac00eb60 comp rx=0 tx=0).stop 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 shutdown_connections 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5db40ca1b0 0x7f5db40ca5c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d98078bc0 0x7f5d9807b0d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5db414a6b0 0x7f5db415db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 --2- 172.21.15.22:0/4193996670 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da40039e0 0x7f5db4159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 >> 172.21.15.22:0/4193996670 conn(0x7f5db40b3f10 msgr2=0x7f5db40b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 shutdown_connections 2024-09-21T05:40:42.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.379+0000 7f5dbae63640 1 -- 172.21.15.22:0/4193996670 wait complete. 2024-09-21T05:40:42.392 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-21T05:40:41.756121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254596,"kb_used_data":5072,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720124,"statfs":{"total":773094113280,"available":772833406976,"internally_reserved":0,"allocated":5193728,"data_stored":2772260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454284"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252178+0000","last_change":"2024-09-21T05:40:38.330274+0000","last_active":"2024-09-21T05:40:40.252178+0000","last_peered":"2024-09-21T05:40:40.252178+0000","last_clean":"2024-09-21T05:40:40.252178+0000","last_became_active":"2024-09-21T05:40:38.329961+0000","last_became_peered":"2024-09-21T05:40:38.329961+0000","last_unstale":"2024-09-21T05:40:40.252178+0000","last_undegraded":"2024-09-21T05:40:40.252178+0000","last_fullsized":"2024-09-21T05:40:40.252178+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:17.674999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290709+0000","last_change":"2024-09-21T05:40:38.332597+0000","last_active":"2024-09-21T05:40:40.290709+0000","last_peered":"2024-09-21T05:40:40.290709+0000","last_clean":"2024-09-21T05:40:40.290709+0000","last_became_active":"2024-09-21T05:40:38.332186+0000","last_became_peered":"2024-09-21T05:40:38.332186+0000","last_unstale":"2024-09-21T05:40:40.290709+0000","last_undegraded":"2024-09-21T05:40:40.290709+0000","last_fullsized":"2024-09-21T05:40:40.290709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:04.861694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326604+0000","last_change":"2024-09-21T05:40:38.326789+0000","last_active":"2024-09-21T05:40:39.326604+0000","last_peered":"2024-09-21T05:40:39.326604+0000","last_clean":"2024-09-21T05:40:39.326604+0000","last_became_active":"2024-09-21T05:40:38.326453+0000","last_became_peered":"2024-09-21T05:40:38.326453+0000","last_unstale":"2024-09-21T05:40:39.326604+0000","last_undegraded":"2024-09-21T05:40:39.326604+0000","last_fullsized":"2024-09-21T05:40:39.326604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:33.045464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290207+0000","last_change":"2024-09-21T05:40:38.335476+0000","last_active":"2024-09-21T05:40:40.290207+0000","last_peered":"2024-09-21T05:40:40.290207+0000","last_clean":"2024-09-21T05:40:40.290207+0000","last_became_active":"2024-09-21T05:40:38.335164+0000","last_became_peered":"2024-09-21T05:40:38.335164+0000","last_unstale":"2024-09-21T05:40:40.290207+0000","last_undegraded":"2024-09-21T05:40:40.290207+0000","last_fullsized":"2024-09-21T05:40:40.290207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:04.619749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290459+0000","last_change":"2024-09-21T05:40:38.336868+0000","last_active":"2024-09-21T05:40:40.290459+0000","last_peered":"2024-09-21T05:40:40.290459+0000","last_clean":"2024-09-21T05:40:40.290459+0000","last_became_active":"2024-09-21T05:40:38.335649+0000","last_became_peered":"2024-09-21T05:40:38.335649+0000","last_unstale":"2024-09-21T05:40:40.290459+0000","last_undegraded":"2024-09-21T05:40:40.290459+0000","last_fullsized":"2024-09-21T05:40:40.290459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.290645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331021+0000","last_change":"2024-09-21T05:40:38.324263+0000","last_active":"2024-09-21T05:40:39.331021+0000","last_peered":"2024-09-21T05:40:39.331021+0000","last_clean":"2024-09-21T05:40:39.331021+0000","last_became_active":"2024-09-21T05:40:38.323929+0000","last_became_peered":"2024-09-21T05:40:38.323929+0000","last_unstale":"2024-09-21T05:40:39.331021+0000","last_undegraded":"2024-09-21T05:40:39.331021+0000","last_fullsized":"2024-09-21T05:40:39.331021+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:54.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252079+0000","last_change":"2024-09-21T05:40:38.324679+0000","last_active":"2024-09-21T05:40:40.252079+0000","last_peered":"2024-09-21T05:40:40.252079+0000","last_clean":"2024-09-21T05:40:40.252079+0000","last_became_active":"2024-09-21T05:40:38.324151+0000","last_became_peered":"2024-09-21T05:40:38.324151+0000","last_unstale":"2024-09-21T05:40:40.252079+0000","last_undegraded":"2024-09-21T05:40:40.252079+0000","last_fullsized":"2024-09-21T05:40:40.252079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:28.977443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.289801+0000","last_change":"2024-09-21T05:40:38.336608+0000","last_active":"2024-09-21T05:40:40.289801+0000","last_peered":"2024-09-21T05:40:40.289801+0000","last_clean":"2024-09-21T05:40:40.289801+0000","last_became_active":"2024-09-21T05:40:38.336313+0000","last_became_peered":"2024-09-21T05:40:38.336313+0000","last_unstale":"2024-09-21T05:40:40.289801+0000","last_undegraded":"2024-09-21T05:40:40.289801+0000","last_fullsized":"2024-09-21T05:40:40.289801+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:10.445362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290580+0000","last_change":"2024-09-21T05:40:38.326602+0000","last_active":"2024-09-21T05:40:40.290580+0000","last_peered":"2024-09-21T05:40:40.290580+0000","last_clean":"2024-09-21T05:40:40.290580+0000","last_became_active":"2024-09-21T05:40:38.326258+0000","last_became_peered":"2024-09-21T05:40:38.326258+0000","last_unstale":"2024-09-21T05:40:40.290580+0000","last_undegraded":"2024-09-21T05:40:40.290580+0000","last_fullsized":"2024-09-21T05:40:40.290580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:04.033127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290383+0000","last_change":"2024-09-21T05:40:38.336204+0000","last_active":"2024-09-21T05:40:40.290383+0000","last_peered":"2024-09-21T05:40:40.290383+0000","last_clean":"2024-09-21T05:40:40.290383+0000","last_became_active":"2024-09-21T05:40:38.335847+0000","last_became_peered":"2024-09-21T05:40:38.335847+0000","last_unstale":"2024-09-21T05:40:40.290383+0000","last_undegraded":"2024-09-21T05:40:40.290383+0000","last_fullsized":"2024-09-21T05:40:40.290383+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:33.044056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290328+0000","last_change":"2024-09-21T05:40:38.327801+0000","last_active":"2024-09-21T05:40:40.290328+0000","last_peered":"2024-09-21T05:40:40.290328+0000","last_clean":"2024-09-21T05:40:40.290328+0000","last_became_active":"2024-09-21T05:40:38.327467+0000","last_became_peered":"2024-09-21T05:40:38.327467+0000","last_unstale":"2024-09-21T05:40:40.290328+0000","last_undegraded":"2024-09-21T05:40:40.290328+0000","last_fullsized":"2024-09-21T05:40:40.290328+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.657690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290582+0000","last_change":"2024-09-21T05:40:38.325233+0000","last_active":"2024-09-21T05:40:40.290582+0000","last_peered":"2024-09-21T05:40:40.290582+0000","last_clean":"2024-09-21T05:40:40.290582+0000","last_became_active":"2024-09-21T05:40:38.324706+0000","last_became_peered":"2024-09-21T05:40:38.324706+0000","last_unstale":"2024-09-21T05:40:40.290582+0000","last_undegraded":"2024-09-21T05:40:40.290582+0000","last_fullsized":"2024-09-21T05:40:40.290582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:06.218583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468086+0000","last_change":"2024-09-21T05:40:38.333890+0000","last_active":"2024-09-21T05:40:39.468086+0000","last_peered":"2024-09-21T05:40:39.468086+0000","last_clean":"2024-09-21T05:40:39.468086+0000","last_became_active":"2024-09-21T05:40:38.333295+0000","last_became_peered":"2024-09-21T05:40:38.333295+0000","last_unstale":"2024-09-21T05:40:39.468086+0000","last_undegraded":"2024-09-21T05:40:39.468086+0000","last_fullsized":"2024-09-21T05:40:39.468086+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:23.094937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290052+0000","last_change":"2024-09-21T05:40:38.327863+0000","last_active":"2024-09-21T05:40:40.290052+0000","last_peered":"2024-09-21T05:40:40.290052+0000","last_clean":"2024-09-21T05:40:40.290052+0000","last_became_active":"2024-09-21T05:40:38.327545+0000","last_became_peered":"2024-09-21T05:40:38.327545+0000","last_unstale":"2024-09-21T05:40:40.290052+0000","last_undegraded":"2024-09-21T05:40:40.290052+0000","last_fullsized":"2024-09-21T05:40:40.290052+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:04.077212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330683+0000","last_change":"2024-09-21T05:40:38.337288+0000","last_active":"2024-09-21T05:40:39.330683+0000","last_peered":"2024-09-21T05:40:39.330683+0000","last_clean":"2024-09-21T05:40:39.330683+0000","last_became_active":"2024-09-21T05:40:38.332735+0000","last_became_peered":"2024-09-21T05:40:38.332735+0000","last_unstale":"2024-09-21T05:40:39.330683+0000","last_undegraded":"2024-09-21T05:40:39.330683+0000","last_fullsized":"2024-09-21T05:40:39.330683+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:06.680601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327067+0000","last_change":"2024-09-21T05:40:38.334118+0000","last_active":"2024-09-21T05:40:39.327067+0000","last_peered":"2024-09-21T05:40:39.327067+0000","last_clean":"2024-09-21T05:40:39.327067+0000","last_became_active":"2024-09-21T05:40:38.333377+0000","last_became_peered":"2024-09-21T05:40:38.333377+0000","last_unstale":"2024-09-21T05:40:39.327067+0000","last_undegraded":"2024-09-21T05:40:39.327067+0000","last_fullsized":"2024-09-21T05:40:39.327067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:56.494541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326953+0000","last_change":"2024-09-21T05:40:38.331506+0000","last_active":"2024-09-21T05:40:39.326953+0000","last_peered":"2024-09-21T05:40:39.326953+0000","last_clean":"2024-09-21T05:40:39.326953+0000","last_became_active":"2024-09-21T05:40:38.330834+0000","last_became_peered":"2024-09-21T05:40:38.330834+0000","last_unstale":"2024-09-21T05:40:39.326953+0000","last_undegraded":"2024-09-21T05:40:39.326953+0000","last_fullsized":"2024-09-21T05:40:39.326953+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:40.005392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327553+0000","last_change":"2024-09-21T05:40:38.330958+0000","last_active":"2024-09-21T05:40:39.327553+0000","last_peered":"2024-09-21T05:40:39.327553+0000","last_clean":"2024-09-21T05:40:39.327553+0000","last_became_active":"2024-09-21T05:40:38.330464+0000","last_became_peered":"2024-09-21T05:40:38.330464+0000","last_unstale":"2024-09-21T05:40:39.327553+0000","last_undegraded":"2024-09-21T05:40:39.327553+0000","last_fullsized":"2024-09-21T05:40:39.327553+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:52.346070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328910+0000","last_change":"2024-09-21T05:40:38.334230+0000","last_active":"2024-09-21T05:40:39.328910+0000","last_peered":"2024-09-21T05:40:39.328910+0000","last_clean":"2024-09-21T05:40:39.328910+0000","last_became_active":"2024-09-21T05:40:38.333657+0000","last_became_peered":"2024-09-21T05:40:38.333657+0000","last_unstale":"2024-09-21T05:40:39.328910+0000","last_undegraded":"2024-09-21T05:40:39.328910+0000","last_fullsized":"2024-09-21T05:40:39.328910+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:34.068405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468178+0000","last_change":"2024-09-21T05:40:38.324727+0000","last_active":"2024-09-21T05:40:39.468178+0000","last_peered":"2024-09-21T05:40:39.468178+0000","last_clean":"2024-09-21T05:40:39.468178+0000","last_became_active":"2024-09-21T05:40:38.324391+0000","last_became_peered":"2024-09-21T05:40:38.324391+0000","last_unstale":"2024-09-21T05:40:39.468178+0000","last_undegraded":"2024-09-21T05:40:39.468178+0000","last_fullsized":"2024-09-21T05:40:39.468178+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:05.433617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330812+0000","last_change":"2024-09-21T05:40:38.323393+0000","last_active":"2024-09-21T05:40:39.330812+0000","last_peered":"2024-09-21T05:40:39.330812+0000","last_clean":"2024-09-21T05:40:39.330812+0000","last_became_active":"2024-09-21T05:40:38.323081+0000","last_became_peered":"2024-09-21T05:40:38.323081+0000","last_unstale":"2024-09-21T05:40:39.330812+0000","last_undegraded":"2024-09-21T05:40:39.330812+0000","last_fullsized":"2024-09-21T05:40:39.330812+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:30.763187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290139+0000","last_change":"2024-09-21T05:40:38.335709+0000","last_active":"2024-09-21T05:40:40.290139+0000","last_peered":"2024-09-21T05:40:40.290139+0000","last_clean":"2024-09-21T05:40:40.290139+0000","last_became_active":"2024-09-21T05:40:38.335326+0000","last_became_peered":"2024-09-21T05:40:38.335326+0000","last_unstale":"2024-09-21T05:40:40.290139+0000","last_undegraded":"2024-09-21T05:40:40.290139+0000","last_fullsized":"2024-09-21T05:40:40.290139+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:31.452726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328175+0000","last_change":"2024-09-21T05:40:38.322036+0000","last_active":"2024-09-21T05:40:39.328175+0000","last_peered":"2024-09-21T05:40:39.328175+0000","last_clean":"2024-09-21T05:40:39.328175+0000","last_became_active":"2024-09-21T05:40:38.321520+0000","last_became_peered":"2024-09-21T05:40:38.321520+0000","last_unstale":"2024-09-21T05:40:39.328175+0000","last_undegraded":"2024-09-21T05:40:39.328175+0000","last_fullsized":"2024-09-21T05:40:39.328175+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:05.675696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327167+0000","last_change":"2024-09-21T05:40:38.316750+0000","last_active":"2024-09-21T05:40:39.327167+0000","last_peered":"2024-09-21T05:40:39.327167+0000","last_clean":"2024-09-21T05:40:39.327167+0000","last_became_active":"2024-09-21T05:40:38.316421+0000","last_became_peered":"2024-09-21T05:40:38.316421+0000","last_unstale":"2024-09-21T05:40:39.327167+0000","last_undegraded":"2024-09-21T05:40:39.327167+0000","last_fullsized":"2024-09-21T05:40:39.327167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:30.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328579+0000","last_change":"2024-09-21T05:40:38.323973+0000","last_active":"2024-09-21T05:40:39.328579+0000","last_peered":"2024-09-21T05:40:39.328579+0000","last_clean":"2024-09-21T05:40:39.328579+0000","last_became_active":"2024-09-21T05:40:38.323679+0000","last_became_peered":"2024-09-21T05:40:38.323679+0000","last_unstale":"2024-09-21T05:40:39.328579+0000","last_undegraded":"2024-09-21T05:40:39.328579+0000","last_fullsized":"2024-09-21T05:40:39.328579+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:55.757724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468201+0000","last_change":"2024-09-21T05:40:38.332424+0000","last_active":"2024-09-21T05:40:39.468201+0000","last_peered":"2024-09-21T05:40:39.468201+0000","last_clean":"2024-09-21T05:40:39.468201+0000","last_became_active":"2024-09-21T05:40:38.332103+0000","last_became_peered":"2024-09-21T05:40:38.332103+0000","last_unstale":"2024-09-21T05:40:39.468201+0000","last_undegraded":"2024-09-21T05:40:39.468201+0000","last_fullsized":"2024-09-21T05:40:39.468201+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:10.099432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326895+0000","last_change":"2024-09-21T05:40:38.331197+0000","last_active":"2024-09-21T05:40:39.326895+0000","last_peered":"2024-09-21T05:40:39.326895+0000","last_clean":"2024-09-21T05:40:39.326895+0000","last_became_active":"2024-09-21T05:40:38.330810+0000","last_became_peered":"2024-09-21T05:40:38.330810+0000","last_unstale":"2024-09-21T05:40:39.326895+0000","last_undegraded":"2024-09-21T05:40:39.326895+0000","last_fullsized":"2024-09-21T05:40:39.326895+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:18.877290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290693+0000","last_change":"2024-09-21T05:40:38.324984+0000","last_active":"2024-09-21T05:40:40.290693+0000","last_peered":"2024-09-21T05:40:40.290693+0000","last_clean":"2024-09-21T05:40:40.290693+0000","last_became_active":"2024-09-21T05:40:38.324689+0000","last_became_peered":"2024-09-21T05:40:38.324689+0000","last_unstale":"2024-09-21T05:40:40.290693+0000","last_undegraded":"2024-09-21T05:40:40.290693+0000","last_fullsized":"2024-09-21T05:40:40.290693+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:01.118251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328403+0000","last_change":"2024-09-21T05:40:38.323067+0000","last_active":"2024-09-21T05:40:39.328403+0000","last_peered":"2024-09-21T05:40:39.328403+0000","last_clean":"2024-09-21T05:40:39.328403+0000","last_became_active":"2024-09-21T05:40:38.322712+0000","last_became_peered":"2024-09-21T05:40:38.322712+0000","last_unstale":"2024-09-21T05:40:39.328403+0000","last_undegraded":"2024-09-21T05:40:39.328403+0000","last_fullsized":"2024-09-21T05:40:39.328403+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:24.225101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326957+0000","last_change":"2024-09-21T05:40:38.331014+0000","last_active":"2024-09-21T05:40:39.326957+0000","last_peered":"2024-09-21T05:40:39.326957+0000","last_clean":"2024-09-21T05:40:39.326957+0000","last_became_active":"2024-09-21T05:40:38.329546+0000","last_became_peered":"2024-09-21T05:40:38.329546+0000","last_unstale":"2024-09-21T05:40:39.326957+0000","last_undegraded":"2024-09-21T05:40:39.326957+0000","last_fullsized":"2024-09-21T05:40:39.326957+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:09.940672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330741+0000","last_change":"2024-09-21T05:40:38.336671+0000","last_active":"2024-09-21T05:40:39.330741+0000","last_peered":"2024-09-21T05:40:39.330741+0000","last_clean":"2024-09-21T05:40:39.330741+0000","last_became_active":"2024-09-21T05:40:38.326405+0000","last_became_peered":"2024-09-21T05:40:38.326405+0000","last_unstale":"2024-09-21T05:40:39.330741+0000","last_undegraded":"2024-09-21T05:40:39.330741+0000","last_fullsized":"2024-09-21T05:40:39.330741+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:29.362091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468084+0000","last_change":"2024-09-21T05:40:38.321598+0000","last_active":"2024-09-21T05:40:39.468084+0000","last_peered":"2024-09-21T05:40:39.468084+0000","last_clean":"2024-09-21T05:40:39.468084+0000","last_became_active":"2024-09-21T05:40:38.321274+0000","last_became_peered":"2024-09-21T05:40:38.321274+0000","last_unstale":"2024-09-21T05:40:39.468084+0000","last_undegraded":"2024-09-21T05:40:39.468084+0000","last_fullsized":"2024-09-21T05:40:39.468084+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:26.990021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325740+0000","last_change":"2024-09-21T05:40:38.332561+0000","last_active":"2024-09-21T05:40:39.325740+0000","last_peered":"2024-09-21T05:40:39.325740+0000","last_clean":"2024-09-21T05:40:39.325740+0000","last_became_active":"2024-09-21T05:40:38.332234+0000","last_became_peered":"2024-09-21T05:40:38.332234+0000","last_unstale":"2024-09-21T05:40:39.325740+0000","last_undegraded":"2024-09-21T05:40:39.325740+0000","last_fullsized":"2024-09-21T05:40:39.325740+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:11.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331307+0000","last_change":"2024-09-21T05:40:37.318315+0000","last_active":"2024-09-21T05:40:39.331307+0000","last_peered":"2024-09-21T05:40:39.331307+0000","last_clean":"2024-09-21T05:40:39.331307+0000","last_became_active":"2024-09-21T05:40:37.317977+0000","last_became_peered":"2024-09-21T05:40:37.317977+0000","last_unstale":"2024-09-21T05:40:39.331307+0000","last_undegraded":"2024-09-21T05:40:39.331307+0000","last_fullsized":"2024-09-21T05:40:39.331307+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:00.112713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328839+0000","last_change":"2024-09-21T05:40:38.334024+0000","last_active":"2024-09-21T05:40:39.328839+0000","last_peered":"2024-09-21T05:40:39.328839+0000","last_clean":"2024-09-21T05:40:39.328839+0000","last_became_active":"2024-09-21T05:40:38.333692+0000","last_became_peered":"2024-09-21T05:40:38.333692+0000","last_unstale":"2024-09-21T05:40:39.328839+0000","last_undegraded":"2024-09-21T05:40:39.328839+0000","last_fullsized":"2024-09-21T05:40:39.328839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:33.272585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327752+0000","last_change":"2024-09-21T05:40:37.320643+0000","last_active":"2024-09-21T05:40:39.327752+0000","last_peered":"2024-09-21T05:40:39.327752+0000","last_clean":"2024-09-21T05:40:39.327752+0000","last_became_active":"2024-09-21T05:40:37.320265+0000","last_became_peered":"2024-09-21T05:40:37.320265+0000","last_unstale":"2024-09-21T05:40:39.327752+0000","last_undegraded":"2024-09-21T05:40:39.327752+0000","last_fullsized":"2024-09-21T05:40:39.327752+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:10.187416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290833+0000","last_change":"2024-09-21T05:40:38.339196+0000","last_active":"2024-09-21T05:40:40.290833+0000","last_peered":"2024-09-21T05:40:40.290833+0000","last_clean":"2024-09-21T05:40:40.290833+0000","last_became_active":"2024-09-21T05:40:38.338873+0000","last_became_peered":"2024-09-21T05:40:38.338873+0000","last_unstale":"2024-09-21T05:40:40.290833+0000","last_undegraded":"2024-09-21T05:40:40.290833+0000","last_fullsized":"2024-09-21T05:40:40.290833+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:21.924171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325838+0000","last_change":"2024-09-21T05:40:37.323969+0000","last_active":"2024-09-21T05:40:39.325838+0000","last_peered":"2024-09-21T05:40:39.325838+0000","last_clean":"2024-09-21T05:40:39.325838+0000","last_became_active":"2024-09-21T05:40:37.323482+0000","last_became_peered":"2024-09-21T05:40:37.323482+0000","last_unstale":"2024-09-21T05:40:39.325838+0000","last_undegraded":"2024-09-21T05:40:39.325838+0000","last_fullsized":"2024-09-21T05:40:39.325838+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:02.536168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325858+0000","last_change":"2024-09-21T05:40:38.320545+0000","last_active":"2024-09-21T05:40:39.325858+0000","last_peered":"2024-09-21T05:40:39.325858+0000","last_clean":"2024-09-21T05:40:39.325858+0000","last_became_active":"2024-09-21T05:40:38.320131+0000","last_became_peered":"2024-09-21T05:40:38.320131+0000","last_unstale":"2024-09-21T05:40:39.325858+0000","last_undegraded":"2024-09-21T05:40:39.325858+0000","last_fullsized":"2024-09-21T05:40:39.325858+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:08.515650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251830+0000","last_change":"2024-09-21T05:40:37.323806+0000","last_active":"2024-09-21T05:40:40.251830+0000","last_peered":"2024-09-21T05:40:40.251830+0000","last_clean":"2024-09-21T05:40:40.251830+0000","last_became_active":"2024-09-21T05:40:37.323294+0000","last_became_peered":"2024-09-21T05:40:37.323294+0000","last_unstale":"2024-09-21T05:40:40.251830+0000","last_undegraded":"2024-09-21T05:40:40.251830+0000","last_fullsized":"2024-09-21T05:40:40.251830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:27.835356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327207+0000","last_change":"2024-09-21T05:40:38.336978+0000","last_active":"2024-09-21T05:40:39.327207+0000","last_peered":"2024-09-21T05:40:39.327207+0000","last_clean":"2024-09-21T05:40:39.327207+0000","last_became_active":"2024-09-21T05:40:38.335623+0000","last_became_peered":"2024-09-21T05:40:38.335623+0000","last_unstale":"2024-09-21T05:40:39.327207+0000","last_undegraded":"2024-09-21T05:40:39.327207+0000","last_fullsized":"2024-09-21T05:40:39.327207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:06.676042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330974+0000","last_change":"2024-09-21T05:40:37.322204+0000","last_active":"2024-09-21T05:40:39.330974+0000","last_peered":"2024-09-21T05:40:39.330974+0000","last_clean":"2024-09-21T05:40:39.330974+0000","last_became_active":"2024-09-21T05:40:37.321480+0000","last_became_peered":"2024-09-21T05:40:37.321480+0000","last_unstale":"2024-09-21T05:40:39.330974+0000","last_undegraded":"2024-09-21T05:40:39.330974+0000","last_fullsized":"2024-09-21T05:40:39.330974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:49.906555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290739+0000","last_change":"2024-09-21T05:40:38.330049+0000","last_active":"2024-09-21T05:40:40.290739+0000","last_peered":"2024-09-21T05:40:40.290739+0000","last_clean":"2024-09-21T05:40:40.290739+0000","last_became_active":"2024-09-21T05:40:38.329736+0000","last_became_peered":"2024-09-21T05:40:38.329736+0000","last_unstale":"2024-09-21T05:40:40.290739+0000","last_undegraded":"2024-09-21T05:40:40.290739+0000","last_fullsized":"2024-09-21T05:40:40.290739+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:14.418415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327284+0000","last_change":"2024-09-21T05:40:37.322664+0000","last_active":"2024-09-21T05:40:39.327284+0000","last_peered":"2024-09-21T05:40:39.327284+0000","last_clean":"2024-09-21T05:40:39.327284+0000","last_became_active":"2024-09-21T05:40:37.322342+0000","last_became_peered":"2024-09-21T05:40:37.322342+0000","last_unstale":"2024-09-21T05:40:39.327284+0000","last_undegraded":"2024-09-21T05:40:39.327284+0000","last_fullsized":"2024-09-21T05:40:39.327284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:22.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290064+0000","last_change":"2024-09-21T05:40:38.337337+0000","last_active":"2024-09-21T05:40:40.290064+0000","last_peered":"2024-09-21T05:40:40.290064+0000","last_clean":"2024-09-21T05:40:40.290064+0000","last_became_active":"2024-09-21T05:40:38.337035+0000","last_became_peered":"2024-09-21T05:40:38.337035+0000","last_unstale":"2024-09-21T05:40:40.290064+0000","last_undegraded":"2024-09-21T05:40:40.290064+0000","last_fullsized":"2024-09-21T05:40:40.290064+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:12.226417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290335+0000","last_change":"2024-09-21T05:40:37.327298+0000","last_active":"2024-09-21T05:40:40.290335+0000","last_peered":"2024-09-21T05:40:40.290335+0000","last_clean":"2024-09-21T05:40:40.290335+0000","last_became_active":"2024-09-21T05:40:37.325933+0000","last_became_peered":"2024-09-21T05:40:37.325933+0000","last_unstale":"2024-09-21T05:40:40.290335+0000","last_undegraded":"2024-09-21T05:40:40.290335+0000","last_fullsized":"2024-09-21T05:40:40.290335+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:13.253336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327926+0000","last_change":"2024-09-21T05:40:38.316726+0000","last_active":"2024-09-21T05:40:39.327926+0000","last_peered":"2024-09-21T05:40:39.327926+0000","last_clean":"2024-09-21T05:40:39.327926+0000","last_became_active":"2024-09-21T05:40:38.316361+0000","last_became_peered":"2024-09-21T05:40:38.316361+0000","last_unstale":"2024-09-21T05:40:39.327926+0000","last_undegraded":"2024-09-21T05:40:39.327926+0000","last_fullsized":"2024-09-21T05:40:39.327926+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:15.982858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290688+0000","last_change":"2024-09-21T05:40:37.327740+0000","last_active":"2024-09-21T05:40:40.290688+0000","last_peered":"2024-09-21T05:40:40.290688+0000","last_clean":"2024-09-21T05:40:40.290688+0000","last_became_active":"2024-09-21T05:40:37.327132+0000","last_became_peered":"2024-09-21T05:40:37.327132+0000","last_unstale":"2024-09-21T05:40:40.290688+0000","last_undegraded":"2024-09-21T05:40:40.290688+0000","last_fullsized":"2024-09-21T05:40:40.290688+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:51.195745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327677+0000","last_change":"2024-09-21T05:40:38.329702+0000","last_active":"2024-09-21T05:40:39.327677+0000","last_peered":"2024-09-21T05:40:39.327677+0000","last_clean":"2024-09-21T05:40:39.327677+0000","last_became_active":"2024-09-21T05:40:38.329327+0000","last_became_peered":"2024-09-21T05:40:38.329327+0000","last_unstale":"2024-09-21T05:40:39.327677+0000","last_undegraded":"2024-09-21T05:40:39.327677+0000","last_fullsized":"2024-09-21T05:40:39.327677+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:12.644093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327123+0000","last_change":"2024-09-21T05:40:37.317918+0000","last_active":"2024-09-21T05:40:39.327123+0000","last_peered":"2024-09-21T05:40:39.327123+0000","last_clean":"2024-09-21T05:40:39.327123+0000","last_became_active":"2024-09-21T05:40:37.317592+0000","last_became_peered":"2024-09-21T05:40:37.317592+0000","last_unstale":"2024-09-21T05:40:39.327123+0000","last_undegraded":"2024-09-21T05:40:39.327123+0000","last_fullsized":"2024-09-21T05:40:39.327123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:57.233895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252291+0000","last_change":"2024-09-21T05:40:38.325511+0000","last_active":"2024-09-21T05:40:40.252291+0000","last_peered":"2024-09-21T05:40:40.252291+0000","last_clean":"2024-09-21T05:40:40.252291+0000","last_became_active":"2024-09-21T05:40:38.325183+0000","last_became_peered":"2024-09-21T05:40:38.325183+0000","last_unstale":"2024-09-21T05:40:40.252291+0000","last_undegraded":"2024-09-21T05:40:40.252291+0000","last_fullsized":"2024-09-21T05:40:40.252291+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:40.251382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290446+0000","last_change":"2024-09-21T05:40:37.330089+0000","last_active":"2024-09-21T05:40:40.290446+0000","last_peered":"2024-09-21T05:40:40.290446+0000","last_clean":"2024-09-21T05:40:40.290446+0000","last_became_active":"2024-09-21T05:40:37.329746+0000","last_became_peered":"2024-09-21T05:40:37.329746+0000","last_unstale":"2024-09-21T05:40:40.290446+0000","last_undegraded":"2024-09-21T05:40:40.290446+0000","last_fullsized":"2024-09-21T05:40:40.290446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:53.134451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331305+0000","last_change":"2024-09-21T05:40:38.337196+0000","last_active":"2024-09-21T05:40:39.331305+0000","last_peered":"2024-09-21T05:40:39.331305+0000","last_clean":"2024-09-21T05:40:39.331305+0000","last_became_active":"2024-09-21T05:40:38.336624+0000","last_became_peered":"2024-09-21T05:40:38.336624+0000","last_unstale":"2024-09-21T05:40:39.331305+0000","last_undegraded":"2024-09-21T05:40:39.331305+0000","last_fullsized":"2024-09-21T05:40:39.331305+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:48.786835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252191+0000","last_change":"2024-09-21T05:40:37.322671+0000","last_active":"2024-09-21T05:40:40.252191+0000","last_peered":"2024-09-21T05:40:40.252191+0000","last_clean":"2024-09-21T05:40:40.252191+0000","last_became_active":"2024-09-21T05:40:37.322365+0000","last_became_peered":"2024-09-21T05:40:37.322365+0000","last_unstale":"2024-09-21T05:40:40.252191+0000","last_undegraded":"2024-09-21T05:40:40.252191+0000","last_fullsized":"2024-09-21T05:40:40.252191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:32.039926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252193+0000","last_change":"2024-09-21T05:40:38.325033+0000","last_active":"2024-09-21T05:40:40.252193+0000","last_peered":"2024-09-21T05:40:40.252193+0000","last_clean":"2024-09-21T05:40:40.252193+0000","last_became_active":"2024-09-21T05:40:38.324671+0000","last_became_peered":"2024-09-21T05:40:38.324671+0000","last_unstale":"2024-09-21T05:40:40.252193+0000","last_undegraded":"2024-09-21T05:40:40.252193+0000","last_fullsized":"2024-09-21T05:40:40.252193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:30.214363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468092+0000","last_change":"2024-09-21T05:40:37.320854+0000","last_active":"2024-09-21T05:40:39.468092+0000","last_peered":"2024-09-21T05:40:39.468092+0000","last_clean":"2024-09-21T05:40:39.468092+0000","last_became_active":"2024-09-21T05:40:37.320330+0000","last_became_peered":"2024-09-21T05:40:37.320330+0000","last_unstale":"2024-09-21T05:40:39.468092+0000","last_undegraded":"2024-09-21T05:40:39.468092+0000","last_fullsized":"2024-09-21T05:40:39.468092+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:07.898462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327082+0000","last_change":"2024-09-21T05:40:38.330390+0000","last_active":"2024-09-21T05:40:39.327082+0000","last_peered":"2024-09-21T05:40:39.327082+0000","last_clean":"2024-09-21T05:40:39.327082+0000","last_became_active":"2024-09-21T05:40:38.330056+0000","last_became_peered":"2024-09-21T05:40:38.330056+0000","last_unstale":"2024-09-21T05:40:39.327082+0000","last_undegraded":"2024-09-21T05:40:39.327082+0000","last_fullsized":"2024-09-21T05:40:39.327082+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:00.304387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330980+0000","last_change":"2024-09-21T05:40:37.331195+0000","last_active":"2024-09-21T05:40:39.330980+0000","last_peered":"2024-09-21T05:40:39.330980+0000","last_clean":"2024-09-21T05:40:39.330980+0000","last_became_active":"2024-09-21T05:40:37.330827+0000","last_became_peered":"2024-09-21T05:40:37.330827+0000","last_unstale":"2024-09-21T05:40:39.330980+0000","last_undegraded":"2024-09-21T05:40:39.330980+0000","last_fullsized":"2024-09-21T05:40:39.330980+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:33.758138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290600+0000","last_change":"2024-09-21T05:40:38.332379+0000","last_active":"2024-09-21T05:40:40.290600+0000","last_peered":"2024-09-21T05:40:40.290600+0000","last_clean":"2024-09-21T05:40:40.290600+0000","last_became_active":"2024-09-21T05:40:38.332081+0000","last_became_peered":"2024-09-21T05:40:38.332081+0000","last_unstale":"2024-09-21T05:40:40.290600+0000","last_undegraded":"2024-09-21T05:40:40.290600+0000","last_fullsized":"2024-09-21T05:40:40.290600+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:00.804368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251948+0000","last_change":"2024-09-21T05:40:37.327625+0000","last_active":"2024-09-21T05:40:40.251948+0000","last_peered":"2024-09-21T05:40:40.251948+0000","last_clean":"2024-09-21T05:40:40.251948+0000","last_became_active":"2024-09-21T05:40:37.327293+0000","last_became_peered":"2024-09-21T05:40:37.327293+0000","last_unstale":"2024-09-21T05:40:40.251948+0000","last_undegraded":"2024-09-21T05:40:40.251948+0000","last_fullsized":"2024-09-21T05:40:40.251948+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:53.084749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330634+0000","last_change":"2024-09-21T05:40:38.317596+0000","last_active":"2024-09-21T05:40:39.330634+0000","last_peered":"2024-09-21T05:40:39.330634+0000","last_clean":"2024-09-21T05:40:39.330634+0000","last_became_active":"2024-09-21T05:40:38.317140+0000","last_became_peered":"2024-09-21T05:40:38.317140+0000","last_unstale":"2024-09-21T05:40:39.330634+0000","last_undegraded":"2024-09-21T05:40:39.330634+0000","last_fullsized":"2024-09-21T05:40:39.330634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.974088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330805+0000","last_change":"2024-09-21T05:40:37.325179+0000","last_active":"2024-09-21T05:40:39.330805+0000","last_peered":"2024-09-21T05:40:39.330805+0000","last_clean":"2024-09-21T05:40:39.330805+0000","last_became_active":"2024-09-21T05:40:37.324826+0000","last_became_peered":"2024-09-21T05:40:37.324826+0000","last_unstale":"2024-09-21T05:40:39.330805+0000","last_undegraded":"2024-09-21T05:40:39.330805+0000","last_fullsized":"2024-09-21T05:40:39.330805+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252135+0000","last_change":"2024-09-21T05:40:38.317787+0000","last_active":"2024-09-21T05:40:40.252135+0000","last_peered":"2024-09-21T05:40:40.252135+0000","last_clean":"2024-09-21T05:40:40.252135+0000","last_became_active":"2024-09-21T05:40:38.317415+0000","last_became_peered":"2024-09-21T05:40:38.317415+0000","last_unstale":"2024-09-21T05:40:40.252135+0000","last_undegraded":"2024-09-21T05:40:40.252135+0000","last_fullsized":"2024-09-21T05:40:40.252135+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:29.746451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290475+0000","last_change":"2024-09-21T05:40:37.329583+0000","last_active":"2024-09-21T05:40:40.290475+0000","last_peered":"2024-09-21T05:40:40.290475+0000","last_clean":"2024-09-21T05:40:40.290475+0000","last_became_active":"2024-09-21T05:40:37.329140+0000","last_became_peered":"2024-09-21T05:40:37.329140+0000","last_unstale":"2024-09-21T05:40:40.290475+0000","last_undegraded":"2024-09-21T05:40:40.290475+0000","last_fullsized":"2024-09-21T05:40:40.290475+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:22.193696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468152+0000","last_change":"2024-09-21T05:40:38.329003+0000","last_active":"2024-09-21T05:40:39.468152+0000","last_peered":"2024-09-21T05:40:39.468152+0000","last_clean":"2024-09-21T05:40:39.468152+0000","last_became_active":"2024-09-21T05:40:38.328642+0000","last_became_peered":"2024-09-21T05:40:38.328642+0000","last_unstale":"2024-09-21T05:40:39.468152+0000","last_undegraded":"2024-09-21T05:40:39.468152+0000","last_fullsized":"2024-09-21T05:40:39.468152+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:34.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468111+0000","last_change":"2024-09-21T05:40:37.318038+0000","last_active":"2024-09-21T05:40:39.468111+0000","last_peered":"2024-09-21T05:40:39.468111+0000","last_clean":"2024-09-21T05:40:39.468111+0000","last_became_active":"2024-09-21T05:40:37.317677+0000","last_became_peered":"2024-09-21T05:40:37.317677+0000","last_unstale":"2024-09-21T05:40:39.468111+0000","last_undegraded":"2024-09-21T05:40:39.468111+0000","last_fullsized":"2024-09-21T05:40:39.468111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:14.950529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290586+0000","last_change":"2024-09-21T05:40:38.339817+0000","last_active":"2024-09-21T05:40:40.290586+0000","last_peered":"2024-09-21T05:40:40.290586+0000","last_clean":"2024-09-21T05:40:40.290586+0000","last_became_active":"2024-09-21T05:40:38.339125+0000","last_became_peered":"2024-09-21T05:40:38.339125+0000","last_unstale":"2024-09-21T05:40:40.290586+0000","last_undegraded":"2024-09-21T05:40:40.290586+0000","last_fullsized":"2024-09-21T05:40:40.290586+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:41.823404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327636+0000","last_change":"2024-09-21T05:40:37.317152+0000","last_active":"2024-09-21T05:40:39.327636+0000","last_peered":"2024-09-21T05:40:39.327636+0000","last_clean":"2024-09-21T05:40:39.327636+0000","last_became_active":"2024-09-21T05:40:37.316833+0000","last_became_peered":"2024-09-21T05:40:37.316833+0000","last_unstale":"2024-09-21T05:40:39.327636+0000","last_undegraded":"2024-09-21T05:40:39.327636+0000","last_fullsized":"2024-09-21T05:40:39.327636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:25.079643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328167+0000","last_change":"2024-09-21T05:40:38.338495+0000","last_active":"2024-09-21T05:40:39.328167+0000","last_peered":"2024-09-21T05:40:39.328167+0000","last_clean":"2024-09-21T05:40:39.328167+0000","last_became_active":"2024-09-21T05:40:38.338223+0000","last_became_peered":"2024-09-21T05:40:38.338223+0000","last_unstale":"2024-09-21T05:40:39.328167+0000","last_undegraded":"2024-09-21T05:40:39.328167+0000","last_fullsized":"2024-09-21T05:40:39.328167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:42.898821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290593+0000","last_change":"2024-09-21T05:40:37.326507+0000","last_active":"2024-09-21T05:40:40.290593+0000","last_peered":"2024-09-21T05:40:40.290593+0000","last_clean":"2024-09-21T05:40:40.290593+0000","last_became_active":"2024-09-21T05:40:37.326144+0000","last_became_peered":"2024-09-21T05:40:37.326144+0000","last_unstale":"2024-09-21T05:40:40.290593+0000","last_undegraded":"2024-09-21T05:40:40.290593+0000","last_fullsized":"2024-09-21T05:40:40.290593+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:01.535809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328810+0000","last_change":"2024-09-21T05:40:38.327930+0000","last_active":"2024-09-21T05:40:39.328810+0000","last_peered":"2024-09-21T05:40:39.328810+0000","last_clean":"2024-09-21T05:40:39.328810+0000","last_became_active":"2024-09-21T05:40:38.327461+0000","last_became_peered":"2024-09-21T05:40:38.327461+0000","last_unstale":"2024-09-21T05:40:39.328810+0000","last_undegraded":"2024-09-21T05:40:39.328810+0000","last_fullsized":"2024-09-21T05:40:39.328810+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:09.114800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331394+0000","last_change":"2024-09-21T05:40:37.319715+0000","last_active":"2024-09-21T05:40:39.331394+0000","last_peered":"2024-09-21T05:40:39.331394+0000","last_clean":"2024-09-21T05:40:39.331394+0000","last_became_active":"2024-09-21T05:40:37.319316+0000","last_became_peered":"2024-09-21T05:40:37.319316+0000","last_unstale":"2024-09-21T05:40:39.331394+0000","last_undegraded":"2024-09-21T05:40:39.331394+0000","last_fullsized":"2024-09-21T05:40:39.331394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:39.247385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330866+0000","last_change":"2024-09-21T05:40:38.322534+0000","last_active":"2024-09-21T05:40:39.330866+0000","last_peered":"2024-09-21T05:40:39.330866+0000","last_clean":"2024-09-21T05:40:39.330866+0000","last_became_active":"2024-09-21T05:40:38.317035+0000","last_became_peered":"2024-09-21T05:40:38.317035+0000","last_unstale":"2024-09-21T05:40:39.330866+0000","last_undegraded":"2024-09-21T05:40:39.330866+0000","last_fullsized":"2024-09-21T05:40:39.330866+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:58.543686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331124+0000","last_change":"2024-09-21T05:40:37.321803+0000","last_active":"2024-09-21T05:40:39.331124+0000","last_peered":"2024-09-21T05:40:39.331124+0000","last_clean":"2024-09-21T05:40:39.331124+0000","last_became_active":"2024-09-21T05:40:37.321051+0000","last_became_peered":"2024-09-21T05:40:37.321051+0000","last_unstale":"2024-09-21T05:40:39.331124+0000","last_undegraded":"2024-09-21T05:40:39.331124+0000","last_fullsized":"2024-09-21T05:40:39.331124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:05.213045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325748+0000","last_change":"2024-09-21T05:40:38.325301+0000","last_active":"2024-09-21T05:40:39.325748+0000","last_peered":"2024-09-21T05:40:39.325748+0000","last_clean":"2024-09-21T05:40:39.325748+0000","last_became_active":"2024-09-21T05:40:38.324945+0000","last_became_peered":"2024-09-21T05:40:38.324945+0000","last_unstale":"2024-09-21T05:40:39.325748+0000","last_undegraded":"2024-09-21T05:40:39.325748+0000","last_fullsized":"2024-09-21T05:40:39.325748+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:52.729564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290292+0000","last_change":"2024-09-21T05:40:37.320338+0000","last_active":"2024-09-21T05:40:40.290292+0000","last_peered":"2024-09-21T05:40:40.290292+0000","last_clean":"2024-09-21T05:40:40.290292+0000","last_became_active":"2024-09-21T05:40:37.318592+0000","last_became_peered":"2024-09-21T05:40:37.318592+0000","last_unstale":"2024-09-21T05:40:40.290292+0000","last_undegraded":"2024-09-21T05:40:40.290292+0000","last_fullsized":"2024-09-21T05:40:40.290292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:47.171187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.467992+0000","last_change":"2024-09-21T05:40:38.318050+0000","last_active":"2024-09-21T05:40:39.467992+0000","last_peered":"2024-09-21T05:40:39.467992+0000","last_clean":"2024-09-21T05:40:39.467992+0000","last_became_active":"2024-09-21T05:40:38.317705+0000","last_became_peered":"2024-09-21T05:40:38.317705+0000","last_unstale":"2024-09-21T05:40:39.467992+0000","last_undegraded":"2024-09-21T05:40:39.467992+0000","last_fullsized":"2024-09-21T05:40:39.467992+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:03.088718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.289797+0000","last_change":"2024-09-21T05:40:37.331045+0000","last_active":"2024-09-21T05:40:40.289797+0000","last_peered":"2024-09-21T05:40:40.289797+0000","last_clean":"2024-09-21T05:40:40.289797+0000","last_became_active":"2024-09-21T05:40:37.330628+0000","last_became_peered":"2024-09-21T05:40:37.330628+0000","last_unstale":"2024-09-21T05:40:40.289797+0000","last_undegraded":"2024-09-21T05:40:40.289797+0000","last_fullsized":"2024-09-21T05:40:40.289797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:29.028421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290005+0000","last_change":"2024-09-21T05:40:38.329174+0000","last_active":"2024-09-21T05:40:40.290005+0000","last_peered":"2024-09-21T05:40:40.290005+0000","last_clean":"2024-09-21T05:40:40.290005+0000","last_became_active":"2024-09-21T05:40:38.328840+0000","last_became_peered":"2024-09-21T05:40:38.328840+0000","last_unstale":"2024-09-21T05:40:40.290005+0000","last_undegraded":"2024-09-21T05:40:40.290005+0000","last_fullsized":"2024-09-21T05:40:40.290005+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:59.294674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251610+0000","last_change":"2024-09-21T05:40:37.328411+0000","last_active":"2024-09-21T05:40:40.251610+0000","last_peered":"2024-09-21T05:40:40.251610+0000","last_clean":"2024-09-21T05:40:40.251610+0000","last_became_active":"2024-09-21T05:40:37.328077+0000","last_became_peered":"2024-09-21T05:40:37.328077+0000","last_unstale":"2024-09-21T05:40:40.251610+0000","last_undegraded":"2024-09-21T05:40:40.251610+0000","last_fullsized":"2024-09-21T05:40:40.251610+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:46.328645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328921+0000","last_change":"2024-09-21T05:40:38.326657+0000","last_active":"2024-09-21T05:40:39.328921+0000","last_peered":"2024-09-21T05:40:39.328921+0000","last_clean":"2024-09-21T05:40:39.328921+0000","last_became_active":"2024-09-21T05:40:38.326314+0000","last_became_peered":"2024-09-21T05:40:38.326314+0000","last_unstale":"2024-09-21T05:40:39.328921+0000","last_undegraded":"2024-09-21T05:40:39.328921+0000","last_fullsized":"2024-09-21T05:40:39.328921+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:19.755767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.332556+0000","last_change":"2024-09-21T05:40:37.320540+0000","last_active":"2024-09-21T05:40:39.332556+0000","last_peered":"2024-09-21T05:40:39.332556+0000","last_clean":"2024-09-21T05:40:39.332556+0000","last_became_active":"2024-09-21T05:40:37.320187+0000","last_became_peered":"2024-09-21T05:40:37.320187+0000","last_unstale":"2024-09-21T05:40:39.332556+0000","last_undegraded":"2024-09-21T05:40:39.332556+0000","last_fullsized":"2024-09-21T05:40:39.332556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:24.131813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290567+0000","last_change":"2024-09-21T05:40:38.331333+0000","last_active":"2024-09-21T05:40:40.290567+0000","last_peered":"2024-09-21T05:40:40.290567+0000","last_clean":"2024-09-21T05:40:40.290567+0000","last_became_active":"2024-09-21T05:40:38.331007+0000","last_became_peered":"2024-09-21T05:40:38.331007+0000","last_unstale":"2024-09-21T05:40:40.290567+0000","last_undegraded":"2024-09-21T05:40:40.290567+0000","last_fullsized":"2024-09-21T05:40:40.290567+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:25.948769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252261+0000","last_change":"2024-09-21T05:40:37.324649+0000","last_active":"2024-09-21T05:40:40.252261+0000","last_peered":"2024-09-21T05:40:40.252261+0000","last_clean":"2024-09-21T05:40:40.252261+0000","last_became_active":"2024-09-21T05:40:37.324273+0000","last_became_peered":"2024-09-21T05:40:37.324273+0000","last_unstale":"2024-09-21T05:40:40.252261+0000","last_undegraded":"2024-09-21T05:40:40.252261+0000","last_fullsized":"2024-09-21T05:40:40.252261+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:01.363659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252083+0000","last_change":"2024-09-21T05:40:38.318225+0000","last_active":"2024-09-21T05:40:40.252083+0000","last_peered":"2024-09-21T05:40:40.252083+0000","last_clean":"2024-09-21T05:40:40.252083+0000","last_became_active":"2024-09-21T05:40:38.317904+0000","last_became_peered":"2024-09-21T05:40:38.317904+0000","last_unstale":"2024-09-21T05:40:40.252083+0000","last_undegraded":"2024-09-21T05:40:40.252083+0000","last_fullsized":"2024-09-21T05:40:40.252083+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:59.612292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325672+0000","last_change":"2024-09-21T05:40:37.323447+0000","last_active":"2024-09-21T05:40:39.325672+0000","last_peered":"2024-09-21T05:40:39.325672+0000","last_clean":"2024-09-21T05:40:39.325672+0000","last_became_active":"2024-09-21T05:40:37.323066+0000","last_became_peered":"2024-09-21T05:40:37.323066+0000","last_unstale":"2024-09-21T05:40:39.325672+0000","last_undegraded":"2024-09-21T05:40:39.325672+0000","last_fullsized":"2024-09-21T05:40:39.325672+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:45.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331236+0000","last_change":"2024-09-21T05:40:38.330835+0000","last_active":"2024-09-21T05:40:39.331236+0000","last_peered":"2024-09-21T05:40:39.331236+0000","last_clean":"2024-09-21T05:40:39.331236+0000","last_became_active":"2024-09-21T05:40:38.330512+0000","last_became_peered":"2024-09-21T05:40:38.330512+0000","last_unstale":"2024-09-21T05:40:39.331236+0000","last_undegraded":"2024-09-21T05:40:39.331236+0000","last_fullsized":"2024-09-21T05:40:39.331236+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:50.874077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252010+0000","last_change":"2024-09-21T05:40:37.321320+0000","last_active":"2024-09-21T05:40:40.252010+0000","last_peered":"2024-09-21T05:40:40.252010+0000","last_clean":"2024-09-21T05:40:40.252010+0000","last_became_active":"2024-09-21T05:40:37.320971+0000","last_became_peered":"2024-09-21T05:40:37.320971+0000","last_unstale":"2024-09-21T05:40:40.252010+0000","last_undegraded":"2024-09-21T05:40:40.252010+0000","last_fullsized":"2024-09-21T05:40:40.252010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330819+0000","last_change":"2024-09-21T05:40:38.326032+0000","last_active":"2024-09-21T05:40:39.330819+0000","last_peered":"2024-09-21T05:40:39.330819+0000","last_clean":"2024-09-21T05:40:39.330819+0000","last_became_active":"2024-09-21T05:40:38.324730+0000","last_became_peered":"2024-09-21T05:40:38.324730+0000","last_unstale":"2024-09-21T05:40:39.330819+0000","last_undegraded":"2024-09-21T05:40:39.330819+0000","last_fullsized":"2024-09-21T05:40:39.330819+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:34.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327440+0000","last_change":"2024-09-21T05:40:37.325747+0000","last_active":"2024-09-21T05:40:39.327440+0000","last_peered":"2024-09-21T05:40:39.327440+0000","last_clean":"2024-09-21T05:40:39.327440+0000","last_became_active":"2024-09-21T05:40:37.325232+0000","last_became_peered":"2024-09-21T05:40:37.325232+0000","last_unstale":"2024-09-21T05:40:39.327440+0000","last_undegraded":"2024-09-21T05:40:39.327440+0000","last_fullsized":"2024-09-21T05:40:39.327440+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:45.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290449+0000","last_change":"2024-09-21T05:40:38.323692+0000","last_active":"2024-09-21T05:40:40.290449+0000","last_peered":"2024-09-21T05:40:40.290449+0000","last_clean":"2024-09-21T05:40:40.290449+0000","last_became_active":"2024-09-21T05:40:38.323389+0000","last_became_peered":"2024-09-21T05:40:38.323389+0000","last_unstale":"2024-09-21T05:40:40.290449+0000","last_undegraded":"2024-09-21T05:40:40.290449+0000","last_fullsized":"2024-09-21T05:40:40.290449+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:40.252922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325496+0000","last_change":"2024-09-21T05:40:37.319349+0000","last_active":"2024-09-21T05:40:39.325496+0000","last_peered":"2024-09-21T05:40:39.325496+0000","last_clean":"2024-09-21T05:40:39.325496+0000","last_became_active":"2024-09-21T05:40:37.318946+0000","last_became_peered":"2024-09-21T05:40:37.318946+0000","last_unstale":"2024-09-21T05:40:39.325496+0000","last_undegraded":"2024-09-21T05:40:39.325496+0000","last_fullsized":"2024-09-21T05:40:39.325496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:22.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326522+0000","last_change":"2024-09-21T05:40:38.340366+0000","last_active":"2024-09-21T05:40:39.326522+0000","last_peered":"2024-09-21T05:40:39.326522+0000","last_clean":"2024-09-21T05:40:39.326522+0000","last_became_active":"2024-09-21T05:40:38.340027+0000","last_became_peered":"2024-09-21T05:40:38.340027+0000","last_unstale":"2024-09-21T05:40:39.326522+0000","last_undegraded":"2024-09-21T05:40:39.326522+0000","last_fullsized":"2024-09-21T05:40:39.326522+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290620+0000","last_change":"2024-09-21T05:40:37.330151+0000","last_active":"2024-09-21T05:40:40.290620+0000","last_peered":"2024-09-21T05:40:40.290620+0000","last_clean":"2024-09-21T05:40:40.290620+0000","last_became_active":"2024-09-21T05:40:37.329765+0000","last_became_peered":"2024-09-21T05:40:37.329765+0000","last_unstale":"2024-09-21T05:40:40.290620+0000","last_undegraded":"2024-09-21T05:40:40.290620+0000","last_fullsized":"2024-09-21T05:40:40.290620+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:53.274256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290273+0000","last_change":"2024-09-21T05:40:38.325562+0000","last_active":"2024-09-21T05:40:40.290273+0000","last_peered":"2024-09-21T05:40:40.290273+0000","last_clean":"2024-09-21T05:40:40.290273+0000","last_became_active":"2024-09-21T05:40:38.325252+0000","last_became_peered":"2024-09-21T05:40:38.325252+0000","last_unstale":"2024-09-21T05:40:40.290273+0000","last_undegraded":"2024-09-21T05:40:40.290273+0000","last_fullsized":"2024-09-21T05:40:40.290273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.290645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290743+0000","last_change":"2024-09-21T05:40:37.325102+0000","last_active":"2024-09-21T05:40:40.290743+0000","last_peered":"2024-09-21T05:40:40.290743+0000","last_clean":"2024-09-21T05:40:40.290743+0000","last_became_active":"2024-09-21T05:40:37.324744+0000","last_became_peered":"2024-09-21T05:40:37.324744+0000","last_unstale":"2024-09-21T05:40:40.290743+0000","last_undegraded":"2024-09-21T05:40:40.290743+0000","last_fullsized":"2024-09-21T05:40:40.290743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:18.518420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":148,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251633+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:40:40.251633+0000","last_peered":"2024-09-21T05:40:40.251633+0000","last_clean":"2024-09-21T05:40:40.251633+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:40:40.251633+0000","last_undegraded":"2024-09-21T05:40:40.251633+0000","last_fullsized":"2024-09-21T05:40:40.251633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":790371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":507904,"data_stored":201525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":487424,"data_stored":193683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":507904,"data_stored":200551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":201547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35836,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":499712,"data_stored":199983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1081344,"data_stored":784015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:40:42.397 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:40:42.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 -- 172.21.15.22:0/1743423657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0d2ed0 msgr2=0x7fa7fc0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 --2- 172.21.15.22:0/1743423657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc0d1a80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0d5690 tx=0x7fa7ec02aa30 comp rx=0 tx=0).stop 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 -- 172.21.15.22:0/1743423657 shutdown_connections 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 --2- 172.21.15.22:0/1743423657 >> v2:172.21.15.134:3300/0 conn(0x7fa7fc0b9b70 0x7fa7fc0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 --2- 172.21.15.22:0/1743423657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc0d1a80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 -- 172.21.15.22:0/1743423657 >> 172.21.15.22:0/1743423657 conn(0x7fa7fc08bfc0 msgr2=0x7fa7fc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 -- 172.21.15.22:0/1743423657 shutdown_connections 2024-09-21T05:40:42.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.547+0000 7fa80158a640 1 -- 172.21.15.22:0/1743423657 wait complete. 2024-09-21T05:40:42.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.549+0000 7fa80158a640 1 Processor -- start 2024-09-21T05:40:42.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.549+0000 7fa80158a640 1 -- start start 2024-09-21T05:40:42.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa80158a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0b9b70 0x7fa7fc0a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa80158a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 0x7fa7fc1f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa80158a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc1f79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa80158a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0cfe80 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa80158a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc0cfd00 con 0x7fa7fc0d2ed0 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa80158a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc0d4940 con 0x7fa7fc0b9b70 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa7fa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 0x7fa7fc1f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa7fa7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 0x7fa7fc1f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59678/0 (socket says 172.21.15.22:59678) 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.550+0000 7fa7fa7fc640 1 -- 172.21.15.22:0/3727705775 learned_addr learned my addr 172.21.15.22:0/3727705775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7faffd640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0b9b70 0x7fa7fc0a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fb7fe640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc1f79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fa7fc640 1 -- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0b9b70 msgr2=0x7fa7fc0a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fa7fc640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0b9b70 0x7fa7fc0a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fa7fc640 1 -- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d2ed0 msgr2=0x7fa7fc1f79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fa7fc640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc1f79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fa7fc640 1 -- 172.21.15.22:0/3727705775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc1f6fa0 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fb7fe640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc1f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7fa7fc640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 0x7fa7fc1f7480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d740 tx=0x7fa7f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0017070 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7f0009d20 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc0c9440 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f00106c0 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.551+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc0c9a00 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.552+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc0d4e40 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.553+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa7f001a070 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.554+0000 7fa7e37fe640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa7e8078bc0 0x7fa7e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.554+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fa7f009ad00 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.554+0000 7fa7faffd640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa7e8078bc0 0x7fa7e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.554+0000 7fa7faffd640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa7e8078bc0 0x7fa7e807b0d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec035000 tx=0x7fa7ec03b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:42.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.558+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa7f0062b80 con 0x7fa7fc0cf5d0 2024-09-21T05:40:42.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.745+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7fc0c91d0 con 0x7fa7e8078bc0 2024-09-21T05:40:42.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.750+0000 7fa7e37fe640 1 -- 172.21.15.22:0/3727705775 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fa7fc0c91d0 con 0x7fa7e8078bc0 2024-09-21T05:40:42.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:42.756 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:40:42.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa7e8078bc0 msgr2=0x7fa7e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa7e8078bc0 0x7fa7e807b0d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec035000 tx=0x7fa7ec03b040 comp rx=0 tx=0).stop 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 msgr2=0x7fa7fc1f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 0x7fa7fc1f7480 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d740 tx=0x7fa7f000dc00 comp rx=0 tx=0).stop 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 shutdown_connections 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7fc0d2ed0 0x7fa7fc1f79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7fc0cf5d0 0x7fa7fc1f7480 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa7e8078bc0 0x7fa7e807b0d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 --2- 172.21.15.22:0/3727705775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7fc0b9b70 0x7fa7fc0a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 >> 172.21.15.22:0/3727705775 conn(0x7fa7fc08bfc0 msgr2=0x7fa7fc0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 shutdown_connections 2024-09-21T05:40:42.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.754+0000 7fa80158a640 1 -- 172.21.15.22:0/3727705775 wait complete. 2024-09-21T05:40:42.767 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-09-21T05:40:41.756121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254596,"kb_used_data":5072,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754720124,"statfs":{"total":773094113280,"available":772833406976,"internally_reserved":0,"allocated":5193728,"data_stored":2772260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.454284"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252178+0000","last_change":"2024-09-21T05:40:38.330274+0000","last_active":"2024-09-21T05:40:40.252178+0000","last_peered":"2024-09-21T05:40:40.252178+0000","last_clean":"2024-09-21T05:40:40.252178+0000","last_became_active":"2024-09-21T05:40:38.329961+0000","last_became_peered":"2024-09-21T05:40:38.329961+0000","last_unstale":"2024-09-21T05:40:40.252178+0000","last_undegraded":"2024-09-21T05:40:40.252178+0000","last_fullsized":"2024-09-21T05:40:40.252178+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:17.674999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290709+0000","last_change":"2024-09-21T05:40:38.332597+0000","last_active":"2024-09-21T05:40:40.290709+0000","last_peered":"2024-09-21T05:40:40.290709+0000","last_clean":"2024-09-21T05:40:40.290709+0000","last_became_active":"2024-09-21T05:40:38.332186+0000","last_became_peered":"2024-09-21T05:40:38.332186+0000","last_unstale":"2024-09-21T05:40:40.290709+0000","last_undegraded":"2024-09-21T05:40:40.290709+0000","last_fullsized":"2024-09-21T05:40:40.290709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:50:04.861694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326604+0000","last_change":"2024-09-21T05:40:38.326789+0000","last_active":"2024-09-21T05:40:39.326604+0000","last_peered":"2024-09-21T05:40:39.326604+0000","last_clean":"2024-09-21T05:40:39.326604+0000","last_became_active":"2024-09-21T05:40:38.326453+0000","last_became_peered":"2024-09-21T05:40:38.326453+0000","last_unstale":"2024-09-21T05:40:39.326604+0000","last_undegraded":"2024-09-21T05:40:39.326604+0000","last_fullsized":"2024-09-21T05:40:39.326604+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:33.045464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290207+0000","last_change":"2024-09-21T05:40:38.335476+0000","last_active":"2024-09-21T05:40:40.290207+0000","last_peered":"2024-09-21T05:40:40.290207+0000","last_clean":"2024-09-21T05:40:40.290207+0000","last_became_active":"2024-09-21T05:40:38.335164+0000","last_became_peered":"2024-09-21T05:40:38.335164+0000","last_unstale":"2024-09-21T05:40:40.290207+0000","last_undegraded":"2024-09-21T05:40:40.290207+0000","last_fullsized":"2024-09-21T05:40:40.290207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:37:04.619749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290459+0000","last_change":"2024-09-21T05:40:38.336868+0000","last_active":"2024-09-21T05:40:40.290459+0000","last_peered":"2024-09-21T05:40:40.290459+0000","last_clean":"2024-09-21T05:40:40.290459+0000","last_became_active":"2024-09-21T05:40:38.335649+0000","last_became_peered":"2024-09-21T05:40:38.335649+0000","last_unstale":"2024-09-21T05:40:40.290459+0000","last_undegraded":"2024-09-21T05:40:40.290459+0000","last_fullsized":"2024-09-21T05:40:40.290459+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.290645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331021+0000","last_change":"2024-09-21T05:40:38.324263+0000","last_active":"2024-09-21T05:40:39.331021+0000","last_peered":"2024-09-21T05:40:39.331021+0000","last_clean":"2024-09-21T05:40:39.331021+0000","last_became_active":"2024-09-21T05:40:38.323929+0000","last_became_peered":"2024-09-21T05:40:38.323929+0000","last_unstale":"2024-09-21T05:40:39.331021+0000","last_undegraded":"2024-09-21T05:40:39.331021+0000","last_fullsized":"2024-09-21T05:40:39.331021+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:54.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252079+0000","last_change":"2024-09-21T05:40:38.324679+0000","last_active":"2024-09-21T05:40:40.252079+0000","last_peered":"2024-09-21T05:40:40.252079+0000","last_clean":"2024-09-21T05:40:40.252079+0000","last_became_active":"2024-09-21T05:40:38.324151+0000","last_became_peered":"2024-09-21T05:40:38.324151+0000","last_unstale":"2024-09-21T05:40:40.252079+0000","last_undegraded":"2024-09-21T05:40:40.252079+0000","last_fullsized":"2024-09-21T05:40:40.252079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:28.977443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.289801+0000","last_change":"2024-09-21T05:40:38.336608+0000","last_active":"2024-09-21T05:40:40.289801+0000","last_peered":"2024-09-21T05:40:40.289801+0000","last_clean":"2024-09-21T05:40:40.289801+0000","last_became_active":"2024-09-21T05:40:38.336313+0000","last_became_peered":"2024-09-21T05:40:38.336313+0000","last_unstale":"2024-09-21T05:40:40.289801+0000","last_undegraded":"2024-09-21T05:40:40.289801+0000","last_fullsized":"2024-09-21T05:40:40.289801+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:10.445362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290580+0000","last_change":"2024-09-21T05:40:38.326602+0000","last_active":"2024-09-21T05:40:40.290580+0000","last_peered":"2024-09-21T05:40:40.290580+0000","last_clean":"2024-09-21T05:40:40.290580+0000","last_became_active":"2024-09-21T05:40:38.326258+0000","last_became_peered":"2024-09-21T05:40:38.326258+0000","last_unstale":"2024-09-21T05:40:40.290580+0000","last_undegraded":"2024-09-21T05:40:40.290580+0000","last_fullsized":"2024-09-21T05:40:40.290580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:04.033127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290383+0000","last_change":"2024-09-21T05:40:38.336204+0000","last_active":"2024-09-21T05:40:40.290383+0000","last_peered":"2024-09-21T05:40:40.290383+0000","last_clean":"2024-09-21T05:40:40.290383+0000","last_became_active":"2024-09-21T05:40:38.335847+0000","last_became_peered":"2024-09-21T05:40:38.335847+0000","last_unstale":"2024-09-21T05:40:40.290383+0000","last_undegraded":"2024-09-21T05:40:40.290383+0000","last_fullsized":"2024-09-21T05:40:40.290383+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:33.044056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290328+0000","last_change":"2024-09-21T05:40:38.327801+0000","last_active":"2024-09-21T05:40:40.290328+0000","last_peered":"2024-09-21T05:40:40.290328+0000","last_clean":"2024-09-21T05:40:40.290328+0000","last_became_active":"2024-09-21T05:40:38.327467+0000","last_became_peered":"2024-09-21T05:40:38.327467+0000","last_unstale":"2024-09-21T05:40:40.290328+0000","last_undegraded":"2024-09-21T05:40:40.290328+0000","last_fullsized":"2024-09-21T05:40:40.290328+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:42.657690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290582+0000","last_change":"2024-09-21T05:40:38.325233+0000","last_active":"2024-09-21T05:40:40.290582+0000","last_peered":"2024-09-21T05:40:40.290582+0000","last_clean":"2024-09-21T05:40:40.290582+0000","last_became_active":"2024-09-21T05:40:38.324706+0000","last_became_peered":"2024-09-21T05:40:38.324706+0000","last_unstale":"2024-09-21T05:40:40.290582+0000","last_undegraded":"2024-09-21T05:40:40.290582+0000","last_fullsized":"2024-09-21T05:40:40.290582+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:06.218583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468086+0000","last_change":"2024-09-21T05:40:38.333890+0000","last_active":"2024-09-21T05:40:39.468086+0000","last_peered":"2024-09-21T05:40:39.468086+0000","last_clean":"2024-09-21T05:40:39.468086+0000","last_became_active":"2024-09-21T05:40:38.333295+0000","last_became_peered":"2024-09-21T05:40:38.333295+0000","last_unstale":"2024-09-21T05:40:39.468086+0000","last_undegraded":"2024-09-21T05:40:39.468086+0000","last_fullsized":"2024-09-21T05:40:39.468086+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:23.094937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290052+0000","last_change":"2024-09-21T05:40:38.327863+0000","last_active":"2024-09-21T05:40:40.290052+0000","last_peered":"2024-09-21T05:40:40.290052+0000","last_clean":"2024-09-21T05:40:40.290052+0000","last_became_active":"2024-09-21T05:40:38.327545+0000","last_became_peered":"2024-09-21T05:40:38.327545+0000","last_unstale":"2024-09-21T05:40:40.290052+0000","last_undegraded":"2024-09-21T05:40:40.290052+0000","last_fullsized":"2024-09-21T05:40:40.290052+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:04.077212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330683+0000","last_change":"2024-09-21T05:40:38.337288+0000","last_active":"2024-09-21T05:40:39.330683+0000","last_peered":"2024-09-21T05:40:39.330683+0000","last_clean":"2024-09-21T05:40:39.330683+0000","last_became_active":"2024-09-21T05:40:38.332735+0000","last_became_peered":"2024-09-21T05:40:38.332735+0000","last_unstale":"2024-09-21T05:40:39.330683+0000","last_undegraded":"2024-09-21T05:40:39.330683+0000","last_fullsized":"2024-09-21T05:40:39.330683+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:06.680601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327067+0000","last_change":"2024-09-21T05:40:38.334118+0000","last_active":"2024-09-21T05:40:39.327067+0000","last_peered":"2024-09-21T05:40:39.327067+0000","last_clean":"2024-09-21T05:40:39.327067+0000","last_became_active":"2024-09-21T05:40:38.333377+0000","last_became_peered":"2024-09-21T05:40:38.333377+0000","last_unstale":"2024-09-21T05:40:39.327067+0000","last_undegraded":"2024-09-21T05:40:39.327067+0000","last_fullsized":"2024-09-21T05:40:39.327067+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:56.494541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326953+0000","last_change":"2024-09-21T05:40:38.331506+0000","last_active":"2024-09-21T05:40:39.326953+0000","last_peered":"2024-09-21T05:40:39.326953+0000","last_clean":"2024-09-21T05:40:39.326953+0000","last_became_active":"2024-09-21T05:40:38.330834+0000","last_became_peered":"2024-09-21T05:40:38.330834+0000","last_unstale":"2024-09-21T05:40:39.326953+0000","last_undegraded":"2024-09-21T05:40:39.326953+0000","last_fullsized":"2024-09-21T05:40:39.326953+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:40.005392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327553+0000","last_change":"2024-09-21T05:40:38.330958+0000","last_active":"2024-09-21T05:40:39.327553+0000","last_peered":"2024-09-21T05:40:39.327553+0000","last_clean":"2024-09-21T05:40:39.327553+0000","last_became_active":"2024-09-21T05:40:38.330464+0000","last_became_peered":"2024-09-21T05:40:38.330464+0000","last_unstale":"2024-09-21T05:40:39.327553+0000","last_undegraded":"2024-09-21T05:40:39.327553+0000","last_fullsized":"2024-09-21T05:40:39.327553+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:52.346070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328910+0000","last_change":"2024-09-21T05:40:38.334230+0000","last_active":"2024-09-21T05:40:39.328910+0000","last_peered":"2024-09-21T05:40:39.328910+0000","last_clean":"2024-09-21T05:40:39.328910+0000","last_became_active":"2024-09-21T05:40:38.333657+0000","last_became_peered":"2024-09-21T05:40:38.333657+0000","last_unstale":"2024-09-21T05:40:39.328910+0000","last_undegraded":"2024-09-21T05:40:39.328910+0000","last_fullsized":"2024-09-21T05:40:39.328910+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:34.068405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468178+0000","last_change":"2024-09-21T05:40:38.324727+0000","last_active":"2024-09-21T05:40:39.468178+0000","last_peered":"2024-09-21T05:40:39.468178+0000","last_clean":"2024-09-21T05:40:39.468178+0000","last_became_active":"2024-09-21T05:40:38.324391+0000","last_became_peered":"2024-09-21T05:40:38.324391+0000","last_unstale":"2024-09-21T05:40:39.468178+0000","last_undegraded":"2024-09-21T05:40:39.468178+0000","last_fullsized":"2024-09-21T05:40:39.468178+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:05.433617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330812+0000","last_change":"2024-09-21T05:40:38.323393+0000","last_active":"2024-09-21T05:40:39.330812+0000","last_peered":"2024-09-21T05:40:39.330812+0000","last_clean":"2024-09-21T05:40:39.330812+0000","last_became_active":"2024-09-21T05:40:38.323081+0000","last_became_peered":"2024-09-21T05:40:38.323081+0000","last_unstale":"2024-09-21T05:40:39.330812+0000","last_undegraded":"2024-09-21T05:40:39.330812+0000","last_fullsized":"2024-09-21T05:40:39.330812+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:30.763187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290139+0000","last_change":"2024-09-21T05:40:38.335709+0000","last_active":"2024-09-21T05:40:40.290139+0000","last_peered":"2024-09-21T05:40:40.290139+0000","last_clean":"2024-09-21T05:40:40.290139+0000","last_became_active":"2024-09-21T05:40:38.335326+0000","last_became_peered":"2024-09-21T05:40:38.335326+0000","last_unstale":"2024-09-21T05:40:40.290139+0000","last_undegraded":"2024-09-21T05:40:40.290139+0000","last_fullsized":"2024-09-21T05:40:40.290139+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:31.452726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328175+0000","last_change":"2024-09-21T05:40:38.322036+0000","last_active":"2024-09-21T05:40:39.328175+0000","last_peered":"2024-09-21T05:40:39.328175+0000","last_clean":"2024-09-21T05:40:39.328175+0000","last_became_active":"2024-09-21T05:40:38.321520+0000","last_became_peered":"2024-09-21T05:40:38.321520+0000","last_unstale":"2024-09-21T05:40:39.328175+0000","last_undegraded":"2024-09-21T05:40:39.328175+0000","last_fullsized":"2024-09-21T05:40:39.328175+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:05.675696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327167+0000","last_change":"2024-09-21T05:40:38.316750+0000","last_active":"2024-09-21T05:40:39.327167+0000","last_peered":"2024-09-21T05:40:39.327167+0000","last_clean":"2024-09-21T05:40:39.327167+0000","last_became_active":"2024-09-21T05:40:38.316421+0000","last_became_peered":"2024-09-21T05:40:38.316421+0000","last_unstale":"2024-09-21T05:40:39.327167+0000","last_undegraded":"2024-09-21T05:40:39.327167+0000","last_fullsized":"2024-09-21T05:40:39.327167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:30.981787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328579+0000","last_change":"2024-09-21T05:40:38.323973+0000","last_active":"2024-09-21T05:40:39.328579+0000","last_peered":"2024-09-21T05:40:39.328579+0000","last_clean":"2024-09-21T05:40:39.328579+0000","last_became_active":"2024-09-21T05:40:38.323679+0000","last_became_peered":"2024-09-21T05:40:38.323679+0000","last_unstale":"2024-09-21T05:40:39.328579+0000","last_undegraded":"2024-09-21T05:40:39.328579+0000","last_fullsized":"2024-09-21T05:40:39.328579+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:55.757724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468201+0000","last_change":"2024-09-21T05:40:38.332424+0000","last_active":"2024-09-21T05:40:39.468201+0000","last_peered":"2024-09-21T05:40:39.468201+0000","last_clean":"2024-09-21T05:40:39.468201+0000","last_became_active":"2024-09-21T05:40:38.332103+0000","last_became_peered":"2024-09-21T05:40:38.332103+0000","last_unstale":"2024-09-21T05:40:39.468201+0000","last_undegraded":"2024-09-21T05:40:39.468201+0000","last_fullsized":"2024-09-21T05:40:39.468201+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:10.099432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326895+0000","last_change":"2024-09-21T05:40:38.331197+0000","last_active":"2024-09-21T05:40:39.326895+0000","last_peered":"2024-09-21T05:40:39.326895+0000","last_clean":"2024-09-21T05:40:39.326895+0000","last_became_active":"2024-09-21T05:40:38.330810+0000","last_became_peered":"2024-09-21T05:40:38.330810+0000","last_unstale":"2024-09-21T05:40:39.326895+0000","last_undegraded":"2024-09-21T05:40:39.326895+0000","last_fullsized":"2024-09-21T05:40:39.326895+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:17:18.877290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290693+0000","last_change":"2024-09-21T05:40:38.324984+0000","last_active":"2024-09-21T05:40:40.290693+0000","last_peered":"2024-09-21T05:40:40.290693+0000","last_clean":"2024-09-21T05:40:40.290693+0000","last_became_active":"2024-09-21T05:40:38.324689+0000","last_became_peered":"2024-09-21T05:40:38.324689+0000","last_unstale":"2024-09-21T05:40:40.290693+0000","last_undegraded":"2024-09-21T05:40:40.290693+0000","last_fullsized":"2024-09-21T05:40:40.290693+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:01.118251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328403+0000","last_change":"2024-09-21T05:40:38.323067+0000","last_active":"2024-09-21T05:40:39.328403+0000","last_peered":"2024-09-21T05:40:39.328403+0000","last_clean":"2024-09-21T05:40:39.328403+0000","last_became_active":"2024-09-21T05:40:38.322712+0000","last_became_peered":"2024-09-21T05:40:38.322712+0000","last_unstale":"2024-09-21T05:40:39.328403+0000","last_undegraded":"2024-09-21T05:40:39.328403+0000","last_fullsized":"2024-09-21T05:40:39.328403+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:24.225101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326957+0000","last_change":"2024-09-21T05:40:38.331014+0000","last_active":"2024-09-21T05:40:39.326957+0000","last_peered":"2024-09-21T05:40:39.326957+0000","last_clean":"2024-09-21T05:40:39.326957+0000","last_became_active":"2024-09-21T05:40:38.329546+0000","last_became_peered":"2024-09-21T05:40:38.329546+0000","last_unstale":"2024-09-21T05:40:39.326957+0000","last_undegraded":"2024-09-21T05:40:39.326957+0000","last_fullsized":"2024-09-21T05:40:39.326957+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:09.940672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330741+0000","last_change":"2024-09-21T05:40:38.336671+0000","last_active":"2024-09-21T05:40:39.330741+0000","last_peered":"2024-09-21T05:40:39.330741+0000","last_clean":"2024-09-21T05:40:39.330741+0000","last_became_active":"2024-09-21T05:40:38.326405+0000","last_became_peered":"2024-09-21T05:40:38.326405+0000","last_unstale":"2024-09-21T05:40:39.330741+0000","last_undegraded":"2024-09-21T05:40:39.330741+0000","last_fullsized":"2024-09-21T05:40:39.330741+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:29.362091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468084+0000","last_change":"2024-09-21T05:40:38.321598+0000","last_active":"2024-09-21T05:40:39.468084+0000","last_peered":"2024-09-21T05:40:39.468084+0000","last_clean":"2024-09-21T05:40:39.468084+0000","last_became_active":"2024-09-21T05:40:38.321274+0000","last_became_peered":"2024-09-21T05:40:38.321274+0000","last_unstale":"2024-09-21T05:40:39.468084+0000","last_undegraded":"2024-09-21T05:40:39.468084+0000","last_fullsized":"2024-09-21T05:40:39.468084+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:28:26.990021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325740+0000","last_change":"2024-09-21T05:40:38.332561+0000","last_active":"2024-09-21T05:40:39.325740+0000","last_peered":"2024-09-21T05:40:39.325740+0000","last_clean":"2024-09-21T05:40:39.325740+0000","last_became_active":"2024-09-21T05:40:38.332234+0000","last_became_peered":"2024-09-21T05:40:38.332234+0000","last_unstale":"2024-09-21T05:40:39.325740+0000","last_undegraded":"2024-09-21T05:40:39.325740+0000","last_fullsized":"2024-09-21T05:40:39.325740+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:11.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331307+0000","last_change":"2024-09-21T05:40:37.318315+0000","last_active":"2024-09-21T05:40:39.331307+0000","last_peered":"2024-09-21T05:40:39.331307+0000","last_clean":"2024-09-21T05:40:39.331307+0000","last_became_active":"2024-09-21T05:40:37.317977+0000","last_became_peered":"2024-09-21T05:40:37.317977+0000","last_unstale":"2024-09-21T05:40:39.331307+0000","last_undegraded":"2024-09-21T05:40:39.331307+0000","last_fullsized":"2024-09-21T05:40:39.331307+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:00.112713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328839+0000","last_change":"2024-09-21T05:40:38.334024+0000","last_active":"2024-09-21T05:40:39.328839+0000","last_peered":"2024-09-21T05:40:39.328839+0000","last_clean":"2024-09-21T05:40:39.328839+0000","last_became_active":"2024-09-21T05:40:38.333692+0000","last_became_peered":"2024-09-21T05:40:38.333692+0000","last_unstale":"2024-09-21T05:40:39.328839+0000","last_undegraded":"2024-09-21T05:40:39.328839+0000","last_fullsized":"2024-09-21T05:40:39.328839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:24:33.272585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327752+0000","last_change":"2024-09-21T05:40:37.320643+0000","last_active":"2024-09-21T05:40:39.327752+0000","last_peered":"2024-09-21T05:40:39.327752+0000","last_clean":"2024-09-21T05:40:39.327752+0000","last_became_active":"2024-09-21T05:40:37.320265+0000","last_became_peered":"2024-09-21T05:40:37.320265+0000","last_unstale":"2024-09-21T05:40:39.327752+0000","last_undegraded":"2024-09-21T05:40:39.327752+0000","last_fullsized":"2024-09-21T05:40:39.327752+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:10.187416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290833+0000","last_change":"2024-09-21T05:40:38.339196+0000","last_active":"2024-09-21T05:40:40.290833+0000","last_peered":"2024-09-21T05:40:40.290833+0000","last_clean":"2024-09-21T05:40:40.290833+0000","last_became_active":"2024-09-21T05:40:38.338873+0000","last_became_peered":"2024-09-21T05:40:38.338873+0000","last_unstale":"2024-09-21T05:40:40.290833+0000","last_undegraded":"2024-09-21T05:40:40.290833+0000","last_fullsized":"2024-09-21T05:40:40.290833+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:21.924171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325838+0000","last_change":"2024-09-21T05:40:37.323969+0000","last_active":"2024-09-21T05:40:39.325838+0000","last_peered":"2024-09-21T05:40:39.325838+0000","last_clean":"2024-09-21T05:40:39.325838+0000","last_became_active":"2024-09-21T05:40:37.323482+0000","last_became_peered":"2024-09-21T05:40:37.323482+0000","last_unstale":"2024-09-21T05:40:39.325838+0000","last_undegraded":"2024-09-21T05:40:39.325838+0000","last_fullsized":"2024-09-21T05:40:39.325838+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:02.536168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325858+0000","last_change":"2024-09-21T05:40:38.320545+0000","last_active":"2024-09-21T05:40:39.325858+0000","last_peered":"2024-09-21T05:40:39.325858+0000","last_clean":"2024-09-21T05:40:39.325858+0000","last_became_active":"2024-09-21T05:40:38.320131+0000","last_became_peered":"2024-09-21T05:40:38.320131+0000","last_unstale":"2024-09-21T05:40:39.325858+0000","last_undegraded":"2024-09-21T05:40:39.325858+0000","last_fullsized":"2024-09-21T05:40:39.325858+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:08.515650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251830+0000","last_change":"2024-09-21T05:40:37.323806+0000","last_active":"2024-09-21T05:40:40.251830+0000","last_peered":"2024-09-21T05:40:40.251830+0000","last_clean":"2024-09-21T05:40:40.251830+0000","last_became_active":"2024-09-21T05:40:37.323294+0000","last_became_peered":"2024-09-21T05:40:37.323294+0000","last_unstale":"2024-09-21T05:40:40.251830+0000","last_undegraded":"2024-09-21T05:40:40.251830+0000","last_fullsized":"2024-09-21T05:40:40.251830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:27.835356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327207+0000","last_change":"2024-09-21T05:40:38.336978+0000","last_active":"2024-09-21T05:40:39.327207+0000","last_peered":"2024-09-21T05:40:39.327207+0000","last_clean":"2024-09-21T05:40:39.327207+0000","last_became_active":"2024-09-21T05:40:38.335623+0000","last_became_peered":"2024-09-21T05:40:38.335623+0000","last_unstale":"2024-09-21T05:40:39.327207+0000","last_undegraded":"2024-09-21T05:40:39.327207+0000","last_fullsized":"2024-09-21T05:40:39.327207+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:06.676042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330974+0000","last_change":"2024-09-21T05:40:37.322204+0000","last_active":"2024-09-21T05:40:39.330974+0000","last_peered":"2024-09-21T05:40:39.330974+0000","last_clean":"2024-09-21T05:40:39.330974+0000","last_became_active":"2024-09-21T05:40:37.321480+0000","last_became_peered":"2024-09-21T05:40:37.321480+0000","last_unstale":"2024-09-21T05:40:39.330974+0000","last_undegraded":"2024-09-21T05:40:39.330974+0000","last_fullsized":"2024-09-21T05:40:39.330974+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:49.906555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290739+0000","last_change":"2024-09-21T05:40:38.330049+0000","last_active":"2024-09-21T05:40:40.290739+0000","last_peered":"2024-09-21T05:40:40.290739+0000","last_clean":"2024-09-21T05:40:40.290739+0000","last_became_active":"2024-09-21T05:40:38.329736+0000","last_became_peered":"2024-09-21T05:40:38.329736+0000","last_unstale":"2024-09-21T05:40:40.290739+0000","last_undegraded":"2024-09-21T05:40:40.290739+0000","last_fullsized":"2024-09-21T05:40:40.290739+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:14.418415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327284+0000","last_change":"2024-09-21T05:40:37.322664+0000","last_active":"2024-09-21T05:40:39.327284+0000","last_peered":"2024-09-21T05:40:39.327284+0000","last_clean":"2024-09-21T05:40:39.327284+0000","last_became_active":"2024-09-21T05:40:37.322342+0000","last_became_peered":"2024-09-21T05:40:37.322342+0000","last_unstale":"2024-09-21T05:40:39.327284+0000","last_undegraded":"2024-09-21T05:40:39.327284+0000","last_fullsized":"2024-09-21T05:40:39.327284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:22.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290064+0000","last_change":"2024-09-21T05:40:38.337337+0000","last_active":"2024-09-21T05:40:40.290064+0000","last_peered":"2024-09-21T05:40:40.290064+0000","last_clean":"2024-09-21T05:40:40.290064+0000","last_became_active":"2024-09-21T05:40:38.337035+0000","last_became_peered":"2024-09-21T05:40:38.337035+0000","last_unstale":"2024-09-21T05:40:40.290064+0000","last_undegraded":"2024-09-21T05:40:40.290064+0000","last_fullsized":"2024-09-21T05:40:40.290064+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:12.226417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290335+0000","last_change":"2024-09-21T05:40:37.327298+0000","last_active":"2024-09-21T05:40:40.290335+0000","last_peered":"2024-09-21T05:40:40.290335+0000","last_clean":"2024-09-21T05:40:40.290335+0000","last_became_active":"2024-09-21T05:40:37.325933+0000","last_became_peered":"2024-09-21T05:40:37.325933+0000","last_unstale":"2024-09-21T05:40:40.290335+0000","last_undegraded":"2024-09-21T05:40:40.290335+0000","last_fullsized":"2024-09-21T05:40:40.290335+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:13.253336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327926+0000","last_change":"2024-09-21T05:40:38.316726+0000","last_active":"2024-09-21T05:40:39.327926+0000","last_peered":"2024-09-21T05:40:39.327926+0000","last_clean":"2024-09-21T05:40:39.327926+0000","last_became_active":"2024-09-21T05:40:38.316361+0000","last_became_peered":"2024-09-21T05:40:38.316361+0000","last_unstale":"2024-09-21T05:40:39.327926+0000","last_undegraded":"2024-09-21T05:40:39.327926+0000","last_fullsized":"2024-09-21T05:40:39.327926+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:15.982858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290688+0000","last_change":"2024-09-21T05:40:37.327740+0000","last_active":"2024-09-21T05:40:40.290688+0000","last_peered":"2024-09-21T05:40:40.290688+0000","last_clean":"2024-09-21T05:40:40.290688+0000","last_became_active":"2024-09-21T05:40:37.327132+0000","last_became_peered":"2024-09-21T05:40:37.327132+0000","last_unstale":"2024-09-21T05:40:40.290688+0000","last_undegraded":"2024-09-21T05:40:40.290688+0000","last_fullsized":"2024-09-21T05:40:40.290688+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:45:51.195745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327677+0000","last_change":"2024-09-21T05:40:38.329702+0000","last_active":"2024-09-21T05:40:39.327677+0000","last_peered":"2024-09-21T05:40:39.327677+0000","last_clean":"2024-09-21T05:40:39.327677+0000","last_became_active":"2024-09-21T05:40:38.329327+0000","last_became_peered":"2024-09-21T05:40:38.329327+0000","last_unstale":"2024-09-21T05:40:39.327677+0000","last_undegraded":"2024-09-21T05:40:39.327677+0000","last_fullsized":"2024-09-21T05:40:39.327677+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:12.644093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327123+0000","last_change":"2024-09-21T05:40:37.317918+0000","last_active":"2024-09-21T05:40:39.327123+0000","last_peered":"2024-09-21T05:40:39.327123+0000","last_clean":"2024-09-21T05:40:39.327123+0000","last_became_active":"2024-09-21T05:40:37.317592+0000","last_became_peered":"2024-09-21T05:40:37.317592+0000","last_unstale":"2024-09-21T05:40:39.327123+0000","last_undegraded":"2024-09-21T05:40:39.327123+0000","last_fullsized":"2024-09-21T05:40:39.327123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:57.233895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252291+0000","last_change":"2024-09-21T05:40:38.325511+0000","last_active":"2024-09-21T05:40:40.252291+0000","last_peered":"2024-09-21T05:40:40.252291+0000","last_clean":"2024-09-21T05:40:40.252291+0000","last_became_active":"2024-09-21T05:40:38.325183+0000","last_became_peered":"2024-09-21T05:40:38.325183+0000","last_unstale":"2024-09-21T05:40:40.252291+0000","last_undegraded":"2024-09-21T05:40:40.252291+0000","last_fullsized":"2024-09-21T05:40:40.252291+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:40.251382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290446+0000","last_change":"2024-09-21T05:40:37.330089+0000","last_active":"2024-09-21T05:40:40.290446+0000","last_peered":"2024-09-21T05:40:40.290446+0000","last_clean":"2024-09-21T05:40:40.290446+0000","last_became_active":"2024-09-21T05:40:37.329746+0000","last_became_peered":"2024-09-21T05:40:37.329746+0000","last_unstale":"2024-09-21T05:40:40.290446+0000","last_undegraded":"2024-09-21T05:40:40.290446+0000","last_fullsized":"2024-09-21T05:40:40.290446+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:53.134451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331305+0000","last_change":"2024-09-21T05:40:38.337196+0000","last_active":"2024-09-21T05:40:39.331305+0000","last_peered":"2024-09-21T05:40:39.331305+0000","last_clean":"2024-09-21T05:40:39.331305+0000","last_became_active":"2024-09-21T05:40:38.336624+0000","last_became_peered":"2024-09-21T05:40:38.336624+0000","last_unstale":"2024-09-21T05:40:39.331305+0000","last_undegraded":"2024-09-21T05:40:39.331305+0000","last_fullsized":"2024-09-21T05:40:39.331305+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:48.786835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":18,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252191+0000","last_change":"2024-09-21T05:40:37.322671+0000","last_active":"2024-09-21T05:40:40.252191+0000","last_peered":"2024-09-21T05:40:40.252191+0000","last_clean":"2024-09-21T05:40:40.252191+0000","last_became_active":"2024-09-21T05:40:37.322365+0000","last_became_peered":"2024-09-21T05:40:37.322365+0000","last_unstale":"2024-09-21T05:40:40.252191+0000","last_undegraded":"2024-09-21T05:40:40.252191+0000","last_fullsized":"2024-09-21T05:40:40.252191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:32.039926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252193+0000","last_change":"2024-09-21T05:40:38.325033+0000","last_active":"2024-09-21T05:40:40.252193+0000","last_peered":"2024-09-21T05:40:40.252193+0000","last_clean":"2024-09-21T05:40:40.252193+0000","last_became_active":"2024-09-21T05:40:38.324671+0000","last_became_peered":"2024-09-21T05:40:38.324671+0000","last_unstale":"2024-09-21T05:40:40.252193+0000","last_undegraded":"2024-09-21T05:40:40.252193+0000","last_fullsized":"2024-09-21T05:40:40.252193+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:30.214363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":21,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468092+0000","last_change":"2024-09-21T05:40:37.320854+0000","last_active":"2024-09-21T05:40:39.468092+0000","last_peered":"2024-09-21T05:40:39.468092+0000","last_clean":"2024-09-21T05:40:39.468092+0000","last_became_active":"2024-09-21T05:40:37.320330+0000","last_became_peered":"2024-09-21T05:40:37.320330+0000","last_unstale":"2024-09-21T05:40:39.468092+0000","last_undegraded":"2024-09-21T05:40:39.468092+0000","last_fullsized":"2024-09-21T05:40:39.468092+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:07.898462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327082+0000","last_change":"2024-09-21T05:40:38.330390+0000","last_active":"2024-09-21T05:40:39.327082+0000","last_peered":"2024-09-21T05:40:39.327082+0000","last_clean":"2024-09-21T05:40:39.327082+0000","last_became_active":"2024-09-21T05:40:38.330056+0000","last_became_peered":"2024-09-21T05:40:38.330056+0000","last_unstale":"2024-09-21T05:40:39.327082+0000","last_undegraded":"2024-09-21T05:40:39.327082+0000","last_fullsized":"2024-09-21T05:40:39.327082+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:00.304387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330980+0000","last_change":"2024-09-21T05:40:37.331195+0000","last_active":"2024-09-21T05:40:39.330980+0000","last_peered":"2024-09-21T05:40:39.330980+0000","last_clean":"2024-09-21T05:40:39.330980+0000","last_became_active":"2024-09-21T05:40:37.330827+0000","last_became_peered":"2024-09-21T05:40:37.330827+0000","last_unstale":"2024-09-21T05:40:39.330980+0000","last_undegraded":"2024-09-21T05:40:39.330980+0000","last_fullsized":"2024-09-21T05:40:39.330980+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:15:33.758138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290600+0000","last_change":"2024-09-21T05:40:38.332379+0000","last_active":"2024-09-21T05:40:40.290600+0000","last_peered":"2024-09-21T05:40:40.290600+0000","last_clean":"2024-09-21T05:40:40.290600+0000","last_became_active":"2024-09-21T05:40:38.332081+0000","last_became_peered":"2024-09-21T05:40:38.332081+0000","last_unstale":"2024-09-21T05:40:40.290600+0000","last_undegraded":"2024-09-21T05:40:40.290600+0000","last_fullsized":"2024-09-21T05:40:40.290600+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:00.804368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251948+0000","last_change":"2024-09-21T05:40:37.327625+0000","last_active":"2024-09-21T05:40:40.251948+0000","last_peered":"2024-09-21T05:40:40.251948+0000","last_clean":"2024-09-21T05:40:40.251948+0000","last_became_active":"2024-09-21T05:40:37.327293+0000","last_became_peered":"2024-09-21T05:40:37.327293+0000","last_unstale":"2024-09-21T05:40:40.251948+0000","last_undegraded":"2024-09-21T05:40:40.251948+0000","last_fullsized":"2024-09-21T05:40:40.251948+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:53.084749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330634+0000","last_change":"2024-09-21T05:40:38.317596+0000","last_active":"2024-09-21T05:40:39.330634+0000","last_peered":"2024-09-21T05:40:39.330634+0000","last_clean":"2024-09-21T05:40:39.330634+0000","last_became_active":"2024-09-21T05:40:38.317140+0000","last_became_peered":"2024-09-21T05:40:38.317140+0000","last_unstale":"2024-09-21T05:40:39.330634+0000","last_undegraded":"2024-09-21T05:40:39.330634+0000","last_fullsized":"2024-09-21T05:40:39.330634+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.974088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330805+0000","last_change":"2024-09-21T05:40:37.325179+0000","last_active":"2024-09-21T05:40:39.330805+0000","last_peered":"2024-09-21T05:40:39.330805+0000","last_clean":"2024-09-21T05:40:39.330805+0000","last_became_active":"2024-09-21T05:40:37.324826+0000","last_became_peered":"2024-09-21T05:40:37.324826+0000","last_unstale":"2024-09-21T05:40:39.330805+0000","last_undegraded":"2024-09-21T05:40:39.330805+0000","last_fullsized":"2024-09-21T05:40:39.330805+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:03.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252135+0000","last_change":"2024-09-21T05:40:38.317787+0000","last_active":"2024-09-21T05:40:40.252135+0000","last_peered":"2024-09-21T05:40:40.252135+0000","last_clean":"2024-09-21T05:40:40.252135+0000","last_became_active":"2024-09-21T05:40:38.317415+0000","last_became_peered":"2024-09-21T05:40:38.317415+0000","last_unstale":"2024-09-21T05:40:40.252135+0000","last_undegraded":"2024-09-21T05:40:40.252135+0000","last_fullsized":"2024-09-21T05:40:40.252135+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:29.746451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":23,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290475+0000","last_change":"2024-09-21T05:40:37.329583+0000","last_active":"2024-09-21T05:40:40.290475+0000","last_peered":"2024-09-21T05:40:40.290475+0000","last_clean":"2024-09-21T05:40:40.290475+0000","last_became_active":"2024-09-21T05:40:37.329140+0000","last_became_peered":"2024-09-21T05:40:37.329140+0000","last_unstale":"2024-09-21T05:40:40.290475+0000","last_undegraded":"2024-09-21T05:40:40.290475+0000","last_fullsized":"2024-09-21T05:40:40.290475+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:22.193696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468152+0000","last_change":"2024-09-21T05:40:38.329003+0000","last_active":"2024-09-21T05:40:39.468152+0000","last_peered":"2024-09-21T05:40:39.468152+0000","last_clean":"2024-09-21T05:40:39.468152+0000","last_became_active":"2024-09-21T05:40:38.328642+0000","last_became_peered":"2024-09-21T05:40:38.328642+0000","last_unstale":"2024-09-21T05:40:39.468152+0000","last_undegraded":"2024-09-21T05:40:39.468152+0000","last_fullsized":"2024-09-21T05:40:39.468152+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:34.100292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.468111+0000","last_change":"2024-09-21T05:40:37.318038+0000","last_active":"2024-09-21T05:40:39.468111+0000","last_peered":"2024-09-21T05:40:39.468111+0000","last_clean":"2024-09-21T05:40:39.468111+0000","last_became_active":"2024-09-21T05:40:37.317677+0000","last_became_peered":"2024-09-21T05:40:37.317677+0000","last_unstale":"2024-09-21T05:40:39.468111+0000","last_undegraded":"2024-09-21T05:40:39.468111+0000","last_fullsized":"2024-09-21T05:40:39.468111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:14.950529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290586+0000","last_change":"2024-09-21T05:40:38.339817+0000","last_active":"2024-09-21T05:40:40.290586+0000","last_peered":"2024-09-21T05:40:40.290586+0000","last_clean":"2024-09-21T05:40:40.290586+0000","last_became_active":"2024-09-21T05:40:38.339125+0000","last_became_peered":"2024-09-21T05:40:38.339125+0000","last_unstale":"2024-09-21T05:40:40.290586+0000","last_undegraded":"2024-09-21T05:40:40.290586+0000","last_fullsized":"2024-09-21T05:40:40.290586+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:41.823404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327636+0000","last_change":"2024-09-21T05:40:37.317152+0000","last_active":"2024-09-21T05:40:39.327636+0000","last_peered":"2024-09-21T05:40:39.327636+0000","last_clean":"2024-09-21T05:40:39.327636+0000","last_became_active":"2024-09-21T05:40:37.316833+0000","last_became_peered":"2024-09-21T05:40:37.316833+0000","last_unstale":"2024-09-21T05:40:39.327636+0000","last_undegraded":"2024-09-21T05:40:39.327636+0000","last_fullsized":"2024-09-21T05:40:39.327636+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:25.079643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328167+0000","last_change":"2024-09-21T05:40:38.338495+0000","last_active":"2024-09-21T05:40:39.328167+0000","last_peered":"2024-09-21T05:40:39.328167+0000","last_clean":"2024-09-21T05:40:39.328167+0000","last_became_active":"2024-09-21T05:40:38.338223+0000","last_became_peered":"2024-09-21T05:40:38.338223+0000","last_unstale":"2024-09-21T05:40:39.328167+0000","last_undegraded":"2024-09-21T05:40:39.328167+0000","last_fullsized":"2024-09-21T05:40:39.328167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:42.898821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290593+0000","last_change":"2024-09-21T05:40:37.326507+0000","last_active":"2024-09-21T05:40:40.290593+0000","last_peered":"2024-09-21T05:40:40.290593+0000","last_clean":"2024-09-21T05:40:40.290593+0000","last_became_active":"2024-09-21T05:40:37.326144+0000","last_became_peered":"2024-09-21T05:40:37.326144+0000","last_unstale":"2024-09-21T05:40:40.290593+0000","last_undegraded":"2024-09-21T05:40:40.290593+0000","last_fullsized":"2024-09-21T05:40:40.290593+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:01.535809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328810+0000","last_change":"2024-09-21T05:40:38.327930+0000","last_active":"2024-09-21T05:40:39.328810+0000","last_peered":"2024-09-21T05:40:39.328810+0000","last_clean":"2024-09-21T05:40:39.328810+0000","last_became_active":"2024-09-21T05:40:38.327461+0000","last_became_peered":"2024-09-21T05:40:38.327461+0000","last_unstale":"2024-09-21T05:40:39.328810+0000","last_undegraded":"2024-09-21T05:40:39.328810+0000","last_fullsized":"2024-09-21T05:40:39.328810+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:07:09.114800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331394+0000","last_change":"2024-09-21T05:40:37.319715+0000","last_active":"2024-09-21T05:40:39.331394+0000","last_peered":"2024-09-21T05:40:39.331394+0000","last_clean":"2024-09-21T05:40:39.331394+0000","last_became_active":"2024-09-21T05:40:37.319316+0000","last_became_peered":"2024-09-21T05:40:37.319316+0000","last_unstale":"2024-09-21T05:40:39.331394+0000","last_undegraded":"2024-09-21T05:40:39.331394+0000","last_fullsized":"2024-09-21T05:40:39.331394+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:39.247385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330866+0000","last_change":"2024-09-21T05:40:38.322534+0000","last_active":"2024-09-21T05:40:39.330866+0000","last_peered":"2024-09-21T05:40:39.330866+0000","last_clean":"2024-09-21T05:40:39.330866+0000","last_became_active":"2024-09-21T05:40:38.317035+0000","last_became_peered":"2024-09-21T05:40:38.317035+0000","last_unstale":"2024-09-21T05:40:39.330866+0000","last_undegraded":"2024-09-21T05:40:39.330866+0000","last_fullsized":"2024-09-21T05:40:39.330866+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:58.543686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331124+0000","last_change":"2024-09-21T05:40:37.321803+0000","last_active":"2024-09-21T05:40:39.331124+0000","last_peered":"2024-09-21T05:40:39.331124+0000","last_clean":"2024-09-21T05:40:39.331124+0000","last_became_active":"2024-09-21T05:40:37.321051+0000","last_became_peered":"2024-09-21T05:40:37.321051+0000","last_unstale":"2024-09-21T05:40:39.331124+0000","last_undegraded":"2024-09-21T05:40:39.331124+0000","last_fullsized":"2024-09-21T05:40:39.331124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:05.213045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325748+0000","last_change":"2024-09-21T05:40:38.325301+0000","last_active":"2024-09-21T05:40:39.325748+0000","last_peered":"2024-09-21T05:40:39.325748+0000","last_clean":"2024-09-21T05:40:39.325748+0000","last_became_active":"2024-09-21T05:40:38.324945+0000","last_became_peered":"2024-09-21T05:40:38.324945+0000","last_unstale":"2024-09-21T05:40:39.325748+0000","last_undegraded":"2024-09-21T05:40:39.325748+0000","last_fullsized":"2024-09-21T05:40:39.325748+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:52.729564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290292+0000","last_change":"2024-09-21T05:40:37.320338+0000","last_active":"2024-09-21T05:40:40.290292+0000","last_peered":"2024-09-21T05:40:40.290292+0000","last_clean":"2024-09-21T05:40:40.290292+0000","last_became_active":"2024-09-21T05:40:37.318592+0000","last_became_peered":"2024-09-21T05:40:37.318592+0000","last_unstale":"2024-09-21T05:40:40.290292+0000","last_undegraded":"2024-09-21T05:40:40.290292+0000","last_fullsized":"2024-09-21T05:40:40.290292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:29:47.171187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.467992+0000","last_change":"2024-09-21T05:40:38.318050+0000","last_active":"2024-09-21T05:40:39.467992+0000","last_peered":"2024-09-21T05:40:39.467992+0000","last_clean":"2024-09-21T05:40:39.467992+0000","last_became_active":"2024-09-21T05:40:38.317705+0000","last_became_peered":"2024-09-21T05:40:38.317705+0000","last_unstale":"2024-09-21T05:40:39.467992+0000","last_undegraded":"2024-09-21T05:40:39.467992+0000","last_fullsized":"2024-09-21T05:40:39.467992+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:03.088718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.289797+0000","last_change":"2024-09-21T05:40:37.331045+0000","last_active":"2024-09-21T05:40:40.289797+0000","last_peered":"2024-09-21T05:40:40.289797+0000","last_clean":"2024-09-21T05:40:40.289797+0000","last_became_active":"2024-09-21T05:40:37.330628+0000","last_became_peered":"2024-09-21T05:40:37.330628+0000","last_unstale":"2024-09-21T05:40:40.289797+0000","last_undegraded":"2024-09-21T05:40:40.289797+0000","last_fullsized":"2024-09-21T05:40:40.289797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:29.028421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290005+0000","last_change":"2024-09-21T05:40:38.329174+0000","last_active":"2024-09-21T05:40:40.290005+0000","last_peered":"2024-09-21T05:40:40.290005+0000","last_clean":"2024-09-21T05:40:40.290005+0000","last_became_active":"2024-09-21T05:40:38.328840+0000","last_became_peered":"2024-09-21T05:40:38.328840+0000","last_unstale":"2024-09-21T05:40:40.290005+0000","last_undegraded":"2024-09-21T05:40:40.290005+0000","last_fullsized":"2024-09-21T05:40:40.290005+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:27:59.294674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251610+0000","last_change":"2024-09-21T05:40:37.328411+0000","last_active":"2024-09-21T05:40:40.251610+0000","last_peered":"2024-09-21T05:40:40.251610+0000","last_clean":"2024-09-21T05:40:40.251610+0000","last_became_active":"2024-09-21T05:40:37.328077+0000","last_became_peered":"2024-09-21T05:40:37.328077+0000","last_unstale":"2024-09-21T05:40:40.251610+0000","last_undegraded":"2024-09-21T05:40:40.251610+0000","last_fullsized":"2024-09-21T05:40:40.251610+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:13:46.328645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.328921+0000","last_change":"2024-09-21T05:40:38.326657+0000","last_active":"2024-09-21T05:40:39.328921+0000","last_peered":"2024-09-21T05:40:39.328921+0000","last_clean":"2024-09-21T05:40:39.328921+0000","last_became_active":"2024-09-21T05:40:38.326314+0000","last_became_peered":"2024-09-21T05:40:38.326314+0000","last_unstale":"2024-09-21T05:40:39.328921+0000","last_undegraded":"2024-09-21T05:40:39.328921+0000","last_fullsized":"2024-09-21T05:40:39.328921+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:19.755767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":16,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.332556+0000","last_change":"2024-09-21T05:40:37.320540+0000","last_active":"2024-09-21T05:40:39.332556+0000","last_peered":"2024-09-21T05:40:39.332556+0000","last_clean":"2024-09-21T05:40:39.332556+0000","last_became_active":"2024-09-21T05:40:37.320187+0000","last_became_peered":"2024-09-21T05:40:37.320187+0000","last_unstale":"2024-09-21T05:40:39.332556+0000","last_undegraded":"2024-09-21T05:40:39.332556+0000","last_fullsized":"2024-09-21T05:40:39.332556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:29:24.131813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290567+0000","last_change":"2024-09-21T05:40:38.331333+0000","last_active":"2024-09-21T05:40:40.290567+0000","last_peered":"2024-09-21T05:40:40.290567+0000","last_clean":"2024-09-21T05:40:40.290567+0000","last_became_active":"2024-09-21T05:40:38.331007+0000","last_became_peered":"2024-09-21T05:40:38.331007+0000","last_unstale":"2024-09-21T05:40:40.290567+0000","last_undegraded":"2024-09-21T05:40:40.290567+0000","last_fullsized":"2024-09-21T05:40:40.290567+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:25.948769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252261+0000","last_change":"2024-09-21T05:40:37.324649+0000","last_active":"2024-09-21T05:40:40.252261+0000","last_peered":"2024-09-21T05:40:40.252261+0000","last_clean":"2024-09-21T05:40:40.252261+0000","last_became_active":"2024-09-21T05:40:37.324273+0000","last_became_peered":"2024-09-21T05:40:37.324273+0000","last_unstale":"2024-09-21T05:40:40.252261+0000","last_undegraded":"2024-09-21T05:40:40.252261+0000","last_fullsized":"2024-09-21T05:40:40.252261+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:01.363659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252083+0000","last_change":"2024-09-21T05:40:38.318225+0000","last_active":"2024-09-21T05:40:40.252083+0000","last_peered":"2024-09-21T05:40:40.252083+0000","last_clean":"2024-09-21T05:40:40.252083+0000","last_became_active":"2024-09-21T05:40:38.317904+0000","last_became_peered":"2024-09-21T05:40:38.317904+0000","last_unstale":"2024-09-21T05:40:40.252083+0000","last_undegraded":"2024-09-21T05:40:40.252083+0000","last_fullsized":"2024-09-21T05:40:40.252083+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:59.612292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325672+0000","last_change":"2024-09-21T05:40:37.323447+0000","last_active":"2024-09-21T05:40:39.325672+0000","last_peered":"2024-09-21T05:40:39.325672+0000","last_clean":"2024-09-21T05:40:39.325672+0000","last_became_active":"2024-09-21T05:40:37.323066+0000","last_became_peered":"2024-09-21T05:40:37.323066+0000","last_unstale":"2024-09-21T05:40:39.325672+0000","last_undegraded":"2024-09-21T05:40:39.325672+0000","last_fullsized":"2024-09-21T05:40:39.325672+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:45.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.331236+0000","last_change":"2024-09-21T05:40:38.330835+0000","last_active":"2024-09-21T05:40:39.331236+0000","last_peered":"2024-09-21T05:40:39.331236+0000","last_clean":"2024-09-21T05:40:39.331236+0000","last_became_active":"2024-09-21T05:40:38.330512+0000","last_became_peered":"2024-09-21T05:40:38.330512+0000","last_unstale":"2024-09-21T05:40:39.331236+0000","last_undegraded":"2024-09-21T05:40:39.331236+0000","last_fullsized":"2024-09-21T05:40:39.331236+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:50.874077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.252010+0000","last_change":"2024-09-21T05:40:37.321320+0000","last_active":"2024-09-21T05:40:40.252010+0000","last_peered":"2024-09-21T05:40:40.252010+0000","last_clean":"2024-09-21T05:40:40.252010+0000","last_became_active":"2024-09-21T05:40:37.320971+0000","last_became_peered":"2024-09-21T05:40:37.320971+0000","last_unstale":"2024-09-21T05:40:40.252010+0000","last_undegraded":"2024-09-21T05:40:40.252010+0000","last_fullsized":"2024-09-21T05:40:40.252010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:22.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.330819+0000","last_change":"2024-09-21T05:40:38.326032+0000","last_active":"2024-09-21T05:40:39.330819+0000","last_peered":"2024-09-21T05:40:39.330819+0000","last_clean":"2024-09-21T05:40:39.330819+0000","last_became_active":"2024-09-21T05:40:38.324730+0000","last_became_peered":"2024-09-21T05:40:38.324730+0000","last_unstale":"2024-09-21T05:40:39.330819+0000","last_undegraded":"2024-09-21T05:40:39.330819+0000","last_fullsized":"2024-09-21T05:40:39.330819+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:34.722618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.327440+0000","last_change":"2024-09-21T05:40:37.325747+0000","last_active":"2024-09-21T05:40:39.327440+0000","last_peered":"2024-09-21T05:40:39.327440+0000","last_clean":"2024-09-21T05:40:39.327440+0000","last_became_active":"2024-09-21T05:40:37.325232+0000","last_became_peered":"2024-09-21T05:40:37.325232+0000","last_unstale":"2024-09-21T05:40:39.327440+0000","last_undegraded":"2024-09-21T05:40:39.327440+0000","last_fullsized":"2024-09-21T05:40:39.327440+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:45.385547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290449+0000","last_change":"2024-09-21T05:40:38.323692+0000","last_active":"2024-09-21T05:40:40.290449+0000","last_peered":"2024-09-21T05:40:40.290449+0000","last_clean":"2024-09-21T05:40:40.290449+0000","last_became_active":"2024-09-21T05:40:38.323389+0000","last_became_peered":"2024-09-21T05:40:38.323389+0000","last_unstale":"2024-09-21T05:40:40.290449+0000","last_undegraded":"2024-09-21T05:40:40.290449+0000","last_fullsized":"2024-09-21T05:40:40.290449+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:30:40.252922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.325496+0000","last_change":"2024-09-21T05:40:37.319349+0000","last_active":"2024-09-21T05:40:39.325496+0000","last_peered":"2024-09-21T05:40:39.325496+0000","last_clean":"2024-09-21T05:40:39.325496+0000","last_became_active":"2024-09-21T05:40:37.318946+0000","last_became_peered":"2024-09-21T05:40:37.318946+0000","last_unstale":"2024-09-21T05:40:39.325496+0000","last_undegraded":"2024-09-21T05:40:39.325496+0000","last_fullsized":"2024-09-21T05:40:39.325496+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:22.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:39.326522+0000","last_change":"2024-09-21T05:40:38.340366+0000","last_active":"2024-09-21T05:40:39.326522+0000","last_peered":"2024-09-21T05:40:39.326522+0000","last_clean":"2024-09-21T05:40:39.326522+0000","last_became_active":"2024-09-21T05:40:38.340027+0000","last_became_peered":"2024-09-21T05:40:38.340027+0000","last_unstale":"2024-09-21T05:40:39.326522+0000","last_undegraded":"2024-09-21T05:40:39.326522+0000","last_fullsized":"2024-09-21T05:40:39.326522+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:23.812435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290620+0000","last_change":"2024-09-21T05:40:37.330151+0000","last_active":"2024-09-21T05:40:40.290620+0000","last_peered":"2024-09-21T05:40:40.290620+0000","last_clean":"2024-09-21T05:40:40.290620+0000","last_became_active":"2024-09-21T05:40:37.329765+0000","last_became_peered":"2024-09-21T05:40:37.329765+0000","last_unstale":"2024-09-21T05:40:40.290620+0000","last_undegraded":"2024-09-21T05:40:40.290620+0000","last_fullsized":"2024-09-21T05:40:40.290620+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:53.274256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290273+0000","last_change":"2024-09-21T05:40:38.325562+0000","last_active":"2024-09-21T05:40:40.290273+0000","last_peered":"2024-09-21T05:40:40.290273+0000","last_clean":"2024-09-21T05:40:40.290273+0000","last_became_active":"2024-09-21T05:40:38.325252+0000","last_became_peered":"2024-09-21T05:40:38.325252+0000","last_unstale":"2024-09-21T05:40:40.290273+0000","last_undegraded":"2024-09-21T05:40:40.290273+0000","last_fullsized":"2024-09-21T05:40:40.290273+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:37.297163+0000","last_clean_scrub_stamp":"2024-09-21T05:40:37.297163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:19.290645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.290743+0000","last_change":"2024-09-21T05:40:37.325102+0000","last_active":"2024-09-21T05:40:40.290743+0000","last_peered":"2024-09-21T05:40:40.290743+0000","last_clean":"2024-09-21T05:40:40.290743+0000","last_became_active":"2024-09-21T05:40:37.324744+0000","last_became_peered":"2024-09-21T05:40:37.324744+0000","last_unstale":"2024-09-21T05:40:40.290743+0000","last_undegraded":"2024-09-21T05:40:40.290743+0000","last_fullsized":"2024-09-21T05:40:40.290743+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:40:36.291626+0000","last_clean_scrub_stamp":"2024-09-21T05:40:36.291626+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:18.518420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":148,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-21T05:40:40.251633+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:40:40.251633+0000","last_peered":"2024-09-21T05:40:40.251633+0000","last_clean":"2024-09-21T05:40:40.251633+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:40:40.251633+0000","last_undegraded":"2024-09-21T05:40:40.251633+0000","last_fullsized":"2024-09-21T05:40:40.251633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1093632,"data_stored":790371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":507904,"data_stored":201525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":503808,"data_stored":200585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":487424,"data_stored":193683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542164,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":507904,"data_stored":200551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":201547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35836,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336004,"statfs":{"total":96636764160,"available":96600068096,"internally_reserved":0,"allocated":499712,"data_stored":199983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36404,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335436,"statfs":{"total":96636764160,"available":96599486464,"internally_reserved":0,"allocated":1081344,"data_stored":784015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:40:42.771 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:40:42.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:40:42.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 -- 172.21.15.22:0/4266807200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 msgr2=0x7f2e2015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 --2- 172.21.15.22:0/4266807200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2015c7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005fd0 tx=0x7f2e1002a820 comp rx=0 tx=0).stop 2024-09-21T05:40:42.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 -- 172.21.15.22:0/4266807200 shutdown_connections 2024-09-21T05:40:42.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 --2- 172.21.15.22:0/4266807200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2015c7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 -- 172.21.15.22:0/4266807200 >> 172.21.15.22:0/4266807200 conn(0x7f2e200b3f10 msgr2=0x7f2e200b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:42.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 -- 172.21.15.22:0/4266807200 shutdown_connections 2024-09-21T05:40:42.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.930+0000 7f2e256de640 1 -- 172.21.15.22:0/4266807200 wait complete. 2024-09-21T05:40:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.932+0000 7f2e256de640 1 Processor -- start 2024-09-21T05:40:42.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.932+0000 7f2e256de640 1 -- start start 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e256de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e200b5210 0x7f2e200b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e256de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e256de640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e200d2310 0x7f2e2014a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e256de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200d19c0 con 0x7f2e200ccbe0 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e256de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2e200d1840 con 0x7f2e200b5210 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e256de640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2e200d1b40 con 0x7f2e200d2310 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e1e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e1e7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2014a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59686/0 (socket says 172.21.15.22:59686) 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e1e7fc640 1 -- 172.21.15.22:0/1741287811 learned_addr learned my addr 172.21.15.22:0/1741287811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:42.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e1effd640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e200b5210 0x7f2e200b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.933+0000 7f2e1f7fe640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e200d2310 0x7f2e2014a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e1e7fc640 1 -- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e200d2310 msgr2=0x7f2e2014a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e1e7fc640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e200d2310 0x7f2e2014a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e1e7fc640 1 -- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e200b5210 msgr2=0x7f2e200b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e1e7fc640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e200b5210 0x7f2e200b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e1e7fc640 1 -- 172.21.15.22:0/1741287811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e201f3ae0 con 0x7f2e200ccbe0 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e1e7fc640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2014a250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800c3c0 tx=0x7f2e0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08017070 con 0x7f2e200ccbe0 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e080035f0 con 0x7f2e200ccbe0 2024-09-21T05:40:42.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08003a80 con 0x7f2e200ccbe0 2024-09-21T05:40:42.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e2014aff0 con 0x7f2e200ccbe0 2024-09-21T05:40:42.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.934+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e2014b5b0 con 0x7f2e200ccbe0 2024-09-21T05:40:42.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.935+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e2014ad40 con 0x7f2e200ccbe0 2024-09-21T05:40:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.936+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2e0801a070 con 0x7f2e200ccbe0 2024-09-21T05:40:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.936+0000 7f2e177fe640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2e00078b90 0x7f2e0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.936+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f2e08099f30 con 0x7f2e200ccbe0 2024-09-21T05:40:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.936+0000 7f2e1effd640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2e00078b90 0x7f2e0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:42.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.937+0000 7f2e1effd640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2e00078b90 0x7f2e0007b0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e10034000 tx=0x7f2e10004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:42.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:42.941+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2e201f3ae0 con 0x7f2e200ccbe0 2024-09-21T05:40:43.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.211+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2e2015c7a0 con 0x7f2e200ccbe0 2024-09-21T05:40:43.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.214+0000 7f2e177fe640 1 -- 172.21.15.22:0/1741287811 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f2e08062490 con 0x7f2e200ccbe0 2024-09-21T05:40:43.216 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:40:43.216 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:40:43.216 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:40:43.216 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:40:43.216 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:40:43.217 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2e00078b90 msgr2=0x7f2e0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2e00078b90 0x7f2e0007b0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2e10034000 tx=0x7f2e10004ab0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 msgr2=0x7f2e2014a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2014a250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800c3c0 tx=0x7f2e0800c880 comp rx=0 tx=0).stop 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 shutdown_connections 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2e00078b90 0x7f2e0007b0a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2e200d2310 0x7f2e2014a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2e200ccbe0 0x7f2e2014a250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 --2- 172.21.15.22:0/1741287811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2e200b5210 0x7f2e200b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 >> 172.21.15.22:0/1741287811 conn(0x7f2e200b3f10 msgr2=0x7f2e200b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 shutdown_connections 2024-09-21T05:40:43.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.216+0000 7f2e256de640 1 -- 172.21.15.22:0/1741287811 wait complete. 2024-09-21T05:40:43.230 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:43.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1552205699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 msgr2=0x7fcdd00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1552205699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00ccbc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005fd0 tx=0x7fcdc002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1552205699 shutdown_connections 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1552205699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdd00cf320 0x7fcdd00d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1552205699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00ccbc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1552205699 >> 172.21.15.22:0/1552205699 conn(0x7fcdd006d4f0 msgr2=0x7fcdd0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1552205699 shutdown_connections 2024-09-21T05:40:43.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.345+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1552205699 wait complete. 2024-09-21T05:40:43.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.347+0000 7fcdd4b57640 1 Processor -- start 2024-09-21T05:40:43.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.347+0000 7fcdd4b57640 1 -- start start 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdd4b57640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd00b7cc0 0x7fcdd01f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdd4b57640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdd00cf320 0x7fcdd00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdd4b57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdd4b57640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd00d1410 con 0x7fcdd01600b0 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdd4b57640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd00d1290 con 0x7fcdd00b7cc0 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdd4b57640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd00d1590 con 0x7fcdd00cf320 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:43.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00b09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59696/0 (socket says 172.21.15.22:59696) 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 -- 172.21.15.22:0/1658350950 learned_addr learned my addr 172.21.15.22:0/1658350950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdce59c640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd00b7cc0 0x7fcdd01f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdcdd9b640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdd00cf320 0x7fcdd00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 -- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdd00cf320 msgr2=0x7fcdd00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdd00cf320 0x7fcdd00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 -- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd00b7cc0 msgr2=0x7fcdd01f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd00b7cc0 0x7fcdd01f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.349+0000 7fcdced9d640 1 -- 172.21.15.22:0/1658350950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd01f6b00 con 0x7fcdd01600b0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.350+0000 7fcdced9d640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00b09d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800d9a0 tx=0x7fcdc800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.350+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8017070 con 0x7fcdd01600b0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.350+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdc800be00 con 0x7fcdd01600b0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.350+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8003bd0 con 0x7fcdd01600b0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.350+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd015b340 con 0x7fcdd01600b0 2024-09-21T05:40:43.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.350+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd006f060 con 0x7fcdd01600b0 2024-09-21T05:40:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.351+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd006eaf0 con 0x7fcdd01600b0 2024-09-21T05:40:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.352+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcdc8010460 con 0x7fcdd01600b0 2024-09-21T05:40:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.352+0000 7fcdbeffd640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcdb0078b90 0x7fcdb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.352+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fcdc809b0b0 con 0x7fcdd01600b0 2024-09-21T05:40:43.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.352+0000 7fcdce59c640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcdb0078b90 0x7fcdb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:43.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.353+0000 7fcdce59c640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcdb0078b90 0x7fcdb007b0a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0033000 tx=0x7fcdc0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:43.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.357+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcdd01f6b00 con 0x7fcdd01600b0 2024-09-21T05:40:43.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.580+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdd00d1b20 con 0x7fcdd01600b0 2024-09-21T05:40:43.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.580+0000 7fcdbeffd640 1 -- 172.21.15.22:0/1658350950 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fcdc8062ea0 con 0x7fcdd01600b0 2024-09-21T05:40:43.582 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:43.582 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:40:43.583 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:40:43.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcdb0078b90 msgr2=0x7fcdb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcdb0078b90 0x7fcdb007b0a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0033000 tx=0x7fcdc0004730 comp rx=0 tx=0).stop 2024-09-21T05:40:43.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 msgr2=0x7fcdd00b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:43.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00b09d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcdc800d9a0 tx=0x7fcdc800de60 comp rx=0 tx=0).stop 2024-09-21T05:40:43.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 shutdown_connections 2024-09-21T05:40:43.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdd01600b0 0x7fcdd00b09d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcdb0078b90 0x7fcdb007b0a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdd00cf320 0x7fcdd00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 --2- 172.21.15.22:0/1658350950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcdd00b7cc0 0x7fcdd01f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:43.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 >> 172.21.15.22:0/1658350950 conn(0x7fcdd006d4f0 msgr2=0x7fcdd015f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:43.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 shutdown_connections 2024-09-21T05:40:43.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:43.583+0000 7fcdd4b57640 1 -- 172.21.15.22:0/1658350950 wait complete. 2024-09-21T05:40:43.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:40:37.702239+0000', 'modified': '2024-09-21T05:40:40.714396+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5298}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5298': {'gid': 5298, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6837/441099152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6835', 'nonce': 441099152}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 441099152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5298, 'qdb_cluster': [5298]} 2024-09-21T05:40:43.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:40:43.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.118+0000 7fddd59ba640 1 -- 172.21.15.22:0/707583298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0155ce0 msgr2=0x7fddd00708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.118+0000 7fddd59ba640 1 --2- 172.21.15.22:0/707583298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0155ce0 0x7fddd00708b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fddd0070e90 tx=0x7fddc802aa30 comp rx=0 tx=0).stop 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 -- 172.21.15.22:0/707583298 shutdown_connections 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 --2- 172.21.15.22:0/707583298 >> v2:172.21.15.134:3300/0 conn(0x7fddd015c580 0x7fddd0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 --2- 172.21.15.22:0/707583298 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0155ce0 0x7fddd00708b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 --2- 172.21.15.22:0/707583298 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd0074420 0x7fddd006fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 -- 172.21.15.22:0/707583298 >> 172.21.15.22:0/707583298 conn(0x7fddd006def0 msgr2=0x7fddd006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 -- 172.21.15.22:0/707583298 shutdown_connections 2024-09-21T05:40:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.119+0000 7fddd59ba640 1 -- 172.21.15.22:0/707583298 wait complete. 2024-09-21T05:40:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.120+0000 7fddd59ba640 1 Processor -- start 2024-09-21T05:40:44.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.120+0000 7fddd59ba640 1 -- start start 2024-09-21T05:40:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddd59ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd0074420 0x7fddd013ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddd59ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0155ce0 0x7fddd0151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddd59ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 0x7fddd01515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddd59ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fddd015f9b0 con 0x7fddd0150070 2024-09-21T05:40:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddd59ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fddd015f830 con 0x7fddd0155ce0 2024-09-21T05:40:44.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddd59ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fddd015fb30 con 0x7fddd0074420 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 0x7fddd01515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 0x7fddd01515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59710/0 (socket says 172.21.15.22:59710) 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 -- 172.21.15.22:0/90720507 learned_addr learned my addr 172.21.15.22:0/90720507 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddc7fff640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0155ce0 0x7fddd0151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddceffd640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd0074420 0x7fddd013ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 -- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd0074420 msgr2=0x7fddd013ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd0074420 0x7fddd013ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 -- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0155ce0 msgr2=0x7fddd0151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0155ce0 0x7fddd0151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 -- 172.21.15.22:0/90720507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd013f120 con 0x7fddd0150070 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.122+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 0x7fddd01515a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddc802b020 tx=0x7fddc8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.123+0000 7fddccff9640 1 -- 172.21.15.22:0/90720507 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8033050 con 0x7fddd0150070 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.123+0000 7fddccff9640 1 -- 172.21.15.22:0/90720507 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddc8004550 con 0x7fddd0150070 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.123+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd00c65d0 con 0x7fddd0150070 2024-09-21T05:40:44.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.123+0000 7fddccff9640 1 -- 172.21.15.22:0/90720507 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8038450 con 0x7fddd0150070 2024-09-21T05:40:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.123+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd00c6b90 con 0x7fddd0150070 2024-09-21T05:40:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.125+0000 7fddccff9640 1 -- 172.21.15.22:0/90720507 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fddc8036070 con 0x7fddd0150070 2024-09-21T05:40:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.125+0000 7fddccff9640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddb0078b90 0x7fddb007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.125+0000 7fddccff9640 1 -- 172.21.15.22:0/90720507 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fddc80bc1a0 con 0x7fddd0150070 2024-09-21T05:40:44.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.125+0000 7fddceffd640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddb0078b90 0x7fddb007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.126+0000 7fdddca01740 1 Processor -- start 2024-09-21T05:40:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.126+0000 7fdddca01740 1 -- start start 2024-09-21T05:40:44.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.126+0000 7fddceffd640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddb0078b90 0x7fddb007b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fddc000a4d0 tx=0x7fddc000a460 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fdddca01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fdddca01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2aa3e9820 con 0x55b2aa0a63b0 2024-09-21T05:40:44.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2aa3d8c50 con 0x55b2aa0ae590 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b2aa3dca60 con 0x55b2aa0cf270 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fddc7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fddc7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59718/0 (socket says 172.21.15.22:59718) 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.127+0000 7fddc7fff640 1 -- 172.21.15.22:0/2608738743 learned_addr learned my addr 172.21.15.22:0/2608738743 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddceffd640 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc7fff640 1 -- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 msgr2=0x55b2aa4a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc7fff640 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc7fff640 1 -- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 msgr2=0x55b2aa4be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc7fff640 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc7fff640 1 -- 172.21.15.22:0/2608738743 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2aa13a850 con 0x55b2aa0a63b0 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc7fff640 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4a14e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fddb800ab10 tx=0x7fddb800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae09538d172d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc57fa640 1 -- 172.21.15.22:0/2608738743 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8005020 con 0x55b2aa0a63b0 2024-09-21T05:40:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc57fa640 1 -- 172.21.15.22:0/2608738743 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb80052b0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.128+0000 7fddc57fa640 1 -- 172.21.15.22:0/2608738743 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80047a0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 -- 172.21.15.22:0/2608738743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 msgr2=0x55b2aa4a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4a14e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fddb800ab10 tx=0x7fddb800f3c0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 -- 172.21.15.22:0/2608738743 shutdown_connections 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4a14e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 --2- 172.21.15.22:0/2608738743 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 -- 172.21.15.22:0/2608738743 >> 172.21.15.22:0/2608738743 conn(0x55b2aa0c33c0 msgr2=0x55b2aa477260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 -- 172.21.15.22:0/2608738743 shutdown_connections 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.129+0000 7fdddca01740 1 -- 172.21.15.22:0/2608738743 wait complete. 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.130+0000 7fdddca01740 1 -- start start 2024-09-21T05:40:44.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.130+0000 7fdddca01740 1 Processor -- start 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa48f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2aa149620 con 0x55b2aa0a63b0 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2aa37e5f0 con 0x55b2aa0ae590 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b2aa4a1a20 con 0x55b2aa0cf270 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fdddca01740 10 client.0 _ll_register_callbacks cb 0x55b2aa3820a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fddceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa48f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fddceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa48f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59734/0 (socket says 172.21.15.22:59734) 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.131+0000 7fddceffd640 1 -- 172.21.15.22:0/344381251 learned_addr learned my addr 172.21.15.22:0/344381251 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddc7fff640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddceffd640 1 -- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 msgr2=0x55b2aa4caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddceffd640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddceffd640 1 -- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 msgr2=0x55b2aa4cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddceffd640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddceffd640 1 -- 172.21.15.22:0/344381251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2aa1b16b0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddceffd640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa48f790 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fddc001daf0 tx=0x7fddc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2fd3162adb08b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddae7fc640 1 -- 172.21.15.22:0/344381251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0016070 con 0x55b2aa0a63b0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddae7fc640 1 -- 172.21.15.22:0/344381251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fddc0028ab0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb80052b0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddae7fc640 1 -- 172.21.15.22:0/344381251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fddc0028cb0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fdddca01740 10 client.5424 fetch_fsmap learned FSMap version 26 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55b2aa4ceab0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddae7fc640 1 -- 172.21.15.22:0/344381251 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb80052b0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.132+0000 7fddae7fc640 1 -- 172.21.15.22:0/344381251 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fddc002f050 con 0x55b2aa0a63b0 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fdddca01740 10 client.5424 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fdddca01740 10 client.5424 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fdddca01740 20 client.5424 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fdddca01740 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa08de40 0x55b2aa4b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fdddca01740 4 client.5424 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2aa3eb770 con 0x55b2aa08de40 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fddc7fff640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa08de40 0x55b2aa4b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fddc7fff640 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa08de40 0x55b2aa4b2b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41de18f1b0f81478 server_cookie=dad54bec678568d2 in_seq=0 out_seq=0 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.133+0000 7fddae7fc640 10 client.5424 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:44.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.134+0000 7fddae7fc640 1 -- 172.21.15.22:0/344381251 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b2aa3eb770 con 0x55b2aa08de40 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.134+0000 7fddae7fc640 10 client.5424 handle_command_reply: tid=0 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.134+0000 7fdddca01740 1 client.5424 shutdown 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.134+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 msgr2=0x55b2aa48f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa48f790 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fddc001daf0 tx=0x7fddc0004730 comp rx=0 tx=0).stop 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 shutdown_connections 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa08de40 0x55b2aa4b2b90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fddae7fc640 0 client.5424 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 --2- 172.21.15.22:0/344381251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0a63b0 0x55b2aa48f790 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 >> 172.21.15.22:0/344381251 conn(0x55b2aa0c33c0 msgr2=0x55b2aa490760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 shutdown_connections 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 1 -- 172.21.15.22:0/344381251 wait complete. 2024-09-21T05:40:44.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.135+0000 7fdddca01740 20 client.5424 trim_cache size 0 max 16384 2024-09-21T05:40:44.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.153+0000 7fdddca01740 1 Processor -- start 2024-09-21T05:40:44.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.153+0000 7fdddca01740 1 -- start start 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fdddca01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 0x55b2aa4b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fdddca01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0201030 con 0x55b2aa0ae590 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01f8fc0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fddd02009a0 con 0x55b2aa0cf270 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 0x55b2aa4b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 0x55b2aa4b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59738/0 (socket says 172.21.15.22:59738) 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 -- 172.21.15.22:0/3979217722 learned_addr learned my addr 172.21.15.22:0/3979217722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddceffd640 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4bfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 -- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 msgr2=0x55b2aa4906f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4906f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 -- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0a63b0 msgr2=0x55b2aa4bfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4bfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 -- 172.21.15.22:0/3979217722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2aa49ea30 con 0x55b2aa0ae590 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.155+0000 7fddc7fff640 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 0x55b2aa4b1f10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fddb800afd0 tx=0x7fddb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce3f5ad009df0e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fddc4ff9640 1 -- 172.21.15.22:0/3979217722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8031070 con 0x55b2aa0ae590 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fddc4ff9640 1 -- 172.21.15.22:0/3979217722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb80052b0 con 0x55b2aa0ae590 2024-09-21T05:40:44.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fddc4ff9640 1 -- 172.21.15.22:0/3979217722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb80054e0 con 0x55b2aa0ae590 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 -- 172.21.15.22:0/3979217722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 msgr2=0x55b2aa4b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 0x55b2aa4b1f10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fddb800afd0 tx=0x7fddb8004730 comp rx=0 tx=0).stop 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 -- 172.21.15.22:0/3979217722 shutdown_connections 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0cf270 0x55b2aa4906f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0ae590 0x55b2aa4b1f10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 --2- 172.21.15.22:0/3979217722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0a63b0 0x55b2aa4bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 -- 172.21.15.22:0/3979217722 >> 172.21.15.22:0/3979217722 conn(0x55b2aa0c33c0 msgr2=0x55b2aa472ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 -- 172.21.15.22:0/3979217722 shutdown_connections 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.156+0000 7fdddca01740 1 -- 172.21.15.22:0/3979217722 wait complete. 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.157+0000 7fdddca01740 1 -- start start 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.157+0000 7fdddca01740 1 Processor -- start 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0a63b0 0x55b2aa538cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 0x55b2aa53ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2aa4cb5c0 con 0x55b2aa0cf270 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2aa4ccaf0 con 0x55b2aa0ae590 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b2aa4cc8e0 con 0x55b2aa0a63b0 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 10 client.0 _ll_register_callbacks cb 0x7fddc000a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 0x55b2aa53ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 0x55b2aa53ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59752/0 (socket says 172.21.15.22:59752) 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 -- 172.21.15.22:0/1822638726 learned_addr learned my addr 172.21.15.22:0/1822638726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddceffd640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0a63b0 0x55b2aa538cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddc7fff640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 -- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0a63b0 msgr2=0x55b2aa538cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0a63b0 0x55b2aa538cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 -- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 msgr2=0x55b2aa4a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 -- 172.21.15.22:0/1822638726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2aa49e380 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddceffd640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0a63b0 0x55b2aa538cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddcf7fe640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 0x55b2aa53ee80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fddc8083fb0 tx=0x7fddc8083ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9864b18d9424bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddae7fc640 1 -- 172.21.15.22:0/1822638726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc8084020 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddae7fc640 1 -- 172.21.15.22:0/1822638726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fddc80aa050 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb80052b0 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.159+0000 7fddae7fc640 1 -- 172.21.15.22:0/1822638726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fddc8088a50 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 10 client.5430 fetch_fsmap learned FSMap version 26 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55b2aa5392c0 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fddae7fc640 1 -- 172.21.15.22:0/1822638726 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddb80052b0 con 0x55b2aa0cf270 2024-09-21T05:40:44.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fddae7fc640 1 -- 172.21.15.22:0/1822638726 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fddc8040070 con 0x55b2aa0cf270 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 10 client.5430 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 10 client.5430 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 20 client.5430 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa49a610 0x55b2aa49aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 4 client.5430 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "status"}) -- 0x55b2aa3e7220 con 0x55b2aa49a610 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fddceffd640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa49a610 0x55b2aa49aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fddceffd640 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa49a610 0x55b2aa49aa20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dfcd73ded3d4f1c server_cookie=cd01f1fb4166c66 in_seq=0 out_seq=0 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.160+0000 7fddae7fc640 10 client.5430 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fddae7fc640 1 -- 172.21.15.22:0/1822638726 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b2aa3e7220 con 0x55b2aa49a610 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fddae7fc640 10 client.5430 handle_command_reply: tid=0 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 client.5430 shutdown 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 msgr2=0x55b2aa53ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 0x55b2aa53ee80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fddc8083fb0 tx=0x7fddc8083ec0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 shutdown_connections 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55b2aa49a610 0x55b2aa49aa20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2aa0cf270 0x55b2aa53ee80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2aa0ae590 0x55b2aa4a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fddae7fc640 0 client.5430 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 --2- 172.21.15.22:0/1822638726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2aa0a63b0 0x55b2aa538cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 >> 172.21.15.22:0/1822638726 conn(0x55b2aa0c33c0 msgr2=0x55b2aa473070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 shutdown_connections 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 1 -- 172.21.15.22:0/1822638726 wait complete. 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.161+0000 7fdddca01740 20 client.5430 trim_cache size 0 max 16384 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:40:44.170 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5298, 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.8416058890000002, 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 26, 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 43, 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 43, 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 10.483760622 2024-09-21T05:40:44.171 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddb0078b90 msgr2=0x7fddb007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddb0078b90 0x7fddb007b0a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fddc000a4d0 tx=0x7fddc000a460 comp rx=0 tx=0).stop 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 msgr2=0x7fddd01515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 0x7fddd01515a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddc802b020 tx=0x7fddc8003f10 comp rx=0 tx=0).stop 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 shutdown_connections 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fddd0150070 0x7fddd01515a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fddd0155ce0 0x7fddd0151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddb0078b90 0x7fddb007b0a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 --2- 172.21.15.22:0/90720507 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fddd0074420 0x7fddd013ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 >> 172.21.15.22:0/90720507 conn(0x7fddd006def0 msgr2=0x7fddd00d11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 shutdown_connections 2024-09-21T05:40:44.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.162+0000 7fddd59ba640 1 -- 172.21.15.22:0/90720507 wait complete. 2024-09-21T05:40:44.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:40:44.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/614192021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880b3540 msgr2=0x7f9b880d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/614192021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880b3540 0x7f9b880d3330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9b70005fd0 tx=0x7f9b7002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/614192021 shutdown_connections 2024-09-21T05:40:44.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/614192021 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880b3540 0x7f9b880d3330 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/614192021 >> 172.21.15.22:0/614192021 conn(0x7f9b8808bfc0 msgr2=0x7f9b8809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/614192021 shutdown_connections 2024-09-21T05:40:44.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.289+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/614192021 wait complete. 2024-09-21T05:40:44.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.291+0000 7f9b8f0a9640 1 Processor -- start 2024-09-21T05:40:44.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.291+0000 7f9b8f0a9640 1 -- start start 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b8f0a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880b3540 0x7f9b881590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b8f0a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 0x7f9b880cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b8f0a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880ccf20 0x7f9b880c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b8f0a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9b880d1070 con 0x7f9b880d3870 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b8f0a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9b880d0ef0 con 0x7f9b880ccf20 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b8f0a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9b880d11f0 con 0x7f9b880b3540 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 0x7f9b880cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b87fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 0x7f9b880cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41982/0 (socket says 172.21.15.22:41982) 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.292+0000 7f9b87fff640 1 -- 172.21.15.22:0/775822269 learned_addr learned my addr 172.21.15.22:0/775822269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b8ce45640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880b3540 0x7f9b881590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b87fff640 1 -- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880b3540 msgr2=0x7f9b881590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b87fff640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880b3540 0x7f9b881590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b8d646640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880ccf20 0x7f9b880c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b87fff640 1 -- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880ccf20 msgr2=0x7f9b880c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b87fff640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880ccf20 0x7f9b880c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b87fff640 1 -- 172.21.15.22:0/775822269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b880c8130 con 0x7f9b880d3870 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b8d646640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880ccf20 0x7f9b880c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b87fff640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 0x7f9b880cc9e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009cd0 tx=0x7f9b70003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70031da0 con 0x7f9b880d3870 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b70003e10 con 0x7f9b880d3870 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70033310 con 0x7f9b880d3870 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b880c8410 con 0x7f9b880d3870 2024-09-21T05:40:44.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.293+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b880c89d0 con 0x7f9b880d3870 2024-09-21T05:40:44.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.294+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b880d16a0 con 0x7f9b880d3870 2024-09-21T05:40:44.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.295+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b700334b0 con 0x7f9b880d3870 2024-09-21T05:40:44.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.296+0000 7f9b857fa640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9b68078b90 0x7f9b6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:44.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.296+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f9b700bc280 con 0x7f9b880d3870 2024-09-21T05:40:44.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.296+0000 7f9b8ce45640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9b68078b90 0x7f9b6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:44.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.297+0000 7f9b8ce45640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9b68078b90 0x7f9b6807b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b78000f60 tx=0x7f9b78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:44.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.301+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9b7003f030 con 0x7f9b880d3870 2024-09-21T05:40:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.529+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9b880cd410 con 0x7f9b880d3870 2024-09-21T05:40:44.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.530+0000 7f9b857fa640 1 -- 172.21.15.22:0/775822269 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f9b700847e0 con 0x7f9b880d3870 2024-09-21T05:40:44.532 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9b68078b90 msgr2=0x7f9b6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9b68078b90 0x7f9b6807b0a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b78000f60 tx=0x7f9b78009040 comp rx=0 tx=0).stop 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 msgr2=0x7f9b880cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 0x7f9b880cc9e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9b70009cd0 tx=0x7f9b70003c20 comp rx=0 tx=0).stop 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 shutdown_connections 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9b880ccf20 0x7f9b880c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9b880d3870 0x7f9b880cc9e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9b68078b90 0x7f9b6807b0a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 --2- 172.21.15.22:0/775822269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9b880b3540 0x7f9b881590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 >> 172.21.15.22:0/775822269 conn(0x7f9b8808bfc0 msgr2=0x7f9b880b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 shutdown_connections 2024-09-21T05:40:44.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44.532+0000 7f9b8f0a9640 1 -- 172.21.15.22:0/775822269 wait complete. 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:40:44.546 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:40:44.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:44.546 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:44.563 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42699sec preferred_lft 42699sec 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:40:44.564 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:40:44.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:44.566 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:40:44.627 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:40:44.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:44.629 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:40:44.685 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:40:44.685 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:40:44.685 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:40:44.685 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:40:44.685 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:40:44.686 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:40:44.686 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:40:44.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:40:44.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:40:44.686 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:40:44.686 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:40:44.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:40:44.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:44.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:44.814 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:40:44.830 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:40:44.830 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:40:44.830 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:40:44.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:44.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:40:44.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:40:44.896 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:44.951 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:40:44.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:40:44.952 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:45.008 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:40:45.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:45.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:45.104 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:45.104 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:40:45.104 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:40:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:40:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:40:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:40:45.116 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:40:45.136 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:40:45.137 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:40:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:40:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:40:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:40:45.137 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:40:45.138 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:40:45.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:45.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:40:45.165 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:40:45.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:40:45.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:45.270 INFO:teuthology.orchestra.run.smithi022.stdout:client5245 2024-09-21T05:40:45.270 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:40:45.271 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:40:45.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:45.287 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-09-21T05:40:45.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:45.348 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-09-21T05:40:45.407 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:40:45.407 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:40:45.407 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:40:45.407 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:40:45.407 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-09-21T05:40:45.408 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:40:45.513 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:40:45.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:40:45.522 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:40:45.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.677+0000 7fd9a3120640 1 -- 172.21.15.22:0/1454699574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c075fa0 msgr2=0x7fd99c070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:45.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.677+0000 7fd9a3120640 1 --2- 172.21.15.22:0/1454699574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c075fa0 0x7fd99c070e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd990005fd0 tx=0x7fd99002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.678+0000 7fd9a3120640 1 -- 172.21.15.22:0/1454699574 shutdown_connections 2024-09-21T05:40:45.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.678+0000 7fd9a3120640 1 --2- 172.21.15.22:0/1454699574 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd99c15f9b0 0x7fd99c0ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.678+0000 7fd9a3120640 1 --2- 172.21.15.22:0/1454699574 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c075fa0 0x7fd99c070e40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.678+0000 7fd9a3120640 1 -- 172.21.15.22:0/1454699574 >> 172.21.15.22:0/1454699574 conn(0x7fd99c0cd530 msgr2=0x7fd99c0cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:45.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.678+0000 7fd9a3120640 1 -- 172.21.15.22:0/1454699574 shutdown_connections 2024-09-21T05:40:45.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.678+0000 7fd9a3120640 1 -- 172.21.15.22:0/1454699574 wait complete. 2024-09-21T05:40:45.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.679+0000 7fd9a3120640 1 Processor -- start 2024-09-21T05:40:45.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.680+0000 7fd9a3120640 1 -- start start 2024-09-21T05:40:45.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a3120640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd99c06e100 0x7fd99c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a3120640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd99c075fa0 0x7fd99c1421c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a3120640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 0x7fd99c142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a3120640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c0cf060 con 0x7fd99c1429d0 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a3120640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c0ceee0 con 0x7fd99c075fa0 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a3120640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd99c0cf1e0 con 0x7fd99c06e100 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 0x7fd99c142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 0x7fd99c142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41990/0 (socket says 172.21.15.22:41990) 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 -- 172.21.15.22:0/3240122652 learned_addr learned my addr 172.21.15.22:0/3240122652 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a0ebc640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd99c06e100 0x7fd99c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd99bfff640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd99c075fa0 0x7fd99c1421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:45.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 -- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd99c06e100 msgr2=0x7fd99c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd99c06e100 0x7fd99c0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 -- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd99c075fa0 msgr2=0x7fd99c1421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd99c075fa0 0x7fd99c1421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.681+0000 7fd9a16bd640 1 -- 172.21.15.22:0/3240122652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c142700 con 0x7fd99c1429d0 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.682+0000 7fd9a16bd640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 0x7fd99c142de0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd99400eab0 tx=0x7fd99400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.682+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99400cea0 con 0x7fd99c1429d0 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.682+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd994004b20 con 0x7fd99c1429d0 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.682+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c0c5e60 con 0x7fd99c1429d0 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.682+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994005020 con 0x7fd99c1429d0 2024-09-21T05:40:45.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.682+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c0c6420 con 0x7fd99c1429d0 2024-09-21T05:40:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.683+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99c155480 con 0x7fd99c1429d0 2024-09-21T05:40:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.684+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9940051c0 con 0x7fd99c1429d0 2024-09-21T05:40:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.684+0000 7fd9997fa640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd980078bc0 0x7fd98007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:45.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.684+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fd99409b860 con 0x7fd99c1429d0 2024-09-21T05:40:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.684+0000 7fd9a0ebc640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd980078bc0 0x7fd98007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:45.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.685+0000 7fd9a0ebc640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd980078bc0 0x7fd98007b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd990033000 tx=0x7fd990004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:45.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.689+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd99c155480 con 0x7fd99c1429d0 2024-09-21T05:40:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.912+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd99c0cf690 con 0x7fd99c1429d0 2024-09-21T05:40:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.913+0000 7fd9997fa640 1 -- 172.21.15.22:0/3240122652 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fd99409b030 con 0x7fd99c1429d0 2024-09-21T05:40:45.915 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:45.915 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:40:45.916 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:40:45.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd980078bc0 msgr2=0x7fd98007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd980078bc0 0x7fd98007b0d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd990033000 tx=0x7fd990004730 comp rx=0 tx=0).stop 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 msgr2=0x7fd99c142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 0x7fd99c142de0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd99400eab0 tx=0x7fd99400ef70 comp rx=0 tx=0).stop 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 shutdown_connections 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd99c1429d0 0x7fd99c142de0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd99c075fa0 0x7fd99c1421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd980078bc0 0x7fd98007b0d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 --2- 172.21.15.22:0/3240122652 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd99c06e100 0x7fd99c0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 >> 172.21.15.22:0/3240122652 conn(0x7fd99c0cd530 msgr2=0x7fd99c070090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 shutdown_connections 2024-09-21T05:40:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:45.915+0000 7fd9a3120640 1 -- 172.21.15.22:0/3240122652 wait complete. 2024-09-21T05:40:45.929 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-21T05:40:46.077 INFO:teuthology.orchestra.run.smithi134.stdout:{"message":"","return_code":0} 2024-09-21T05:40:46.077 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:40:46.077 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-21T05:40:46.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.132+0000 7fb08deb2f40 1 Processor -- start 2024-09-21T05:40:46.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.132+0000 7fb08deb2f40 1 -- start start 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08deb2f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 0x559f1caf8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08deb2f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca2fea0 0x559f1cb00900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08deb2f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca31e60 0x559f1cb00e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08deb2f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559f1c97fd60 con 0x559f1ca2e3d0 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08deb2f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f1c9de430 con 0x559f1ca31e60 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08deb2f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559f1ca329a0 con 0x559f1ca2fea0 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08796e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 0x559f1caf8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08796e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 0x559f1caf8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42004/0 (socket says 172.21.15.22:42004) 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08796e640 1 -- 172.21.15.22:0/3679102883 learned_addr learned my addr 172.21.15.22:0/3679102883 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08716d640 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca2fea0 0x559f1cb00900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.133+0000 7fb08816f640 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca31e60 0x559f1cb00e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08796e640 1 -- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca2fea0 msgr2=0x559f1cb00900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08796e640 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca2fea0 0x559f1cb00900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08796e640 1 -- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca31e60 msgr2=0x559f1cb00e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08796e640 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca31e60 0x559f1cb00e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08796e640 1 -- 172.21.15.22:0/3679102883 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1cb032d0 con 0x559f1ca2e3d0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08796e640 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 0x559f1caf8640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c550 tx=0x7fb07800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8a89b0b083daa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08616b640 1 -- 172.21.15.22:0/3679102883 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07800a960 con 0x559f1ca2e3d0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08616b640 1 -- 172.21.15.22:0/3679102883 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb07800ab00 con 0x559f1ca2e3d0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.134+0000 7fb08616b640 1 -- 172.21.15.22:0/3679102883 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078008420 con 0x559f1ca2e3d0 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3679102883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 msgr2=0x559f1caf8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 0x559f1caf8640 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c550 tx=0x7fb07800c950 comp rx=0 tx=0).stop 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3679102883 shutdown_connections 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca31e60 0x559f1cb00e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca2fea0 0x559f1cb00900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3679102883 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2e3d0 0x559f1caf8640 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3679102883 >> 172.21.15.22:0/3679102883 conn(0x559f1c950ac0 msgr2=0x559f1caff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3679102883 shutdown_connections 2024-09-21T05:40:46.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.135+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3679102883 wait complete. 2024-09-21T05:40:46.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.136+0000 7fb08deb2f40 1 Processor -- start 2024-09-21T05:40:46.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.136+0000 7fb08deb2f40 1 -- start start 2024-09-21T05:40:46.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08deb2f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca2e3d0 0x559f1caf7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08deb2f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2fea0 0x559f1caf3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08deb2f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca31e60 0x559f1caf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08deb2f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559f1cb04830 con 0x559f1ca2fea0 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08deb2f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f1ca7b680 con 0x559f1ca2e3d0 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08deb2f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559f1caf98a0 con 0x559f1ca31e60 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2fea0 0x559f1caf3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2fea0 0x559f1caf3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42008/0 (socket says 172.21.15.22:42008) 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 -- 172.21.15.22:0/3010099018 learned_addr learned my addr 172.21.15.22:0/3010099018 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08816f640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca31e60 0x559f1caf8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08796e640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca2e3d0 0x559f1caf7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 -- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca31e60 msgr2=0x559f1caf8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca31e60 0x559f1caf8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 -- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca2e3d0 msgr2=0x559f1caf7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f1ca2e3d0 0x559f1caf7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08716d640 1 -- 172.21.15.22:0/3010099018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1caf3c90 con 0x559f1ca2fea0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.137+0000 7fb08816f640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f1ca31e60 0x559f1caf8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.138+0000 7fb08716d640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2fea0 0x559f1caf3580 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb07c025e30 tx=0x7fb07c04d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.138+0000 7fb085169640 1 -- 172.21.15.22:0/3010099018 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c068070 con 0x559f1ca2fea0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.138+0000 7fb085169640 1 -- 172.21.15.22:0/3010099018 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb07c056620 con 0x559f1ca2fea0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.138+0000 7fb085169640 1 -- 172.21.15.22:0/3010099018 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c056a30 con 0x559f1ca2fea0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.138+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f1caf5fa0 con 0x559f1ca2fea0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.138+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1caf6560 con 0x559f1ca2fea0 2024-09-21T05:40:46.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.140+0000 7fb085169640 1 -- 172.21.15.22:0/3010099018 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb07c06b070 con 0x559f1ca2fea0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.140+0000 7fb085169640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0740775a0 0x7fb074079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.140+0000 7fb085169640 1 -- 172.21.15.22:0/3010099018 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7fb07c0dd5d0 con 0x559f1ca2fea0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.140+0000 7fb08796e640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0740775a0 0x7fb074079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.140+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559f1ca302b0 0x559f1ca306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.140+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x559f1cba8e10 con 0x559f1ca302b0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08816f640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559f1ca302b0 0x559f1ca306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08796e640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0740775a0 0x7fb074079ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c470 tx=0x7fb07800e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08816f640 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559f1ca302b0 0x559f1ca306c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08816f640 1 -- 172.21.15.22:0/3010099018 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fb08000dc00 con 0x559f1ca302b0 2024-09-21T05:40:46.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559f1ca302b0 msgr2=0x559f1ca306c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559f1ca302b0 0x559f1ca306c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0740775a0 msgr2=0x7fb074079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0740775a0 0x7fb074079ab0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb07800c470 tx=0x7fb07800e040 comp rx=0 tx=0).stop 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2fea0 msgr2=0x559f1caf3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 --2- 172.21.15.22:0/3010099018 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f1ca2fea0 0x559f1caf3580 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb07c025e30 tx=0x7fb07c04d7b0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08816f640 1 -- 172.21.15.22:0/3010099018 reap_dead start 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 shutdown_connections 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.141+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 >> 172.21.15.22:0/3010099018 conn(0x559f1c950ac0 msgr2=0x559f1caff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.142+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 shutdown_connections 2024-09-21T05:40:46.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.142+0000 7fb08deb2f40 1 -- 172.21.15.22:0/3010099018 wait complete. 2024-09-21T05:40:46.147 INFO:teuthology.orchestra.run.smithi022.stdout:Yalphaparent_a 2024-09-21T05:40:46.147 DEBUG:teuthology.orchestra.run.smithi022:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627777, 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: "ancestors": [ 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: "dirino": 1099511627776, 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "alpha", 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 2 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:40:46.256 INFO:teuthology.orchestra.run.smithi022.stdout: "dirino": 1, 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "parent_a", 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 4 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 7, 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: "old_pools": [] 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:40:46.257 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:46.264 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-09-21T05:40:46.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.320+0000 7faa4bd36f40 1 Processor -- start 2024-09-21T05:40:46.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.320+0000 7faa4bd36f40 1 -- start start 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa4bd36f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d23f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa4bd36f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d246810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa4bd36f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d246d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa4bd36f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556d4d0c6d60 con 0x556d4d178e60 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa4bd36f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556d4d125430 con 0x556d4d176ea0 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa4bd36f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556d4d178400 con 0x556d4d1753d0 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa45ff1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d246d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa457f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d23f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa45ff1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d246d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42016/0 (socket says 172.21.15.22:42016) 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa45ff1640 1 -- 172.21.15.22:0/2159828144 learned_addr learned my addr 172.21.15.22:0/2159828144 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa44fef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d246810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.321+0000 7faa45ff1640 1 -- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 msgr2=0x556d4d23f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa45ff1640 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d23f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa45ff1640 1 -- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 msgr2=0x556d4d246810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa45ff1640 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d246810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa45ff1640 1 -- 172.21.15.22:0/2159828144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4d1a4070 con 0x556d4d178e60 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa457f0640 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d23f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa44fef640 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d246810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa45ff1640 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d246d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b7e0 tx=0x7faa4000bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e065cfbefaa85e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa377fe640 1 -- 172.21.15.22:0/2159828144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000db30 con 0x556d4d178e60 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa377fe640 1 -- 172.21.15.22:0/2159828144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa4000ddc0 con 0x556d4d178e60 2024-09-21T05:40:46.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.322+0000 7faa377fe640 1 -- 172.21.15.22:0/2159828144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000f690 con 0x556d4d178e60 2024-09-21T05:40:46.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2159828144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 msgr2=0x556d4d246d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d246d50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b7e0 tx=0x7faa4000bca0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2159828144 shutdown_connections 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d246d50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d246810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2159828144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d23f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2159828144 >> 172.21.15.22:0/2159828144 conn(0x556d4d097ac0 msgr2=0x556d4d1772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2159828144 shutdown_connections 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.323+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2159828144 wait complete. 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.324+0000 7faa4bd36f40 1 Processor -- start 2024-09-21T05:40:46.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.324+0000 7faa4bd36f40 1 -- start start 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa4bd36f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d2399f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa4bd36f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d1a17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa4bd36f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d2e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa4bd36f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556d4d2455b0 con 0x556d4d178e60 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa4bd36f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556d4d1a4350 con 0x556d4d176ea0 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa4bd36f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556d4d2e5290 con 0x556d4d1753d0 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d2e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa457f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d2399f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d2e46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42026/0 (socket says 172.21.15.22:42026) 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa44fef640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d1a17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa457f0640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d2399f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:33090/0 (socket says 172.21.15.22:33090) 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 -- 172.21.15.22:0/2789454113 learned_addr learned my addr 172.21.15.22:0/2789454113 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 -- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 msgr2=0x556d4d2399f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556d4d1753d0 0x556d4d2399f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 -- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 msgr2=0x556d4d1a17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556d4d176ea0 0x556d4d1a17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 -- 172.21.15.22:0/2789454113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4d1a3570 con 0x556d4d178e60 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.326+0000 7faa45ff1640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d2e46f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faa4000dc10 tx=0x7faa4000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.327+0000 7faa367fc640 1 -- 172.21.15.22:0/2789454113 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa40004220 con 0x556d4d178e60 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.327+0000 7faa367fc640 1 -- 172.21.15.22:0/2789454113 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa4000f730 con 0x556d4d178e60 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.327+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556d4d1a3850 con 0x556d4d178e60 2024-09-21T05:40:46.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.327+0000 7faa367fc640 1 -- 172.21.15.22:0/2789454113 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4000fb80 con 0x556d4d178e60 2024-09-21T05:40:46.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.327+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4d2e4fd0 con 0x556d4d178e60 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.328+0000 7faa367fc640 1 -- 172.21.15.22:0/2789454113 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa40003040 con 0x556d4d178e60 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.329+0000 7faa367fc640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa280775a0 0x7faa28079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.329+0000 7faa367fc640 1 -- 172.21.15.22:0/2789454113 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7faa4009b840 con 0x556d4d178e60 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.329+0000 7faa457f0640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa280775a0 0x7faa28079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.329+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x556d4d2ec880 0x556d4d2eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.329+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) -- 0x556d4d2467f0 con 0x556d4d2ec880 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.330+0000 7faa44fef640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x556d4d2ec880 0x556d4d2eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.330+0000 7faa457f0640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa280775a0 0x7faa28079ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa38055000 tx=0x7faa38003020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.330+0000 7faa44fef640 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x556d4d2ec880 0x556d4d2eeca0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.331+0000 7faa44fef640 1 -- 172.21.15.22:0/2789454113 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v43'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faa3c00f600 con 0x556d4d2ec880 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x556d4d2ec880 msgr2=0x556d4d2eeca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x556d4d2ec880 0x556d4d2eeca0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa280775a0 msgr2=0x7faa28079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa280775a0 0x7faa28079ab0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faa38055000 tx=0x7faa38003020 comp rx=0 tx=0).stop 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 msgr2=0x556d4d2e46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 --2- 172.21.15.22:0/2789454113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556d4d178e60 0x556d4d2e46f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faa4000dc10 tx=0x7faa4000dc40 comp rx=0 tx=0).stop 2024-09-21T05:40:46.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa45ff1640 1 -- 172.21.15.22:0/2789454113 reap_dead start 2024-09-21T05:40:46.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 shutdown_connections 2024-09-21T05:40:46.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 >> 172.21.15.22:0/2789454113 conn(0x556d4d097ac0 msgr2=0x556d4d245870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 shutdown_connections 2024-09-21T05:40:46.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.332+0000 7faa4bd36f40 1 -- 172.21.15.22:0/2789454113 wait complete. 2024-09-21T05:40:46.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-21T05:40:46.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:46.339 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-09-21T05:40:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/3914072411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b5bc0 msgr2=0x7f8d180b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/3914072411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b5bc0 0x7f8d180b5fd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8d18072840 tx=0x7f8d1002aa30 comp rx=0 tx=0).stop 2024-09-21T05:40:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/3914072411 shutdown_connections 2024-09-21T05:40:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/3914072411 >> v2:172.21.15.134:3300/0 conn(0x7f8d180ae030 0x7f8d180b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/3914072411 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b5bc0 0x7f8d180b5fd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/3914072411 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b4070 0x7f8d1815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/3914072411 >> 172.21.15.22:0/3914072411 conn(0x7f8d180b2d50 msgr2=0x7f8d180b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/3914072411 shutdown_connections 2024-09-21T05:40:46.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.460+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/3914072411 wait complete. 2024-09-21T05:40:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.462+0000 7f8d1c8d5640 1 Processor -- start 2024-09-21T05:40:46.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.462+0000 7f8d1c8d5640 1 -- start start 2024-09-21T05:40:46.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1c8d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d180ae030 0x7f8d180cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1c8d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 0x7f8d180cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1c8d5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b5bc0 0x7f8d1815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1c8d5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18164820 con 0x7f8d180b4070 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1c8d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181646a0 con 0x7f8d180ae030 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1c8d5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d181649a0 con 0x7f8d180b5bc0 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d15d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 0x7f8d180cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d16d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b5bc0 0x7f8d1815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d180ae030 0x7f8d180cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d15d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 0x7f8d180cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42042/0 (socket says 172.21.15.22:42042) 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d15d9b640 1 -- 172.21.15.22:0/2528706425 learned_addr learned my addr 172.21.15.22:0/2528706425 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.463+0000 7f8d16d9d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b5bc0 0x7f8d1815a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3301/0 says I am v2:172.21.15.22:33096/0 (socket says 172.21.15.22:33096) 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d15d9b640 1 -- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b5bc0 msgr2=0x7f8d1815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b5bc0 0x7f8d1815a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d15d9b640 1 -- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d180ae030 msgr2=0x7f8d180cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d180ae030 0x7f8d180cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d15d9b640 1 -- 172.21.15.22:0/2528706425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d180cbc90 con 0x7f8d180b4070 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d1659c640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d180ae030 0x7f8d180cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 0x7f8d180cd230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009a50 tx=0x7f8d0c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d06ffd640 1 -- 172.21.15.22:0/2528706425 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c01e070 con 0x7f8d180b4070 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d06ffd640 1 -- 172.21.15.22:0/2528706425 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d0c00bc10 con 0x7f8d180b4070 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d06ffd640 1 -- 172.21.15.22:0/2528706425 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c019410 con 0x7f8d180b4070 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d180cc190 con 0x7f8d180b4070 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.464+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d180cc750 con 0x7f8d180b4070 2024-09-21T05:40:46.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.467+0000 7f8d06ffd640 1 -- 172.21.15.22:0/2528706425 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8d0c021070 con 0x7f8d180b4070 2024-09-21T05:40:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.467+0000 7f8d06ffd640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8cf8078bc0 0x7f8cf807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.467+0000 7f8d06ffd640 1 -- 172.21.15.22:0/2528706425 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8d0c09e9a0 con 0x7f8d180b4070 2024-09-21T05:40:46.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.468+0000 7f8d23840740 1 Processor -- start 2024-09-21T05:40:46.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.468+0000 7f8d1659c640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8cf8078bc0 0x7f8cf807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.468+0000 7f8d23840740 1 -- start start 2024-09-21T05:40:46.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.468+0000 7f8d1659c640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8cf8078bc0 0x7f8cf807b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d1815f3c0 tx=0x7f8d08005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d23840740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a26150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d23840740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577f8951120 con 0x5577f860d550 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577f8940550 con 0x5577f8615730 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5577f8944360 con 0x5577f8636410 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d15d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d15d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a09250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42054/0 (socket says 172.21.15.22:42054) 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d15d9b640 1 -- 172.21.15.22:0/1714275058 learned_addr learned my addr 172.21.15.22:0/1714275058 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d16d9d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a26150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.469+0000 7f8d1659c640 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d15d9b640 1 -- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 msgr2=0x5577f8a08d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d15d9b640 1 -- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 msgr2=0x5577f8a26150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a26150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d15d9b640 1 -- 172.21.15.22:0/1714275058 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f8632470 con 0x5577f860d550 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d16d9d640 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a26150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a09250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0664a0 tx=0x7f8d0c065fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d362606a71df93c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1714275058 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c04f070 con 0x5577f860d550 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1714275058 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d0c070080 con 0x5577f860d550 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.470+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1714275058 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c065c40 con 0x5577f860d550 2024-09-21T05:40:46.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 -- 172.21.15.22:0/1714275058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 msgr2=0x5577f8a09250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a09250 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0664a0 tx=0x7f8d0c065fd0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 -- 172.21.15.22:0/1714275058 shutdown_connections 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a26150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a09250 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 --2- 172.21.15.22:0/1714275058 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 -- 172.21.15.22:0/1714275058 >> 172.21.15.22:0/1714275058 conn(0x5577f862a560 msgr2=0x5577f89defd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 -- 172.21.15.22:0/1714275058 shutdown_connections 2024-09-21T05:40:46.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.471+0000 7f8d23840740 1 -- 172.21.15.22:0/1714275058 wait complete. 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.472+0000 7f8d23840740 1 -- start start 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.472+0000 7f8d23840740 1 Processor -- start 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f89ed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a19ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a05670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5577f86b07c0 con 0x5577f860d550 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577f88e5890 con 0x5577f8615730 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5577f8a09790 con 0x5577f8636410 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d23840740 10 client.0 _ll_register_callbacks cb 0x5577f8939ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f89ed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f89ed120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42068/0 (socket says 172.21.15.22:42068) 2024-09-21T05:40:46.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 -- 172.21.15.22:0/3812629484 learned_addr learned my addr 172.21.15.22:0/3812629484 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d16d9d640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a05670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a19ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 -- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 msgr2=0x5577f8a05670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a05670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 -- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 msgr2=0x5577f8a19ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a19ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 -- 172.21.15.22:0/3812629484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f87167e0 con 0x5577f860d550 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8d1659c640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f89ed120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d0801ba70 tx=0x7f8d08013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8dfe7e7e49c0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8cf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8cf57fa640 1 -- 172.21.15.22:0/3812629484 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0802c060 con 0x5577f860d550 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.474+0000 7f8cf57fa640 1 -- 172.21.15.22:0/3812629484 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8d08027430 con 0x5577f860d550 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0c070080 con 0x5577f860d550 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8cf57fa640 1 -- 172.21.15.22:0/3812629484 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d08032050 con 0x5577f860d550 2024-09-21T05:40:46.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 10 client.5472 fetch_fsmap learned FSMap version 26 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5577f8776f70 con 0x5577f860d550 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8cf57fa640 1 -- 172.21.15.22:0/3812629484 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d0c070080 con 0x5577f860d550 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8cf57fa640 1 -- 172.21.15.22:0/3812629484 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f8d08035020 con 0x5577f860d550 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 10 client.5472 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 10 client.5472 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 20 client.5472 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f85f51f0 0x5577f89e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 4 client.5472 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577f8953070 con 0x5577f85f51f0 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.475+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f85f51f0 0x5577f89e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.476+0000 7f8cf57fa640 10 client.5472 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.476+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f85f51f0 0x5577f89e9110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e350372ec1707a3e server_cookie=3adef9181dec4a45 in_seq=0 out_seq=0 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.476+0000 7f8cf57fa640 1 -- 172.21.15.22:0/3812629484 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577f8953070 con 0x5577f85f51f0 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.476+0000 7f8cf57fa640 10 client.5472 handle_command_reply: tid=0 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.476+0000 7f8d23840740 1 client.5472 shutdown 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 msgr2=0x5577f89ed120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f89ed120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d0801ba70 tx=0x7f8d08013040 comp rx=0 tx=0).stop 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 shutdown_connections 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f85f51f0 0x5577f89e9110 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a05670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a19ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 --2- 172.21.15.22:0/3812629484 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f89ed120 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 >> 172.21.15.22:0/3812629484 conn(0x5577f862a560 msgr2=0x5577f89d8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8cf57fa640 0 client.5472 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 shutdown_connections 2024-09-21T05:40:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 1 -- 172.21.15.22:0/3812629484 wait complete. 2024-09-21T05:40:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.477+0000 7f8d23840740 20 client.5472 trim_cache size 0 max 16384 2024-09-21T05:40:46.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 -- 172.21.15.22:0/2739096179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 msgr2=0x7f08c80ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 --2- 172.21.15.22:0/2739096179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 0x7f08c80ccc10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009830 tx=0x7f08b402a480 comp rx=0 tx=0).stop 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 -- 172.21.15.22:0/2739096179 shutdown_connections 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 --2- 172.21.15.22:0/2739096179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 0x7f08c80ccc10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 --2- 172.21.15.22:0/2739096179 >> v2:172.21.15.134:3300/0 conn(0x7f08c80b5200 0x7f08c80cd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 -- 172.21.15.22:0/2739096179 >> 172.21.15.22:0/2739096179 conn(0x7f08c80b3ee0 msgr2=0x7f08c80b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 -- 172.21.15.22:0/2739096179 shutdown_connections 2024-09-21T05:40:46.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.481+0000 7f08ced32640 1 -- 172.21.15.22:0/2739096179 wait complete. 2024-09-21T05:40:46.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.483+0000 7f08ced32640 1 Processor -- start 2024-09-21T05:40:46.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.483+0000 7f08ced32640 1 -- start start 2024-09-21T05:40:46.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ced32640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c80b5200 0x7f08c81f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ced32640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 0x7f08c814bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ced32640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08c8160290 0x7f08c80b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ced32640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08c80d2320 con 0x7f08c80b5200 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ced32640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08c80d21a0 con 0x7f08c8160290 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ced32640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08c80d24a0 con 0x7f08c815c6c0 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ccace640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c80b5200 0x7f08c81f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.484+0000 7f08ccace640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c80b5200 0x7f08c81f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42074/0 (socket says 172.21.15.22:42074) 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 -- 172.21.15.22:0/693309668 learned_addr learned my addr 172.21.15.22:0/693309668 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08c7fff640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 0x7f08c814bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08cd2cf640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08c8160290 0x7f08c80b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 -- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 msgr2=0x7f08c814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08c815c6c0 0x7f08c814bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 -- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08c8160290 msgr2=0x7f08c80b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08c8160290 0x7f08c80b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 -- 172.21.15.22:0/693309668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c814c7c0 con 0x7f08c80b5200 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ccace640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08c80b5200 0x7f08c81f6700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00e9f0 tx=0x7f08bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc00cd80 con 0x7f08c80b5200 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08bc004020 con 0x7f08c80b5200 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ced32640 1 -- 172.21.15.22:0/693309668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c814caa0 con 0x7f08c80b5200 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc004470 con 0x7f08c80b5200 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.485+0000 7f08ced32640 1 -- 172.21.15.22:0/693309668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c8149ea0 con 0x7f08c80b5200 2024-09-21T05:40:46.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.486+0000 7f08ced32640 1 -- 172.21.15.22:0/693309668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f08c80cbe00 con 0x7f08c80b5200 2024-09-21T05:40:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.487+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08bc0035f0 con 0x7f08c80b5200 2024-09-21T05:40:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.488+0000 7f08c57fa640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f08ac078bc0 0x7f08ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.488+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f08bc09bcc0 con 0x7f08c80b5200 2024-09-21T05:40:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.488+0000 7f08c7fff640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f08ac078bc0 0x7f08ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.488+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+508 (secure 0 0 0) 0x7f08bc09c170 con 0x7f08c80b5200 2024-09-21T05:40:46.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.488+0000 7f08ced32640 1 -- 172.21.15.22:0/693309668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f08c815bba0 con 0x7f08c80b5200 2024-09-21T05:40:46.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.489+0000 7f08c7fff640 1 --2- 172.21.15.22:0/693309668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f08ac078bc0 0x7f08ac07b0d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08b4034000 tx=0x7f08b4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.497+0000 7f8d23840740 1 Processor -- start 2024-09-21T05:40:46.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.497+0000 7f8d23840740 1 -- start start 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d23840740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a04ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d23840740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d181fdcf0 con 0x5577f860d550 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18156070 con 0x5577f8615730 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1806d6f0 con 0x5577f8636410 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a47630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42086/0 (socket says 172.21.15.22:42086) 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 -- 172.21.15.22:0/1001709520 learned_addr learned my addr 172.21.15.22:0/1001709520 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 -- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 msgr2=0x5577f8a04ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d16d9d640 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a04ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a04ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 -- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 msgr2=0x5577f8a045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.498+0000 7f8d1659c640 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.499+0000 7f8d1659c640 1 -- 172.21.15.22:0/1001709520 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f867f720 con 0x5577f860d550 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.499+0000 7f8d16d9d640 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a04ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.499+0000 7f8d1659c640 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a47630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009a20 tx=0x7f8d0801b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937acc71f81fd1c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.499+0000 7f8cf77fe640 1 -- 172.21.15.22:0/1001709520 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0803a030 con 0x5577f860d550 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.499+0000 7f8cf77fe640 1 -- 172.21.15.22:0/1001709520 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d0800edb0 con 0x5577f860d550 2024-09-21T05:40:46.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.499+0000 7f8cf77fe640 1 -- 172.21.15.22:0/1001709520 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08005040 con 0x5577f860d550 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 -- 172.21.15.22:0/1001709520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 msgr2=0x5577f8a47630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a47630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009a20 tx=0x7f8d0801b8d0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 -- 172.21.15.22:0/1001709520 shutdown_connections 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a04ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 --2- 172.21.15.22:0/1001709520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8a47630 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 -- 172.21.15.22:0/1001709520 >> 172.21.15.22:0/1001709520 conn(0x5577f862a560 msgr2=0x5577f8a05430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 -- 172.21.15.22:0/1001709520 shutdown_connections 2024-09-21T05:40:46.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.500+0000 7f8d23840740 1 -- 172.21.15.22:0/1001709520 wait complete. 2024-09-21T05:40:46.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.501+0000 7f8d23840740 1 -- start start 2024-09-21T05:40:46.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.501+0000 7f8d23840740 1 Processor -- start 2024-09-21T05:40:46.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8aa2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a089d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8d18155a20 con 0x5577f860d550 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5577f89ed5b0 con 0x5577f8615730 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5577f8a16870 con 0x5577f8636410 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d23840740 10 client.0 _ll_register_callbacks cb 0x7f8d0801b400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8aa2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8aa2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42100/0 (socket says 172.21.15.22:42100) 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d1659c640 1 -- 172.21.15.22:0/2301673291 learned_addr learned my addr 172.21.15.22:0/2301673291 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a089d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.502+0000 7f8d16d9d640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d1659c640 1 -- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 msgr2=0x5577f8a08f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d1659c640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d1659c640 1 -- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 msgr2=0x5577f8a089d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d1659c640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a089d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d1659c640 1 -- 172.21.15.22:0/2301673291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f8a19880 con 0x5577f860d550 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d1659c640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8aa2a90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8d08005f60 tx=0x7f8d08005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6740e4a68dc4fdc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8cf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8cf57fa640 1 -- 172.21.15.22:0/2301673291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08043020 con 0x5577f860d550 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8cf57fa640 1 -- 172.21.15.22:0/2301673291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8d08004b40 con 0x5577f860d550 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0800edb0 con 0x5577f860d550 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8cf57fa640 1 -- 172.21.15.22:0/2301673291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d0804d050 con 0x5577f860d550 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d23840740 10 client.5484 fetch_fsmap learned FSMap version 26 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5577f8a1c7f0 con 0x5577f860d550 2024-09-21T05:40:46.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.503+0000 7f8cf57fa640 1 -- 172.21.15.22:0/2301673291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d0800edb0 con 0x5577f860d550 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8cf57fa640 1 -- 172.21.15.22:0/2301673291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f8d08050020 con 0x5577f860d550 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d23840740 10 client.5484 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d23840740 10 client.5484 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d23840740 20 client.5484 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d23840740 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f8a00530 0x5577f8a159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d23840740 4 client.5484 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5577f894eb20 con 0x5577f8a00530 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f8a00530 0x5577f8a159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8cf57fa640 10 client.5484 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.504+0000 7f8d15d9b640 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f8a00530 0x5577f8a159e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ab20f97759969c server_cookie=4147db82a41c506 in_seq=0 out_seq=0 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8cf57fa640 1 -- 172.21.15.22:0/2301673291 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5577f894eb20 con 0x5577f8a00530 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8cf57fa640 10 client.5484 handle_command_reply: tid=0 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 client.5484 shutdown 2024-09-21T05:40:46.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 msgr2=0x5577f8aa2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8aa2a90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8d08005f60 tx=0x7f8d08005650 comp rx=0 tx=0).stop 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 shutdown_connections 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x5577f8a00530 0x5577f8a159e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5577f8636410 0x5577f8a08f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5577f8615730 0x5577f8a089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8cf57fa640 0 client.5484 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 --2- 172.21.15.22:0/2301673291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5577f860d550 0x5577f8aa2a90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.505+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 >> 172.21.15.22:0/2301673291 conn(0x5577f862a560 msgr2=0x5577f8a04520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 shutdown_connections 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d23840740 1 -- 172.21.15.22:0/2301673291 wait complete. 2024-09-21T05:40:46.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d23840740 20 client.5484 trim_cache size 0 max 16384 2024-09-21T05:40:46.513 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:40:46.513 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:40:46.513 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "0f18ddfb-f9c3-45a0-96fd-1e23c629b28a", 2024-09-21T05:40:46.513 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:40:46.513 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8cf8078bc0 msgr2=0x7f8cf807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8cf8078bc0 0x7f8cf807b0d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8d1815f3c0 tx=0x7f8d08005f90 comp rx=0 tx=0).stop 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 msgr2=0x7f8d180cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 0x7f8d180cd230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009a50 tx=0x7f8d0c009f10 comp rx=0 tx=0).stop 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 shutdown_connections 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8cf8078bc0 0x7f8cf807b0d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.506+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8d180b5bc0 0x7f8d1815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.507+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8d180b4070 0x7f8d180cd230 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.507+0000 7f8d1c8d5640 1 --2- 172.21.15.22:0/2528706425 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8d180ae030 0x7f8d180cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.507+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 >> 172.21.15.22:0/2528706425 conn(0x7f8d180b2d50 msgr2=0x7f8d18158450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.507+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 shutdown_connections 2024-09-21T05:40:46.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.507+0000 7f8d1c8d5640 1 -- 172.21.15.22:0/2528706425 wait complete. 2024-09-21T05:40:46.515 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-21T05:40:46.505+0000 7fc2c4aa3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 pv4 ap=4 DIRTYPARENT s=0 n(v0 1=1+0)->n(v0 rc2024-09-21T05:40:45.404725+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={5245=0-4194304@1} caps={5245=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=5245 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x56094c4a0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:40:46.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:40:46.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7737077640 1 -- 172.21.15.22:0/759401831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7718004800 msgr2=0x7f7718004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7737077640 1 --2- 172.21.15.22:0/759401831 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7718004800 0x7f7718004c10 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7720009cd0 tx=0x7f7718005830 comp rx=0 tx=0).stop 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7735614640 1 -- 172.21.15.22:0/759401831 reap_dead start 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7737077640 1 -- 172.21.15.22:0/759401831 shutdown_connections 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7737077640 1 -- 172.21.15.22:0/759401831 >> 172.21.15.22:0/759401831 conn(0x7f7730072ae0 msgr2=0x7f7730072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7737077640 1 -- 172.21.15.22:0/759401831 shutdown_connections 2024-09-21T05:40:46.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.637+0000 7f7737077640 1 -- 172.21.15.22:0/759401831 wait complete. 2024-09-21T05:40:46.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.639+0000 7f7737077640 1 Processor -- start 2024-09-21T05:40:46.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.639+0000 7f7737077640 1 -- start start 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.640+0000 7f7737077640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7718004800 0x7f77301f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.640+0000 7f7737077640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 0x7f773006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7737077640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f773006e430 0x7f773006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7737077640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77300d16d0 con 0x7f77300ae9b0 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7737077640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f773015c9a0 con 0x7f773006e430 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7737077640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77300d1850 con 0x7f7718004800 2024-09-21T05:40:46.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 0x7f773006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 0x7f773006def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42116/0 (socket says 172.21.15.22:42116) 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 -- 172.21.15.22:0/929104183 learned_addr learned my addr 172.21.15.22:0/929104183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7735614640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f773006e430 0x7f773006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7734e13640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7718004800 0x7f77301f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 -- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7718004800 msgr2=0x7f77301f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7718004800 0x7f77301f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 -- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f773006e430 msgr2=0x7f773006e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f773006e430 0x7f773006e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 -- 172.21.15.22:0/929104183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773014cbd0 con 0x7f77300ae9b0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7735614640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f773006e430 0x7f773006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7727fff640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 0x7f773006def0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7720009df0 tx=0x7f7720003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f77257fa640 1 -- 172.21.15.22:0/929104183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720038040 con 0x7f77300ae9b0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f77257fa640 1 -- 172.21.15.22:0/929104183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772002ba30 con 0x7f77300ae9b0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773014ceb0 con 0x7f77300ae9b0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f77257fa640 1 -- 172.21.15.22:0/929104183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720003c70 con 0x7f77300ae9b0 2024-09-21T05:40:46.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.641+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773014d470 con 0x7f77300ae9b0 2024-09-21T05:40:46.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.643+0000 7f77257fa640 1 -- 172.21.15.22:0/929104183 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7720043020 con 0x7f77300ae9b0 2024-09-21T05:40:46.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.644+0000 7f77257fa640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f771c078bc0 0x7f771c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.644+0000 7f77257fa640 1 -- 172.21.15.22:0/929104183 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f77200bb180 con 0x7f77300ae9b0 2024-09-21T05:40:46.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.644+0000 7f7734e13640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f771c078bc0 0x7f771c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.645+0000 7f7734e13640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f771c078bc0 0x7f771c07b0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f772c005eb0 tx=0x7f772c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.645+0000 7f773dfdf740 1 Processor -- start 2024-09-21T05:40:46.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.645+0000 7f773dfdf740 1 -- start start 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc2ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbeea4d0 0x556efc2d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbef26b0 0x556efc2d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f773dfdf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556efc22da60 con 0x556efbf13390 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556efc21ce90 con 0x556efbeea4d0 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556efc220ca0 con 0x556efbef26b0 2024-09-21T05:40:46.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc2ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc2ffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42122/0 (socket says 172.21.15.22:42122) 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 -- 172.21.15.22:0/3734760811 learned_addr learned my addr 172.21.15.22:0/3734760811 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7727fff640 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbeea4d0 0x556efc2d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7735614640 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbef26b0 0x556efc2d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 -- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbef26b0 msgr2=0x556efc2d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbef26b0 0x556efc2d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 -- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbeea4d0 msgr2=0x556efc2d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbeea4d0 0x556efc2d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7734e13640 1 -- 172.21.15.22:0/3734760811 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556efbff57d0 con 0x556efbf13390 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.646+0000 7f7735614640 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbef26b0 0x556efc2d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.647+0000 7f7734e13640 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc2ffa10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f772c009e50 tx=0x7f772c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168ceba4a151a3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.647+0000 7f77167fc640 1 -- 172.21.15.22:0/3734760811 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c003b70 con 0x556efbf13390 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.647+0000 7f77167fc640 1 -- 172.21.15.22:0/3734760811 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772c003d10 con 0x556efbf13390 2024-09-21T05:40:46.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.647+0000 7f77167fc640 1 -- 172.21.15.22:0/3734760811 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772c0296a0 con 0x556efbf13390 2024-09-21T05:40:46.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.647+0000 7f773dfdf740 1 -- 172.21.15.22:0/3734760811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 msgr2=0x556efc2ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc2ffa10 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f772c009e50 tx=0x7f772c017000 comp rx=0 tx=0).stop 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 -- 172.21.15.22:0/3734760811 shutdown_connections 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbef26b0 0x556efc2d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbeea4d0 0x556efc2d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3734760811 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc2ffa10 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 -- 172.21.15.22:0/3734760811 >> 172.21.15.22:0/3734760811 conn(0x556efbf074e0 msgr2=0x556efc2bad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 -- 172.21.15.22:0/3734760811 shutdown_connections 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 -- 172.21.15.22:0/3734760811 wait complete. 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.648+0000 7f773dfdf740 1 -- start start 2024-09-21T05:40:46.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.649+0000 7f773dfdf740 1 Processor -- start 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 0x556efc2d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbf13390 0x556efc2f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556efbf8d740 con 0x556efbef26b0 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556efc1c2970 con 0x556efbf13390 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556efc2fb400 con 0x556efbeea4d0 2024-09-21T05:40:46.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f7727fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 0x556efc2d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f773dfdf740 10 client.0 _ll_register_callbacks cb 0x556efc1d1780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f7727fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 0x556efc2d37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42126/0 (socket says 172.21.15.22:42126) 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f7727fff640 1 -- 172.21.15.22:0/2305110984 learned_addr learned my addr 172.21.15.22:0/2305110984 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f7734e13640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f7735614640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbf13390 0x556efc2f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.650+0000 7f7727fff640 1 -- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 msgr2=0x556efc2fc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f7727fff640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2fc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f7727fff640 1 -- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbf13390 msgr2=0x556efc2f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f7727fff640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbf13390 0x556efc2f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f7727fff640 1 -- 172.21.15.22:0/2305110984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556efbf7e970 con 0x556efbef26b0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f7727fff640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 0x556efc2d37a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f77200bb130 tx=0x7f77200826d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c13bc32b73bcc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f770f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:46.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f770f7fe640 1 -- 172.21.15.22:0/2305110984 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7720082f60 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f770f7fe640 1 -- 172.21.15.22:0/2305110984 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f772008c3b0 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f772c003d10 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f770f7fe640 1 -- 172.21.15.22:0/2305110984 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7720087990 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 10 client.5496 fetch_fsmap learned FSMap version 26 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x556efbf5c6a0 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f770f7fe640 1 -- 172.21.15.22:0/2305110984 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f772c003d10 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f770f7fe640 1 -- 172.21.15.22:0/2305110984 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f772003b070 con 0x556efbef26b0 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 10 client.5496 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 10 client.5496 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 20 client.5496 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efbed1f60 0x556efc311f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 4 client.5496 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.651+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556efc22f9b0 con 0x556efbed1f60 2024-09-21T05:40:46.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.652+0000 7f7734e13640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efbed1f60 0x556efc311f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.652+0000 7f7734e13640 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efbed1f60 0x556efc311f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc02a2e272206f23 server_cookie=8bcf407877e8a749 in_seq=0 out_seq=0 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.652+0000 7f770f7fe640 10 client.5496 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f770f7fe640 1 -- 172.21.15.22:0/2305110984 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556efc22f9b0 con 0x556efbed1f60 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f770f7fe640 10 client.5496 handle_command_reply: tid=0 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 client.5496 shutdown 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 msgr2=0x556efc2d37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 0x556efc2d37a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f77200bb130 tx=0x7f77200826d0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 shutdown_connections 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efbed1f60 0x556efc311f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbf13390 0x556efc2f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbef26b0 0x556efc2d37a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f770f7fe640 0 client.5496 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 --2- 172.21.15.22:0/2305110984 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2fc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 >> 172.21.15.22:0/2305110984 conn(0x556efbf074e0 msgr2=0x556efc2b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 shutdown_connections 2024-09-21T05:40:46.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 1 -- 172.21.15.22:0/2305110984 wait complete. 2024-09-21T05:40:46.660 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.653+0000 7f773dfdf740 20 client.5496 trim_cache size 0 max 16384 2024-09-21T05:40:46.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.671+0000 7f773dfdf740 1 Processor -- start 2024-09-21T05:40:46.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.671+0000 7f773dfdf740 1 -- start start 2024-09-21T05:40:46.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc3106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc310c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f773dfdf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7730202770 con 0x556efbf13390 2024-09-21T05:40:46.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7730201790 con 0x556efbef26b0 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f773014f840 con 0x556efbeea4d0 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.672+0000 7f7735614640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc310c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc310c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42138/0 (socket says 172.21.15.22:42138) 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 -- 172.21.15.22:0/3207941392 learned_addr learned my addr 172.21.15.22:0/3207941392 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7727fff640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc3106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7734e13640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 -- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 msgr2=0x556efc2ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 -- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 msgr2=0x556efc3106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc3106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 -- 172.21.15.22:0/3207941392 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556efc2b7860 con 0x556efbf13390 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7727fff640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc3106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7734e13640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7735614640 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc310c00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f77300d02e0 tx=0x7f7728004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f69d0763a7a287 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7715ffb640 1 -- 172.21.15.22:0/3207941392 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772800c260 con 0x556efbf13390 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7715ffb640 1 -- 172.21.15.22:0/3207941392 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7728006530 con 0x556efbf13390 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.673+0000 7f7715ffb640 1 -- 172.21.15.22:0/3207941392 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77280068e0 con 0x556efbf13390 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 -- 172.21.15.22:0/3207941392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 msgr2=0x556efc310c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc310c00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f77300d02e0 tx=0x7f7728004d50 comp rx=0 tx=0).stop 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 -- 172.21.15.22:0/3207941392 shutdown_connections 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc310c00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc3106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 --2- 172.21.15.22:0/3207941392 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc2ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 -- 172.21.15.22:0/3207941392 >> 172.21.15.22:0/3207941392 conn(0x556efbf074e0 msgr2=0x556efc2d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 -- 172.21.15.22:0/3207941392 shutdown_connections 2024-09-21T05:40:46.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.674+0000 7f773dfdf740 1 -- 172.21.15.22:0/3207941392 wait complete. 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.675+0000 7f773dfdf740 1 -- start start 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.675+0000 7f773dfdf740 1 Processor -- start 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc301f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc381b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc370aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7730201140 con 0x556efbf13390 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556efc2b10a0 con 0x556efbef26b0 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556efc3024b0 con 0x556efbeea4d0 2024-09-21T05:40:46.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f773dfdf740 10 client.0 _ll_register_callbacks cb 0x7f7720005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f7735614640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc370aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f7735614640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc370aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:42146/0 (socket says 172.21.15.22:42146) 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.676+0000 7f7735614640 1 -- 172.21.15.22:0/1708765918 learned_addr learned my addr 172.21.15.22:0/1708765918 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7727fff640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc381b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7734e13640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc301f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7735614640 1 -- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 msgr2=0x556efc301f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7735614640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc301f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7735614640 1 -- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 msgr2=0x556efc381b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7735614640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc381b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7735614640 1 -- 172.21.15.22:0/1708765918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556efc372690 con 0x556efbf13390 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7727fff640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc381b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7734e13640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc301f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f7735614640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc370aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x556efc323040 tx=0x7f772801cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bed2801ae085df1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f770f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:40:46.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f770f7fe640 1 -- 172.21.15.22:0/1708765918 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728012440 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f770f7fe640 1 -- 172.21.15.22:0/1708765918 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f7728018050 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7728006530 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f770f7fe640 1 -- 172.21.15.22:0/1708765918 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f77280125e0 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f773dfdf740 10 client.5502 fetch_fsmap learned FSMap version 26 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.677+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x556efc382070 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f770f7fe640 1 -- 172.21.15.22:0/1708765918 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7728006530 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f770f7fe640 1 -- 172.21.15.22:0/1708765918 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f772802e050 con 0x556efbf13390 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f773dfdf740 10 client.5502 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f773dfdf740 10 client.5502 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f773dfdf740 20 client.5502 populate_metadata read hostname 'smithi022' 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f773dfdf740 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efc2e1840 0x556efc2e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f773dfdf740 4 client.5502 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556efc22b460 con 0x556efc2e1840 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f7734e13640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efc2e1840 0x556efc2e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:40:46.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f770f7fe640 10 client.5502 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f7734e13640 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efc2e1840 0x556efc2e4b50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8def37827090305b server_cookie=44908931c7ff98a in_seq=0 out_seq=0 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.678+0000 7f770f7fe640 1 -- 172.21.15.22:0/1708765918 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x556efc22b460 con 0x556efc2e1840 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f770f7fe640 10 client.5502 handle_command_reply: tid=0 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 client.5502 shutdown 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 msgr2=0x556efc370aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc370aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x556efc323040 tx=0x7f772801cbb0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 shutdown_connections 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x556efc2e1840 0x556efc2e4b50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556efbf13390 0x556efc370aa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556efbef26b0 0x556efc381b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f770f7fe640 0 client.5502 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 --2- 172.21.15.22:0/1708765918 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556efbeea4d0 0x556efc301f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 >> 172.21.15.22:0/1708765918 conn(0x556efbf074e0 msgr2=0x556efc374b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 shutdown_connections 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 1 -- 172.21.15.22:0/1708765918 wait complete. 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f773dfdf740 20 client.5502 trim_cache size 0 max 16384 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout: "0f18ddfb-f9c3-45a0-96fd-1e23c629b28a": { 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "0f18ddfb-f9c3-45a0-96fd-1e23c629b28a", 2024-09-21T05:40:46.686 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,repair" 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "0f18ddfb-f9c3-45a0-96fd-1e23c629b28a": { 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:40:46.504906+0000", 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:40:46.687 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:40:46.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f771c078bc0 msgr2=0x7f771c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f7737077640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f771c078bc0 0x7f771c07b0d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f772c005eb0 tx=0x7f772c005f90 comp rx=0 tx=0).stop 2024-09-21T05:40:46.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 msgr2=0x7f773006def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:40:46.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.679+0000 7f7737077640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 0x7f773006def0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7720009df0 tx=0x7f7720003820 comp rx=0 tx=0).stop 2024-09-21T05:40:46.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 shutdown_connections 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f773006e430 0x7f773006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77300ae9b0 0x7f773006def0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f771c078bc0 0x7f771c07b0d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 --2- 172.21.15.22:0/929104183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7718004800 0x7f77301f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 >> 172.21.15.22:0/929104183 conn(0x7f7730072ae0 msgr2=0x7f77300b5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 shutdown_connections 2024-09-21T05:40:46.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:46.680+0000 7f7737077640 1 -- 172.21.15.22:0/929104183 wait complete. 2024-09-21T05:40:46.693 INFO:tasks.cephfs.filesystem:scrub status for tag:0f18ddfb-f9c3-45a0-96fd-1e23c629b28a - {'path': '/', 'tag': '0f18ddfb-f9c3-45a0-96fd-1e23c629b28a', 'options': 'recursive,repair'} 2024-09-21T05:40:47.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:47.358+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 7 ==== log(6 entries from seq 1 at 2024-09-21T05:40:46.505873+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f08bc009490 con 0x7f08c80b5200 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: cluster: 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: id: cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: health: HEALTH_OK 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: services: 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: mon: 3 daemons, quorum a,b,c (age 109s) 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: mgr: x(active, since 104s), standbys: y 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: mds: 1/1 daemons up, 3 standby 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: osd: 8 osds: 8 up (since 95s), 8 in (since 105s) 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:47.361 INFO:teuthology.orchestra.run.smithi022.stdout: data: 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: volumes: 1/1 healthy 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: pools: 3 pools, 97 pgs 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: objects: 24 objects, 582 KiB 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: usage: 249 MiB used, 720 GiB / 720 GiB avail 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: pgs: 97 active+clean 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: io: 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: client: 2.2 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:46.505873+0000 mds.b [INF] scrub queued for path: / 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:46.505876+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-09-21T05:40:47.362 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:46.505882+0000 mds.b [INF] scrub summary: active paths [/] 2024-09-21T05:40:47.363 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:46.507314+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-09-21T05:40:47.363 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:46.507321+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-09-21T05:40:47.363 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:46.508977+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-09-21T05:40:50.368 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:40:48.680630+0000 mds.b [INF] scrub summary: idle 2024-09-21T05:40:50.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:40:50.366+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 8 ==== log(1 entries from seq 7 at 2024-09-21T05:40:48.680630+0000) ==== 189+0+0 (secure 0 0 0) 0x7f08bc09a300 con 0x7f08c80b5200 2024-09-21T05:40:51.194 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:40:51.198 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:41:02.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:02.389+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 9 ==== log(4 entries from seq 113 at 2024-09-21T05:41:01.750446+0000) ==== 1114+0+0 (secure 0 0 0) 0x7f08bc05fd30 con 0x7f08c80b5200 2024-09-21T05:41:03.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:03.390+0000 7f08c57fa640 1 -- 172.21.15.22:0/693309668 <== mon.0 v2:172.21.15.22:3300/0 10 ==== log(2 entries from seq 388 at 2024-09-21T05:41:02.389359+0000) ==== 556+0+0 (secure 0 0 0) 0x7f08bc063d20 con 0x7f08c80b5200 2024-09-21T05:41:16.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:16.694 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 -- 172.21.15.22:0/103389854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580cc1e0 msgr2=0x7eff580af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 --2- 172.21.15.22:0/103389854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580cc1e0 0x7eff580af3c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7eff48005fd0 tx=0x7eff4802a530 comp rx=0 tx=0).stop 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 -- 172.21.15.22:0/103389854 shutdown_connections 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 --2- 172.21.15.22:0/103389854 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff580ccb50 0x7eff580ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 --2- 172.21.15.22:0/103389854 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580cc1e0 0x7eff580af3c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 -- 172.21.15.22:0/103389854 >> 172.21.15.22:0/103389854 conn(0x7eff5806d7d0 msgr2=0x7eff580b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 -- 172.21.15.22:0/103389854 shutdown_connections 2024-09-21T05:41:16.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.826+0000 7eff5f150640 1 -- 172.21.15.22:0/103389854 wait complete. 2024-09-21T05:41:16.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.828+0000 7eff5f150640 1 Processor -- start 2024-09-21T05:41:16.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.828+0000 7eff5f150640 1 -- start start 2024-09-21T05:41:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5f150640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff580cc1e0 0x7eff58150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5f150640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 0x7eff5815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5f150640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff5815d8a0 0x7eff580cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5f150640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff58153520 con 0x7eff580ccb50 2024-09-21T05:41:16.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5f150640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff581533a0 con 0x7eff580cc1e0 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5f150640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7eff581536a0 con 0x7eff5815d8a0 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 0x7eff5815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 0x7eff5815eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54896/0 (socket says 172.21.15.22:54896) 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 -- 172.21.15.22:0/3075533320 learned_addr learned my addr 172.21.15.22:0/3075533320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5ceec640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff580cc1e0 0x7eff58150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff5815d8a0 0x7eff580cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 -- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff5815d8a0 msgr2=0x7eff580cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff5815d8a0 0x7eff580cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 -- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff580cc1e0 msgr2=0x7eff58150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff580cc1e0 0x7eff58150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.830+0000 7eff57fff640 1 -- 172.21.15.22:0/3075533320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5815fb00 con 0x7eff580ccb50 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.831+0000 7eff57fff640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 0x7eff5815eb80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b420 tx=0x7eff4c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.831+0000 7eff557fa640 1 -- 172.21.15.22:0/3075533320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c011830 con 0x7eff580ccb50 2024-09-21T05:41:16.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.831+0000 7eff557fa640 1 -- 172.21.15.22:0/3075533320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4c016070 con 0x7eff580ccb50 2024-09-21T05:41:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.831+0000 7eff557fa640 1 -- 172.21.15.22:0/3075533320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c011d60 con 0x7eff580ccb50 2024-09-21T05:41:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.831+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff580ca610 con 0x7eff580ccb50 2024-09-21T05:41:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.831+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff580cabd0 con 0x7eff580ccb50 2024-09-21T05:41:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.833+0000 7eff557fa640 1 -- 172.21.15.22:0/3075533320 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7eff4c01b460 con 0x7eff580ccb50 2024-09-21T05:41:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.833+0000 7eff557fa640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7eff44078bc0 0x7eff4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.834+0000 7eff557fa640 1 -- 172.21.15.22:0/3075533320 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7eff4c09a850 con 0x7eff580ccb50 2024-09-21T05:41:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.834+0000 7eff5ceec640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7eff44078bc0 0x7eff4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.834+0000 7eff5ceec640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7eff44078bc0 0x7eff4407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff48033000 tx=0x7eff48004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.834+0000 7eff660ba740 1 Processor -- start 2024-09-21T05:41:16.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.835+0000 7eff660ba740 1 -- start start 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff660ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc5a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc5a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1b76b0 0x55edfc5c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff660ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55edfc4f2a60 con 0x55edfc1d8390 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55edfc4e1e90 con 0x55edfc1af4d0 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55edfc4e5ca0 con 0x55edfc1b76b0 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc5a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc5a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54908/0 (socket says 172.21.15.22:54908) 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 -- 172.21.15.22:0/1817270676 learned_addr learned my addr 172.21.15.22:0/1817270676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff57fff640 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc5a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1b76b0 0x55edfc5c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 -- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1b76b0 msgr2=0x55edfc5c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1b76b0 0x55edfc5c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 -- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 msgr2=0x55edfc5a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc5a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.836+0000 7eff5ceec640 1 -- 172.21.15.22:0/1817270676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfc2ba7d0 con 0x55edfc1d8390 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.837+0000 7eff5ceec640 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc5a9740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff48005f70 tx=0x7eff4803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe08967bd3b769b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.837+0000 7eff3e7fc640 1 -- 172.21.15.22:0/1817270676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48035040 con 0x55edfc1d8390 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.837+0000 7eff3e7fc640 1 -- 172.21.15.22:0/1817270676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4804c070 con 0x55edfc1d8390 2024-09-21T05:41:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.837+0000 7eff3e7fc640 1 -- 172.21.15.22:0/1817270676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48003d00 con 0x55edfc1d8390 2024-09-21T05:41:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.837+0000 7eff660ba740 1 -- 172.21.15.22:0/1817270676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 msgr2=0x55edfc5a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.837+0000 7eff660ba740 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc5a9740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7eff48005f70 tx=0x7eff4803c000 comp rx=0 tx=0).stop 2024-09-21T05:41:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 -- 172.21.15.22:0/1817270676 shutdown_connections 2024-09-21T05:41:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1b76b0 0x55edfc5c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc5a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 --2- 172.21.15.22:0/1817270676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc5a9740 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 -- 172.21.15.22:0/1817270676 >> 172.21.15.22:0/1817270676 conn(0x55edfc1cc4e0 msgr2=0x55edfc57fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 -- 172.21.15.22:0/1817270676 shutdown_connections 2024-09-21T05:41:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 -- 172.21.15.22:0/1817270676 wait complete. 2024-09-21T05:41:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.838+0000 7eff660ba740 1 -- start start 2024-09-21T05:41:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.839+0000 7eff660ba740 1 Processor -- start 2024-09-21T05:41:16.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1af4d0 0x55edfc5d4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc5bc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc597f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55edfc252740 con 0x55edfc1d8390 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55edfc487970 con 0x55edfc1b76b0 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55edfc5aa1c0 con 0x55edfc1af4d0 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff660ba740 10 client.0 _ll_register_callbacks cb 0x55edfc48abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff5d6ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc597f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff5d6ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc597f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54912/0 (socket says 172.21.15.22:54912) 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff5d6ed640 1 -- 172.21.15.22:0/181169327 learned_addr learned my addr 172.21.15.22:0/181169327 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff5ceec640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1af4d0 0x55edfc5d4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.840+0000 7eff57fff640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc5bc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff5d6ed640 1 -- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1af4d0 msgr2=0x55edfc5d4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1af4d0 0x55edfc5d4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff5d6ed640 1 -- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 msgr2=0x55edfc5bc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc5bc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff5d6ed640 1 -- 172.21.15.22:0/181169327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfc243970 con 0x55edfc1d8390 2024-09-21T05:41:16.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc597f60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7eff50012840 tx=0x7eff50012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb775a5983e71b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff377fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff377fe640 1 -- 172.21.15.22:0/181169327 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5000cd20 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff377fe640 1 -- 172.21.15.22:0/181169327 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7eff5001cb30 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4804c070 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff377fe640 1 -- 172.21.15.22:0/181169327 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff5001cd30 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 10 client.5514 fetch_fsmap learned FSMap version 26 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55edfc2216a0 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff377fe640 1 -- 172.21.15.22:0/181169327 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4804c070 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff377fe640 1 -- 172.21.15.22:0/181169327 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7eff50023050 con 0x55edfc1d8390 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 10 client.5514 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 10 client.5514 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 20 client.5514 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc196f60 0x55edfc5bbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 4 client.5514 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:41:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.841+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edfc4f49b0 con 0x55edfc196f60 2024-09-21T05:41:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.842+0000 7eff5ceec640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc196f60 0x55edfc5bbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.842+0000 7eff377fe640 10 client.5514 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:41:16.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.842+0000 7eff5ceec640 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc196f60 0x55edfc5bbb30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758625bc9e181ec8 server_cookie=b6bbc209e693fd85 in_seq=0 out_seq=0 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff377fe640 1 -- 172.21.15.22:0/181169327 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55edfc4f49b0 con 0x55edfc196f60 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff377fe640 10 client.5514 handle_command_reply: tid=0 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 client.5514 shutdown 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 msgr2=0x55edfc597f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc597f60 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7eff50012840 tx=0x7eff50012c40 comp rx=0 tx=0).stop 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 shutdown_connections 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc196f60 0x55edfc5bbb30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1d8390 0x55edfc597f60 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc5bc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff377fe640 0 client.5514 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:41:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 --2- 172.21.15.22:0/181169327 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1af4d0 0x55edfc5d4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 >> 172.21.15.22:0/181169327 conn(0x55edfc1cc4e0 msgr2=0x55edfc5c6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 shutdown_connections 2024-09-21T05:41:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 1 -- 172.21.15.22:0/181169327 wait complete. 2024-09-21T05:41:16.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.843+0000 7eff660ba740 20 client.5514 trim_cache size 0 max 16384 2024-09-21T05:41:16.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.861+0000 7eff660ba740 1 Processor -- start 2024-09-21T05:41:16.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.861+0000 7eff660ba740 1 -- start start 2024-09-21T05:41:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc597fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff660ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff660ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff58202330 con 0x55edfc1b76b0 2024-09-21T05:41:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7eff582014c0 con 0x55edfc1af4d0 2024-09-21T05:41:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7eff581f9390 con 0x55edfc1d8390 2024-09-21T05:41:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54918/0 (socket says 172.21.15.22:54918) 2024-09-21T05:41:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 -- 172.21.15.22:0/1931112724 learned_addr learned my addr 172.21.15.22:0/1931112724 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 -- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 msgr2=0x55edfc579230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff5ceec640 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc597fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 -- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 msgr2=0x55edfc597fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc597fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 -- 172.21.15.22:0/1931112724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfc581110 con 0x55edfc1b76b0 2024-09-21T05:41:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff57fff640 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b3f0 tx=0x7eff4c062420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0966767e26a237 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.863+0000 7eff3dffb640 1 -- 172.21.15.22:0/1931112724 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c06bed0 con 0x55edfc1b76b0 2024-09-21T05:41:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff3dffb640 1 -- 172.21.15.22:0/1931112724 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4c0617a0 con 0x55edfc1b76b0 2024-09-21T05:41:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff3dffb640 1 -- 172.21.15.22:0/1931112724 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4c069220 con 0x55edfc1b76b0 2024-09-21T05:41:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 -- 172.21.15.22:0/1931112724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 msgr2=0x55edfc578cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b3f0 tx=0x7eff4c062420 comp rx=0 tx=0).stop 2024-09-21T05:41:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 -- 172.21.15.22:0/1931112724 shutdown_connections 2024-09-21T05:41:16.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 --2- 172.21.15.22:0/1931112724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1af4d0 0x55edfc597fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 -- 172.21.15.22:0/1931112724 >> 172.21.15.22:0/1931112724 conn(0x55edfc1cc4e0 msgr2=0x55edfc580a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 -- 172.21.15.22:0/1931112724 shutdown_connections 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.864+0000 7eff660ba740 1 -- 172.21.15.22:0/1931112724 wait complete. 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.865+0000 7eff660ba740 1 -- start start 2024-09-21T05:41:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.865+0000 7eff660ba740 1 Processor -- start 2024-09-21T05:41:16.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 0x55edfc6430c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7eff581f99e0 con 0x55edfc1af4d0 2024-09-21T05:41:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55edfc57ba40 con 0x55edfc1b76b0 2024-09-21T05:41:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55edfc5801a0 con 0x55edfc1d8390 2024-09-21T05:41:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff660ba740 10 client.0 _ll_register_callbacks cb 0x7eff500099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:16.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 0x55edfc6430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 0x55edfc6430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54930/0 (socket says 172.21.15.22:54930) 2024-09-21T05:41:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 -- 172.21.15.22:0/3128658304 learned_addr learned my addr 172.21.15.22:0/3128658304 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff57fff640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 -- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 msgr2=0x55edfc579230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 -- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 msgr2=0x55edfc578cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 -- 172.21.15.22:0/3128658304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfc5dc8e0 con 0x55edfc1af4d0 2024-09-21T05:41:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5d6ed640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff57fff640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:16.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff5ceec640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 0x55edfc6430c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7eff48000c00 tx=0x7eff480035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d27c6a268254417 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.867+0000 7eff377fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff377fe640 1 -- 172.21.15.22:0/3128658304 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480047a0 con 0x55edfc1af4d0 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff377fe640 1 -- 172.21.15.22:0/3128658304 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7eff48004940 con 0x55edfc1af4d0 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4c0617a0 con 0x55edfc1af4d0 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff377fe640 1 -- 172.21.15.22:0/3128658304 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7eff48069050 con 0x55edfc1af4d0 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 10 client.5520 fetch_fsmap learned FSMap version 26 2024-09-21T05:41:16.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55edfc6436c0 con 0x55edfc1af4d0 2024-09-21T05:41:16.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff377fe640 1 -- 172.21.15.22:0/3128658304 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff4c0617a0 con 0x55edfc1af4d0 2024-09-21T05:41:16.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff377fe640 1 -- 172.21.15.22:0/3128658304 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7eff4806c020 con 0x55edfc1af4d0 2024-09-21T05:41:16.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 10 client.5520 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T05:41:16.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 10 client.5520 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-21T05:41:16.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 20 client.5520 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:16.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc5a6ee0 0x55edfc5bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 4 client.5520 mds_command: new command op to 5298 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:41:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 --> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55edfc4f0460 con 0x55edfc5a6ee0 2024-09-21T05:41:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.868+0000 7eff57fff640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc5a6ee0 0x55edfc5bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff57fff640 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc5a6ee0 0x55edfc5bffd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c92f5b6b6386d55 server_cookie=d9dee9ac0c057c7 in_seq=0 out_seq=0 2024-09-21T05:41:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff377fe640 10 client.5520 ms_handle_connect on v2:172.21.15.134:6835/441099152 2024-09-21T05:41:16.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff377fe640 1 -- 172.21.15.22:0/3128658304 <== mds.0 v2:172.21.15.134:6835/441099152 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55edfc4f0460 con 0x55edfc5a6ee0 2024-09-21T05:41:16.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff377fe640 10 client.5520 handle_command_reply: tid=0 2024-09-21T05:41:16.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 client.5520 shutdown 2024-09-21T05:41:16.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 msgr2=0x55edfc6430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 0x55edfc6430c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7eff48000c00 tx=0x7eff480035f0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 shutdown_connections 2024-09-21T05:41:16.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:6835/441099152,v1:172.21.15.134:6837/441099152] conn(0x55edfc5a6ee0 0x55edfc5bffd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55edfc1d8390 0x55edfc579230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55edfc1b76b0 0x55edfc578cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff377fe640 0 client.5520 ms_handle_reset on v2:172.21.15.134:6835/441099152 2024-09-21T05:41:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 --2- 172.21.15.22:0/3128658304 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55edfc1af4d0 0x55edfc6430c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 >> 172.21.15.22:0/3128658304 conn(0x55edfc1cc4e0 msgr2=0x55edfc57a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:16.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 shutdown_connections 2024-09-21T05:41:16.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 1 -- 172.21.15.22:0/3128658304 wait complete. 2024-09-21T05:41:16.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.869+0000 7eff660ba740 20 client.5520 trim_cache size 0 max 16384 2024-09-21T05:41:16.888 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:16.888 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "0f18ddfb-f9c3-45a0-96fd-1e23c629b28a": { 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:40:46.504906+0000", 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:41:16.889 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:41:16.890 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:41:16.890 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:41:16.890 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:16.890 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:16.890 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7eff44078bc0 msgr2=0x7eff4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7eff44078bc0 0x7eff4407b0d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff48033000 tx=0x7eff48004730 comp rx=0 tx=0).stop 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 msgr2=0x7eff5815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 0x7eff5815eb80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b420 tx=0x7eff4c00b8e0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 shutdown_connections 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7eff44078bc0 0x7eff4407b0d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7eff5815d8a0 0x7eff580cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7eff580ccb50 0x7eff5815eb80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 --2- 172.21.15.22:0/3075533320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7eff580cc1e0 0x7eff58150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 >> 172.21.15.22:0/3075533320 conn(0x7eff5806d7d0 msgr2=0x7eff580b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 shutdown_connections 2024-09-21T05:41:16.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:16.870+0000 7eff5f150640 1 -- 172.21.15.22:0/3075533320 wait complete. 2024-09-21T05:41:16.892 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:41:16.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:17.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 -- 172.21.15.22:0/2822298442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c00d4d80 msgr2=0x7fc0c00cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 --2- 172.21.15.22:0/2822298442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c00d4d80 0x7fc0c00cf540 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009860 tx=0x7fc0b802a4d0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 -- 172.21.15.22:0/2822298442 shutdown_connections 2024-09-21T05:41:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 --2- 172.21.15.22:0/2822298442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c00d4d80 0x7fc0c00cf540 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 -- 172.21.15.22:0/2822298442 >> 172.21.15.22:0/2822298442 conn(0x7fc0c009d7f0 msgr2=0x7fc0c00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 -- 172.21.15.22:0/2822298442 shutdown_connections 2024-09-21T05:41:17.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.108+0000 7fc0c5382640 1 -- 172.21.15.22:0/2822298442 wait complete. 2024-09-21T05:41:17.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.110+0000 7fc0c5382640 1 Processor -- start 2024-09-21T05:41:17.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.110+0000 7fc0c5382640 1 -- start start 2024-09-21T05:41:17.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0c5382640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00b9b60 0x7fc0c015f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0c5382640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0c00d4d80 0x7fc0c015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0c5382640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 0x7fc0c00c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0c5382640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c006f830 con 0x7fc0c01576c0 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0c5382640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c006f6b0 con 0x7fc0c00b9b60 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0c5382640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc0c006f9b0 con 0x7fc0c00d4d80 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 0x7fc0c00c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0bf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 0x7fc0c00c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54944/0 (socket says 172.21.15.22:54944) 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0bf7fe640 1 -- 172.21.15.22:0/3238974739 learned_addr learned my addr 172.21.15.22:0/3238974739 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0be7fc640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0c00d4d80 0x7fc0c015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.111+0000 7fc0beffd640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00b9b60 0x7fc0c015f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0bf7fe640 1 -- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0c00d4d80 msgr2=0x7fc0c015fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0bf7fe640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0c00d4d80 0x7fc0c015fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0bf7fe640 1 -- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00b9b60 msgr2=0x7fc0c015f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0bf7fe640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00b9b60 0x7fc0c015f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0bf7fe640 1 -- 172.21.15.22:0/3238974739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c0157480 con 0x7fc0c01576c0 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0beffd640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00b9b60 0x7fc0c015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0be7fc640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0c00d4d80 0x7fc0c015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0bf7fe640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 0x7fc0c00c8050 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009950 tx=0x7fc0b8033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8031b50 con 0x7fc0c01576c0 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0b8031de0 con 0x7fc0c01576c0 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8033330 con 0x7fc0c01576c0 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c00ca820 con 0x7fc0c01576c0 2024-09-21T05:41:17.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.112+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c015f230 con 0x7fc0c01576c0 2024-09-21T05:41:17.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.113+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c00cf540 con 0x7fc0c01576c0 2024-09-21T05:41:17.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.114+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0b80334d0 con 0x7fc0c01576c0 2024-09-21T05:41:17.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.114+0000 7fc0b77fe640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc0a0078b90 0x7fc0a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.114+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fc0b80bc090 con 0x7fc0c01576c0 2024-09-21T05:41:17.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.115+0000 7fc0beffd640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc0a0078b90 0x7fc0a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.115+0000 7fc0beffd640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc0a0078b90 0x7fc0a007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0001410 tx=0x7fc0b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:17.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.119+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc0b803f030 con 0x7fc0c01576c0 2024-09-21T05:41:17.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.348+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0c00d3fb0 con 0x7fc0c01576c0 2024-09-21T05:41:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.349+0000 7fc0b77fe640 1 -- 172.21.15.22:0/3238974739 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fc0b8084520 con 0x7fc0c01576c0 2024-09-21T05:41:17.351 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:17.351 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:41:17.351 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc0a0078b90 msgr2=0x7fc0a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc0a0078b90 0x7fc0a007b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0001410 tx=0x7fc0b0009040 comp rx=0 tx=0).stop 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 msgr2=0x7fc0c00c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 0x7fc0c00c8050 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009950 tx=0x7fc0b8033ba0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 shutdown_connections 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0c01576c0 0x7fc0c00c8050 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc0a0078b90 0x7fc0a007b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0c00d4d80 0x7fc0c015fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 --2- 172.21.15.22:0/3238974739 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0c00b9b60 0x7fc0c015f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 >> 172.21.15.22:0/3238974739 conn(0x7fc0c009d7f0 msgr2=0x7fc0c00b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 shutdown_connections 2024-09-21T05:41:17.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.351+0000 7fc0c5382640 1 -- 172.21.15.22:0/3238974739 wait complete. 2024-09-21T05:41:17.365 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-21T05:41:17.508 INFO:teuthology.orchestra.run.smithi134.stdout:{"message":"","return_code":0} 2024-09-21T05:41:17.508 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:41:17.508 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-21T05:41:17.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.563+0000 7fa746577f40 1 Processor -- start 2024-09-21T05:41:17.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.563+0000 7fa746577f40 1 -- start start 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa746577f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a1779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa746577f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a6ea0 0x558c1a176810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa746577f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a8e60 0x558c1a176d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa746577f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558c1a055430 con 0x558c1a0a53d0 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa746577f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c1a0621f0 con 0x558c1a0a6ea0 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa746577f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558c19ff6d60 con 0x558c1a0a8e60 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a1779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a1779e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54954/0 (socket says 172.21.15.22:54954) 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 -- 172.21.15.22:0/2688511841 learned_addr learned my addr 172.21.15.22:0/2688511841 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa73f830640 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a6ea0 0x558c1a176810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740832640 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a8e60 0x558c1a176d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 -- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a8e60 msgr2=0x558c1a176d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a8e60 0x558c1a176d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 -- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a6ea0 msgr2=0x558c1a176810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a6ea0 0x558c1a176810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.564+0000 7fa740031640 1 -- 172.21.15.22:0/2688511841 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c1a1756e0 con 0x558c1a0a53d0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa740031640 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a1779e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa730046700 tx=0x7fa730002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d933f3885ea6ae3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa73e82e640 1 -- 172.21.15.22:0/2688511841 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730066070 con 0x558c1a0a53d0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa73e82e640 1 -- 172.21.15.22:0/2688511841 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa730056470 con 0x558c1a0a53d0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa73e82e640 1 -- 172.21.15.22:0/2688511841 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7300568c0 con 0x558c1a0a53d0 2024-09-21T05:41:17.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 -- 172.21.15.22:0/2688511841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 msgr2=0x558c1a1779e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a1779e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa730046700 tx=0x7fa730002fb0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 -- 172.21.15.22:0/2688511841 shutdown_connections 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a8e60 0x558c1a176d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a6ea0 0x558c1a176810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 --2- 172.21.15.22:0/2688511841 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a1779e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.565+0000 7fa746577f40 1 -- 172.21.15.22:0/2688511841 >> 172.21.15.22:0/2688511841 conn(0x558c19fc7ac0 msgr2=0x558c1a0a72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.566+0000 7fa746577f40 1 -- 172.21.15.22:0/2688511841 shutdown_connections 2024-09-21T05:41:17.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.566+0000 7fa746577f40 1 -- 172.21.15.22:0/2688511841 wait complete. 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.566+0000 7fa746577f40 1 Processor -- start 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.566+0000 7fa746577f40 1 -- start start 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa746577f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a172450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa746577f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a6ea0 0x558c1a0d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa746577f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a8e60 0x558c1a0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa746577f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558c1a176650 con 0x558c1a0a53d0 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa746577f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558c1a0d4550 con 0x558c1a0a8e60 2024-09-21T05:41:17.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa746577f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558c1a215a10 con 0x558c1a0a6ea0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a172450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a172450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54970/0 (socket says 172.21.15.22:54970) 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 -- 172.21.15.22:0/2862072359 learned_addr learned my addr 172.21.15.22:0/2862072359 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740832640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a8e60 0x558c1a0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa73f830640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a6ea0 0x558c1a0d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 -- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a6ea0 msgr2=0x558c1a0d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558c1a0a6ea0 0x558c1a0d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 -- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a8e60 msgr2=0x558c1a0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558c1a0a8e60 0x558c1a0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 -- 172.21.15.22:0/2862072359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c1a219e00 con 0x558c1a0a53d0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.568+0000 7fa740031640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a172450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa730004e90 tx=0x7fa730056980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.569+0000 7fa73d82c640 1 -- 172.21.15.22:0/2862072359 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730066040 con 0x558c1a0a53d0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.569+0000 7fa73d82c640 1 -- 172.21.15.22:0/2862072359 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa730056c60 con 0x558c1a0a53d0 2024-09-21T05:41:17.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.569+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558c1a176e50 con 0x558c1a0a53d0 2024-09-21T05:41:17.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.569+0000 7fa73d82c640 1 -- 172.21.15.22:0/2862072359 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73006b6e0 con 0x558c1a0a53d0 2024-09-21T05:41:17.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.569+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c1a0eb650 con 0x558c1a0a53d0 2024-09-21T05:41:17.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.570+0000 7fa73d82c640 1 -- 172.21.15.22:0/2862072359 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa730069070 con 0x558c1a0a53d0 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.571+0000 7fa73d82c640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa72c0775a0 0x7fa72c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.571+0000 7fa73d82c640 1 -- 172.21.15.22:0/2862072359 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fa7300df3f0 con 0x558c1a0a53d0 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.571+0000 7fa746577f40 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x558c1a219890 0x558c1a222090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.571+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45) -- 0x558c1a0f19c0 con 0x558c1a219890 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.571+0000 7fa73f830640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa72c0775a0 0x7fa72c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.571+0000 7fa740832640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x558c1a219890 0x558c1a222090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa73f830640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa72c0775a0 0x7fa72c079ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x558c1a17ba40 tx=0x7fa734002ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa740832640 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x558c1a219890 0x558c1a222090 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa740832640 1 -- 172.21.15.22:0/2862072359 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fa7380025f0 con 0x558c1a219890 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x558c1a219890 msgr2=0x558c1a222090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x558c1a219890 0x558c1a222090 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa72c0775a0 msgr2=0x7fa72c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa72c0775a0 0x7fa72c079ab0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x558c1a17ba40 tx=0x7fa734002ad0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 msgr2=0x558c1a172450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 --2- 172.21.15.22:0/2862072359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558c1a0a53d0 0x558c1a172450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa730004e90 tx=0x7fa730056980 comp rx=0 tx=0).stop 2024-09-21T05:41:17.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa740832640 1 -- 172.21.15.22:0/2862072359 reap_dead start 2024-09-21T05:41:17.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 shutdown_connections 2024-09-21T05:41:17.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 >> 172.21.15.22:0/2862072359 conn(0x558c19fc7ac0 msgr2=0x558c1a178c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:17.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 shutdown_connections 2024-09-21T05:41:17.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.572+0000 7fa746577f40 1 -- 172.21.15.22:0/2862072359 wait complete. 2024-09-21T05:41:17.578 INFO:teuthology.orchestra.run.smithi022.stdout:Yalphaparent_a 2024-09-21T05:41:17.578 DEBUG:teuthology.orchestra.run.smithi022:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627777, 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "ancestors": [ 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "dirino": 1099511627776, 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "alpha", 2024-09-21T05:41:17.634 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 4 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "dirino": 1, 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "parent_a", 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 7 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 7, 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: "old_pools": [] 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:17.635 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:17.637 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:41:17.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:41:17.724 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:41:17.765 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:41:17.765 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:41:17.765 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:41:17.765 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:17.781 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:41:17.782 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:17.839 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:41:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 -- 172.21.15.22:0/1972971798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82741600b0 msgr2=0x7f82740ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 --2- 172.21.15.22:0/1972971798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82741600b0 0x7f82740ccbc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f82740ce1b0 tx=0x7f826402aa30 comp rx=0 tx=0).stop 2024-09-21T05:41:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 -- 172.21.15.22:0/1972971798 shutdown_connections 2024-09-21T05:41:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 --2- 172.21.15.22:0/1972971798 >> v2:172.21.15.134:3300/0 conn(0x7f82740b7cc0 0x7f82740cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 --2- 172.21.15.22:0/1972971798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82741600b0 0x7f82740ccbc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 -- 172.21.15.22:0/1972971798 >> 172.21.15.22:0/1972971798 conn(0x7f827406d4f0 msgr2=0x7f8274071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 -- 172.21.15.22:0/1972971798 shutdown_connections 2024-09-21T05:41:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.994+0000 7f8279112640 1 -- 172.21.15.22:0/1972971798 wait complete. 2024-09-21T05:41:17.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.996+0000 7f8279112640 1 Processor -- start 2024-09-21T05:41:17.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.996+0000 7f8279112640 1 -- start start 2024-09-21T05:41:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8279112640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 0x7f8274151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8279112640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82740cf150 0x7f82740cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8279112640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82741600b0 0x7f827415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8279112640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82740b4460 con 0x7f82740b7cc0 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8279112640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82740d1ba0 con 0x7f82741600b0 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8279112640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f82740b45e0 con 0x7f82740cf150 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.997+0000 7f8272d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 0x7f8274151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 0x7f8274151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54988/0 (socket says 172.21.15.22:54988) 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 -- 172.21.15.22:0/1809658813 learned_addr learned my addr 172.21.15.22:0/1809658813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f827359e640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82741600b0 0x7f827415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f827259c640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82740cf150 0x7f82740cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 -- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82740cf150 msgr2=0x7f82740cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82740cf150 0x7f82740cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 -- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82741600b0 msgr2=0x7f827415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82741600b0 0x7f827415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 -- 172.21.15.22:0/1809658813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8274151e90 con 0x7f82740b7cc0 2024-09-21T05:41:18.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8272d9d640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 0x7f8274151950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8264009d60 tx=0x7f8264003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:18.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264033050 con 0x7f82740b7cc0 2024-09-21T05:41:18.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8264004550 con 0x7f82740b7cc0 2024-09-21T05:41:18.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82740cb7e0 con 0x7f82740b7cc0 2024-09-21T05:41:18.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264038450 con 0x7f82740b7cc0 2024-09-21T05:41:18.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:17.998+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827406e840 con 0x7f82740b7cc0 2024-09-21T05:41:18.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.000+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8274076800 con 0x7f82740b7cc0 2024-09-21T05:41:18.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.000+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8264036070 con 0x7f82740b7cc0 2024-09-21T05:41:18.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.001+0000 7f825b7fe640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8260078bc0 0x7f826007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.001+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f82640bc370 con 0x7f82740b7cc0 2024-09-21T05:41:18.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.001+0000 7f827259c640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8260078bc0 0x7f826007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.002+0000 7f827259c640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8260078bc0 0x7f826007b0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82680058c0 tx=0x7f8268005f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:18.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.006+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8274076800 con 0x7f82740b7cc0 2024-09-21T05:41:18.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.206+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f82740cec10 con 0x7f82740b7cc0 2024-09-21T05:41:18.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.206+0000 7f825b7fe640 1 -- 172.21.15.22:0/1809658813 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f82740cb7e0 con 0x7f82740b7cc0 2024-09-21T05:41:18.209 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:41:18.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8260078bc0 msgr2=0x7f826007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8260078bc0 0x7f826007b0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82680058c0 tx=0x7f8268005f60 comp rx=0 tx=0).stop 2024-09-21T05:41:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 msgr2=0x7f8274151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 0x7f8274151950 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8264009d60 tx=0x7f8264003f10 comp rx=0 tx=0).stop 2024-09-21T05:41:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 shutdown_connections 2024-09-21T05:41:18.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82741600b0 0x7f827415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8260078bc0 0x7f826007b0d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82740cf150 0x7f82740cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 --2- 172.21.15.22:0/1809658813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82740b7cc0 0x7f8274151950 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 >> 172.21.15.22:0/1809658813 conn(0x7f827406d4f0 msgr2=0x7f82740b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 shutdown_connections 2024-09-21T05:41:18.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.209+0000 7f8279112640 1 -- 172.21.15.22:0/1809658813 wait complete. 2024-09-21T05:41:18.222 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:41:18.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:18.222 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:41:18.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:41:18.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:18.273 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:41:18.301 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:41:18.301 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:41:18.301 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:18.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:41:18.319 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:41:18.319 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:41:18.319 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:18.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:41:18.358 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:41:18.358 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:41:18.358 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:18.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:41:18.375 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:18.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 -- 172.21.15.22:0/3307986655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b66b0 msgr2=0x7f84a80d3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3307986655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b66b0 0x7f84a80d3530 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f84a0005fd0 tx=0x7f84a002aa00 comp rx=0 tx=0).stop 2024-09-21T05:41:18.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 -- 172.21.15.22:0/3307986655 shutdown_connections 2024-09-21T05:41:18.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3307986655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b66b0 0x7f84a80d3530 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3307986655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84a80b9d00 0x7f84a80cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 -- 172.21.15.22:0/3307986655 >> 172.21.15.22:0/3307986655 conn(0x7f84a809d870 msgr2=0x7f84a80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 -- 172.21.15.22:0/3307986655 shutdown_connections 2024-09-21T05:41:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.512+0000 7f84aeba1640 1 -- 172.21.15.22:0/3307986655 wait complete. 2024-09-21T05:41:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.514+0000 7f84aeba1640 1 Processor -- start 2024-09-21T05:41:18.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.514+0000 7f84aeba1640 1 -- start start 2024-09-21T05:41:18.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84aeba1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 0x7f84a815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84aeba1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84a81620e0 0x7f84a80b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84aeba1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a81f5330 0x7f84a80c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84aeba1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84a806e450 con 0x7f84a80b9d00 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84aeba1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84a806e2d0 con 0x7f84a81f5330 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84aeba1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f84a806e5d0 con 0x7f84a81620e0 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84ac93d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 0x7f84a815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.515+0000 7f84ac93d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 0x7f84a815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54998/0 (socket says 172.21.15.22:54998) 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 -- 172.21.15.22:0/3628540226 learned_addr learned my addr 172.21.15.22:0/3628540226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ad13e640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a81f5330 0x7f84a80c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84a7fff640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84a81620e0 0x7f84a80b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 -- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84a81620e0 msgr2=0x7f84a80b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84a81620e0 0x7f84a80b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 -- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a81f5330 msgr2=0x7f84a80c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a81f5330 0x7f84a80c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 -- 172.21.15.22:0/3628540226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a81f5f60 con 0x7f84a80b9d00 2024-09-21T05:41:18.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84ac93d640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 0x7f84a815bbc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f849800d700 tx=0x7f849800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8498017070 con 0x7f84a80b9d00 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8498009d20 con 0x7f84a80b9d00 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980106c0 con 0x7f84a80b9d00 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a81f6240 con 0x7f84a80b9d00 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.516+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a80b14b0 con 0x7f84a80b9d00 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.517+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a806eaf0 con 0x7f84a80b9d00 2024-09-21T05:41:18.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.518+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f849801a070 con 0x7f84a80b9d00 2024-09-21T05:41:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.519+0000 7f84a57fa640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8494078b90 0x7f849407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.519+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f849809af50 con 0x7f84a80b9d00 2024-09-21T05:41:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.519+0000 7f84a7fff640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8494078b90 0x7f849407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.520+0000 7f84a7fff640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8494078b90 0x7f849407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84a80cdfa0 tx=0x7f849c009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:18.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.523+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8498062c70 con 0x7f84a80b9d00 2024-09-21T05:41:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.750+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84a80d45c0 con 0x7f84a80b9d00 2024-09-21T05:41:18.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.751+0000 7f84a57fa640 1 -- 172.21.15.22:0/3628540226 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f84980676a0 con 0x7f84a80b9d00 2024-09-21T05:41:18.753 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:18.753 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:41:18.753 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:41:18.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8494078b90 msgr2=0x7f849407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8494078b90 0x7f849407b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f84a80cdfa0 tx=0x7f849c009060 comp rx=0 tx=0).stop 2024-09-21T05:41:18.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 msgr2=0x7f84a815bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 0x7f84a815bbc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f849800d700 tx=0x7f849800dbc0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 shutdown_connections 2024-09-21T05:41:18.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84a81f5330 0x7f84a80c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8494078b90 0x7f849407b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84a81620e0 0x7f84a80b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 --2- 172.21.15.22:0/3628540226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84a80b9d00 0x7f84a815bbc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 >> 172.21.15.22:0/3628540226 conn(0x7f84a809d870 msgr2=0x7f84a815f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:18.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 shutdown_connections 2024-09-21T05:41:18.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.753+0000 7f84aeba1640 1 -- 172.21.15.22:0/3628540226 wait complete. 2024-09-21T05:41:18.767 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 -- 172.21.15.22:0/1675898315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0198005250 msgr2=0x7f019800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 --2- 172.21.15.22:0/1675898315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0198005250 0x7f019800b9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f01a8005fd0 tx=0x7f0198005830 comp rx=0 tx=0).stop 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 -- 172.21.15.22:0/1675898315 shutdown_connections 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 --2- 172.21.15.22:0/1675898315 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0198005250 0x7f019800b9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 -- 172.21.15.22:0/1675898315 >> 172.21.15.22:0/1675898315 conn(0x7f01b00b1cb0 msgr2=0x7f01b00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 -- 172.21.15.22:0/1675898315 shutdown_connections 2024-09-21T05:41:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.882+0000 7f01b7015640 1 -- 172.21.15.22:0/1675898315 wait complete. 2024-09-21T05:41:18.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.884+0000 7f01b7015640 1 Processor -- start 2024-09-21T05:41:18.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.884+0000 7f01b7015640 1 -- start start 2024-09-21T05:41:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b7015640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0198005250 0x7f01b015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b7015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 0x7f01b01f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b7015640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01b009c000 0x7f01b009c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b7015640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0074640 con 0x7f01b0157a40 2024-09-21T05:41:18.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b7015640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01b015cc60 con 0x7f01b009c000 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b7015640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01b00747c0 con 0x7f0198005250 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 0x7f01b01f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 0x7f01b01f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55018/0 (socket says 172.21.15.22:55018) 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 -- 172.21.15.22:0/3111315323 learned_addr learned my addr 172.21.15.22:0/3111315323 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b4db1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0198005250 0x7f01b015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01b55b2640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01b009c000 0x7f01b009c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 -- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0198005250 msgr2=0x7f01b015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0198005250 0x7f01b015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 -- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01b009c000 msgr2=0x7f01b009c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01b009c000 0x7f01b009c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 -- 172.21.15.22:0/3111315323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b009c950 con 0x7f01b0157a40 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.886+0000 7f01affff640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 0x7f01b01f34c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f01a4006680 tx=0x7f01a40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:18.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.887+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4015070 con 0x7f01b0157a40 2024-09-21T05:41:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.887+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01a4011840 con 0x7f01b0157a40 2024-09-21T05:41:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.887+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b01f4380 con 0x7f01b0157a40 2024-09-21T05:41:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.887+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4011c90 con 0x7f01b0157a40 2024-09-21T05:41:18.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.887+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b015a590 con 0x7f01b0157a40 2024-09-21T05:41:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.888+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b01f3e10 con 0x7f01b0157a40 2024-09-21T05:41:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.889+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01a4018070 con 0x7f01b0157a40 2024-09-21T05:41:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.889+0000 7f01ad7fa640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f019c078bc0 0x7f019c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.889+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f01a409b9e0 con 0x7f01b0157a40 2024-09-21T05:41:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.889+0000 7f01b4db1640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f019c078bc0 0x7f019c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:18.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.890+0000 7f01b4db1640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f019c078bc0 0x7f019c07b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01a0000c00 tx=0x7f01a0006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:18.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:18.894+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01b009c950 con 0x7f01b0157a40 2024-09-21T05:41:19.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.119+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01b0157e50 con 0x7f01b0157a40 2024-09-21T05:41:19.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.120+0000 7f01ad7fa640 1 -- 172.21.15.22:0/3111315323 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f01a40637a0 con 0x7f01b0157a40 2024-09-21T05:41:19.122 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:19.122 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:41:19.122 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:41:19.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f019c078bc0 msgr2=0x7f019c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f019c078bc0 0x7f019c07b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01a0000c00 tx=0x7f01a0006b70 comp rx=0 tx=0).stop 2024-09-21T05:41:19.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 msgr2=0x7f01b01f34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 0x7f01b01f34c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f01a4006680 tx=0x7f01a40039c0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 shutdown_connections 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01b009c000 0x7f01b009c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01b0157a40 0x7f01b01f34c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f019c078bc0 0x7f019c07b0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 --2- 172.21.15.22:0/3111315323 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0198005250 0x7f01b015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 >> 172.21.15.22:0/3111315323 conn(0x7f01b00b1cb0 msgr2=0x7f01b00aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 shutdown_connections 2024-09-21T05:41:19.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.122+0000 7f01b7015640 1 -- 172.21.15.22:0/3111315323 wait complete. 2024-09-21T05:41:19.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/1671163500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180cffd0 msgr2=0x7f8f180d4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/1671163500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180cffd0 0x7f8f180d4940 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8f10009830 tx=0x7f8f1002a480 comp rx=0 tx=0).stop 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/1671163500 shutdown_connections 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/1671163500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180cffd0 0x7f8f180d4940 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/1671163500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f180b3910 0x7f8f180b4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/1671163500 >> 172.21.15.22:0/1671163500 conn(0x7f8f1808bfc0 msgr2=0x7f8f1809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/1671163500 shutdown_connections 2024-09-21T05:41:19.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.249+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/1671163500 wait complete. 2024-09-21T05:41:19.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.251+0000 7f8f1ce4d640 1 Processor -- start 2024-09-21T05:41:19.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.251+0000 7f8f1ce4d640 1 -- start start 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 0x7f8f181f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f180cfbc0 0x7f8f18158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f180cffd0 0x7f8f180b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d5000 con 0x7f8f180b3910 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f180d4e80 con 0x7f8f180cfbc0 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8f180b3340 con 0x7f8f180cffd0 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 0x7f8f181f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 0x7f8f181f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55030/0 (socket says 172.21.15.22:55030) 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 -- 172.21.15.22:0/696745665 learned_addr learned my addr 172.21.15.22:0/696745665 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f16d9d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f180cffd0 0x7f8f180b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f15d9b640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f180cfbc0 0x7f8f18158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 -- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f180cffd0 msgr2=0x7f8f180b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f180cffd0 0x7f8f180b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 -- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f180cfbc0 msgr2=0x7f8f18158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f180cfbc0 0x7f8f18158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 -- 172.21.15.22:0/696745665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f180c9390 con 0x7f8f180b3910 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f15d9b640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f180cfbc0 0x7f8f18158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1659c640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 0x7f8f181f6360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800eb40 tx=0x7f8f0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08010430 con 0x7f8f180b3910 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f080106c0 con 0x7f8f180b3910 2024-09-21T05:41:19.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f180c9670 con 0x7f8f180b3910 2024-09-21T05:41:19.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.253+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f08010af0 con 0x7f8f180b3910 2024-09-21T05:41:19.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.254+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f180c9c30 con 0x7f8f180b3910 2024-09-21T05:41:19.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.255+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f180b5bf0 con 0x7f8f180b3910 2024-09-21T05:41:19.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.255+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f08010c90 con 0x7f8f180b3910 2024-09-21T05:41:19.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.256+0000 7f8f06ffd640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ef8078b90 0x7f8ef807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.256+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f8f0809bb90 con 0x7f8f180b3910 2024-09-21T05:41:19.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.256+0000 7f8f15d9b640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ef8078b90 0x7f8ef807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.257+0000 7f8f15d9b640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ef8078b90 0x7f8ef807b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b9870 tx=0x7f8f0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:19.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.260+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f080638c0 con 0x7f8f180b3910 2024-09-21T05:41:19.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.448+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f180b4230 con 0x7f8f180b3910 2024-09-21T05:41:19.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.449+0000 7f8f06ffd640 1 -- 172.21.15.22:0/696745665 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16811 (secure 0 0 0) 0x7f8f080682f0 con 0x7f8f180b3910 2024-09-21T05:41:19.452 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:19.452 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":45,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:03.389561+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:35.364998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:36.653978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:41:19.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ef8078b90 msgr2=0x7f8ef807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ef8078b90 0x7f8ef807b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8f180b9870 tx=0x7f8f0c009040 comp rx=0 tx=0).stop 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 msgr2=0x7f8f181f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 0x7f8f181f6360 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800eb40 tx=0x7f8f0800c2d0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 shutdown_connections 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ef8078b90 0x7f8ef807b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f180cffd0 0x7f8f180b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f180cfbc0 0x7f8f18158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 --2- 172.21.15.22:0/696745665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f180b3910 0x7f8f181f6360 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.451+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 >> 172.21.15.22:0/696745665 conn(0x7f8f1808bfc0 msgr2=0x7f8f180d1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:19.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.452+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 shutdown_connections 2024-09-21T05:41:19.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.452+0000 7f8f1ce4d640 1 -- 172.21.15.22:0/696745665 wait complete. 2024-09-21T05:41:19.465 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:41:19.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 -- 172.21.15.22:0/3577387326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700b66b0 msgr2=0x7fb6700d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 --2- 172.21.15.22:0/3577387326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700b66b0 0x7fb6700d4d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb664009830 tx=0x7fb66402a3f0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 -- 172.21.15.22:0/3577387326 shutdown_connections 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 --2- 172.21.15.22:0/3577387326 >> v2:172.21.15.134:3300/0 conn(0x7fb6700cf540 0x7fb67006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 --2- 172.21.15.22:0/3577387326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700b66b0 0x7fb6700d4d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 -- 172.21.15.22:0/3577387326 >> 172.21.15.22:0/3577387326 conn(0x7fb67009d7f0 msgr2=0x7fb6700d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 -- 172.21.15.22:0/3577387326 shutdown_connections 2024-09-21T05:41:19.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.579+0000 7fb676227640 1 -- 172.21.15.22:0/3577387326 wait complete. 2024-09-21T05:41:19.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.581+0000 7fb676227640 1 Processor -- start 2024-09-21T05:41:19.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.581+0000 7fb676227640 1 -- start start 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb676227640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6700b9b60 0x7fb6700ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb676227640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700cf540 0x7fb670150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb676227640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 0x7fb67014e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb676227640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb67006f5d0 con 0x7fb670150ee0 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb676227640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb67006f450 con 0x7fb6700b9b60 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb676227640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb67006f750 con 0x7fb6700cf540 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 0x7fb67014e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 0x7fb67014e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55040/0 (socket says 172.21.15.22:55040) 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 -- 172.21.15.22:0/24469506 learned_addr learned my addr 172.21.15.22:0/24469506 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66effd640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700cf540 0x7fb670150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66f7fe640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6700b9b60 0x7fb6700ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 -- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700cf540 msgr2=0x7fb670150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700cf540 0x7fb670150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 -- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6700b9b60 msgr2=0x7fb6700ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6700b9b60 0x7fb6700ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 -- 172.21.15.22:0/24469506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6700ca9b0 con 0x7fb670150ee0 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66f7fe640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6700b9b60 0x7fb6700ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:19.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb66ffff640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 0x7fb67014e280 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb66800ea40 tx=0x7fb66800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:19.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.583+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66800ce40 con 0x7fb670150ee0 2024-09-21T05:41:19.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.584+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb668004c10 con 0x7fb670150ee0 2024-09-21T05:41:19.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.584+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb668005020 con 0x7fb670150ee0 2024-09-21T05:41:19.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.584+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6700c93d0 con 0x7fb670150ee0 2024-09-21T05:41:19.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.584+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6700c9990 con 0x7fb670150ee0 2024-09-21T05:41:19.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.585+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67006fc70 con 0x7fb670150ee0 2024-09-21T05:41:19.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.585+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6680051c0 con 0x7fb670150ee0 2024-09-21T05:41:19.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.586+0000 7fb657fff640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb65c078b90 0x7fb65c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.586+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7fb66809bca0 con 0x7fb670150ee0 2024-09-21T05:41:19.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.586+0000 7fb66f7fe640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb65c078b90 0x7fb65c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.587+0000 7fb66f7fe640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb65c078b90 0x7fb65c07b0a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb660001660 tx=0x7fb660009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:19.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.590+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb668063f80 con 0x7fb670150ee0 2024-09-21T05:41:19.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.817+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6700d3fb0 con 0x7fb670150ee0 2024-09-21T05:41:19.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.818+0000 7fb657fff640 1 -- 172.21.15.22:0/24469506 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7fb668063880 con 0x7fb670150ee0 2024-09-21T05:41:19.820 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:19.820 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:41:19.820 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:41:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb65c078b90 msgr2=0x7fb65c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb65c078b90 0x7fb65c07b0a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb660001660 tx=0x7fb660009040 comp rx=0 tx=0).stop 2024-09-21T05:41:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 msgr2=0x7fb67014e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 0x7fb67014e280 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb66800ea40 tx=0x7fb66800ef00 comp rx=0 tx=0).stop 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 shutdown_connections 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb670150ee0 0x7fb67014e280 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb65c078b90 0x7fb65c07b0a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6700cf540 0x7fb670150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 --2- 172.21.15.22:0/24469506 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb6700b9b60 0x7fb6700ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 >> 172.21.15.22:0/24469506 conn(0x7fb67009d7f0 msgr2=0x7fb6700b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 shutdown_connections 2024-09-21T05:41:19.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.820+0000 7fb676227640 1 -- 172.21.15.22:0/24469506 wait complete. 2024-09-21T05:41:19.833 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.947+0000 7f941f52b640 1 -- 172.21.15.22:0/3813722054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 msgr2=0x7f94180b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.947+0000 7f941f52b640 1 --2- 172.21.15.22:0/3813722054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180b4790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f94140098e0 tx=0x7f941402a530 comp rx=0 tx=0).stop 2024-09-21T05:41:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.947+0000 7f941f52b640 1 -- 172.21.15.22:0/3813722054 shutdown_connections 2024-09-21T05:41:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.947+0000 7f941f52b640 1 --2- 172.21.15.22:0/3813722054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cffd0 0x7f94180d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.947+0000 7f941f52b640 1 --2- 172.21.15.22:0/3813722054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180b4790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.947+0000 7f941f52b640 1 -- 172.21.15.22:0/3813722054 >> 172.21.15.22:0/3813722054 conn(0x7f941808bfc0 msgr2=0x7f941809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:19.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.948+0000 7f941f52b640 1 -- 172.21.15.22:0/3813722054 shutdown_connections 2024-09-21T05:41:19.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.948+0000 7f941f52b640 1 -- 172.21.15.22:0/3813722054 wait complete. 2024-09-21T05:41:19.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.949+0000 7f941f52b640 1 Processor -- start 2024-09-21T05:41:19.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.949+0000 7f941f52b640 1 -- start start 2024-09-21T05:41:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941f52b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941f52b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cfbc0 0x7f941815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941f52b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 0x7f9418150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941f52b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f94180d5000 con 0x7f94180cffd0 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941f52b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f94180d4e80 con 0x7f94180b3910 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941f52b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f94180b3340 con 0x7f94180cfbc0 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 0x7f9418150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 0x7f9418150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55056/0 (socket says 172.21.15.22:55056) 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 -- 172.21.15.22:0/2235303015 learned_addr learned my addr 172.21.15.22:0/2235303015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941d2c7640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941cac6640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cfbc0 0x7f941815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 -- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cfbc0 msgr2=0x7f941815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cfbc0 0x7f941815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 -- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 msgr2=0x7f94180ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:19.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 -- 172.21.15.22:0/2235303015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94180cde80 con 0x7f94180cffd0 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941cac6640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cfbc0 0x7f941815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941d2c7640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.951+0000 7f941dac8640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 0x7f9418150790 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f941000ea80 tx=0x7f941000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.952+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410004b20 con 0x7f94180cffd0 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.952+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9410004ce0 con 0x7f94180cffd0 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.952+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94180ce0f0 con 0x7f94180cffd0 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.952+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410005280 con 0x7f94180cffd0 2024-09-21T05:41:19.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.952+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94180c9e30 con 0x7f94180cffd0 2024-09-21T05:41:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.953+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94180b6f20 con 0x7f94180cffd0 2024-09-21T05:41:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.954+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9410005420 con 0x7f94180cffd0 2024-09-21T05:41:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.954+0000 7f940dffb640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93fc078b90 0x7f93fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.955+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f941009ba90 con 0x7f94180cffd0 2024-09-21T05:41:19.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.955+0000 7f941d2c7640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93fc078b90 0x7f93fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:19.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.955+0000 7f941d2c7640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93fc078b90 0x7f93fc07b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9414034000 tx=0x7f9414004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:19.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:19.959+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94180b6f20 con 0x7f94180cffd0 2024-09-21T05:41:20.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.184+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94180b5bf0 con 0x7f94180cffd0 2024-09-21T05:41:20.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.185+0000 7f940dffb640 1 -- 172.21.15.22:0/2235303015 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5223 (secure 0 0 0) 0x7f9410063e10 con 0x7f94180cffd0 2024-09-21T05:41:20.187 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:20.187 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":26,"btime":"2024-09-21T05:40:41:329184+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:40:37.702239+0000","modified":"2024-09-21T05:40:40.714396+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5298":{"gid":5298,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/441099152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":441099152},{"type":"v1","addr":"172.21.15.134:6837","nonce":441099152}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5298,"qdb_cluster":[5298]},"id":3}]} 2024-09-21T05:41:20.188 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 26 2024-09-21T05:41:20.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93fc078b90 msgr2=0x7f93fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93fc078b90 0x7f93fc07b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9414034000 tx=0x7f9414004730 comp rx=0 tx=0).stop 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 msgr2=0x7f9418150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 0x7f9418150790 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f941000ea80 tx=0x7f941000ef40 comp rx=0 tx=0).stop 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 shutdown_connections 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f94180cffd0 0x7f9418150790 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93fc078b90 0x7f93fc07b0a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f94180cfbc0 0x7f941815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 --2- 172.21.15.22:0/2235303015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f94180b3910 0x7f94180ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 >> 172.21.15.22:0/2235303015 conn(0x7f941808bfc0 msgr2=0x7f941806e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 shutdown_connections 2024-09-21T05:41:20.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.187+0000 7f941f52b640 1 -- 172.21.15.22:0/2235303015 wait complete. 2024-09-21T05:41:20.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 -- 172.21.15.22:0/440991576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 msgr2=0x7f6814003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 --2- 172.21.15.22:0/440991576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f6814003ec0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f681c006fc0 tx=0x7f6814005830 comp rx=0 tx=0).stop 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 -- 172.21.15.22:0/440991576 shutdown_connections 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 --2- 172.21.15.22:0/440991576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f6814003ec0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 -- 172.21.15.22:0/440991576 >> 172.21.15.22:0/440991576 conn(0x7f682c09d840 msgr2=0x7f682c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 -- 172.21.15.22:0/440991576 shutdown_connections 2024-09-21T05:41:20.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.316+0000 7f6832ed5640 1 -- 172.21.15.22:0/440991576 wait complete. 2024-09-21T05:41:20.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.318+0000 7f6832ed5640 1 Processor -- start 2024-09-21T05:41:20.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.318+0000 7f6832ed5640 1 -- start start 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6832ed5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f682c14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6832ed5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c1f9950 0x7f682c0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6832ed5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c150760 0x7f682c0ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6832ed5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f682c06fb20 con 0x7f6814003a50 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6832ed5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f682c06f9a0 con 0x7f682c150760 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6832ed5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f682c06e610 con 0x7f682c1f9950 2024-09-21T05:41:20.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f682c14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f682c14feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55070/0 (socket says 172.21.15.22:55070) 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 -- 172.21.15.22:0/1011368693 learned_addr learned my addr 172.21.15.22:0/1011368693 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f682bfff640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c1f9950 0x7f682c0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6831472640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c150760 0x7f682c0ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 -- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c1f9950 msgr2=0x7f682c0c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c1f9950 0x7f682c0c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 -- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c150760 msgr2=0x7f682c0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c150760 0x7f682c0ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 -- 172.21.15.22:0/1011368693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682c1f8eb0 con 0x7f6814003a50 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f682bfff640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c1f9950 0x7f682c0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6831472640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c150760 0x7f682c0ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f6830c71640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f682c14feb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c350 tx=0x7f681c006380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.320+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c031070 con 0x7f6814003a50 2024-09-21T05:41:20.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.321+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f681c003af0 con 0x7f6814003a50 2024-09-21T05:41:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.321+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c001d40 con 0x7f6814003a50 2024-09-21T05:41:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.321+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c15fc50 con 0x7f6814003a50 2024-09-21T05:41:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.321+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c15bbc0 con 0x7f6814003a50 2024-09-21T05:41:20.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.321+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682c06ea50 con 0x7f6814003a50 2024-09-21T05:41:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.322+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f681c037430 con 0x7f6814003a50 2024-09-21T05:41:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.323+0000 7f68297fa640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6818078b90 0x7f681807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.323+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f681c0baf50 con 0x7f6814003a50 2024-09-21T05:41:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.323+0000 7f682bfff640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6818078b90 0x7f681807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.323+0000 7f682bfff640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6818078b90 0x7f681807b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6820005eb0 tx=0x7f6820005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:20.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.327+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f681c0832d0 con 0x7f6814003a50 2024-09-21T05:41:20.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.516+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f682c0cad30 con 0x7f6814003a50 2024-09-21T05:41:20.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.517+0000 7f68297fa640 1 -- 172.21.15.22:0/1011368693 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16811 (secure 0 0 0) 0x7f681c082bd0 con 0x7f6814003a50 2024-09-21T05:41:20.519 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:20.520 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":45,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:03.389561+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:40:35.364998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T05:40:36.653978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":5}]},{"pgid":"7.f","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6818078b90 msgr2=0x7f681807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6818078b90 0x7f681807b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6820005eb0 tx=0x7f6820005f90 comp rx=0 tx=0).stop 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 msgr2=0x7f682c14feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f682c14feb0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c350 tx=0x7f681c006380 comp rx=0 tx=0).stop 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 shutdown_connections 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f682c150760 0x7f682c0ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6818078b90 0x7f681807b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f682c1f9950 0x7f682c0c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 --2- 172.21.15.22:0/1011368693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6814003a50 0x7f682c14feb0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 >> 172.21.15.22:0/1011368693 conn(0x7f682c09d840 msgr2=0x7f682c0c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:20.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 shutdown_connections 2024-09-21T05:41:20.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.519+0000 7f6832ed5640 1 -- 172.21.15.22:0/1011368693 wait complete. 2024-09-21T05:41:20.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:41:20.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 -- 172.21.15.22:0/118724215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 msgr2=0x7f32f40708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 --2- 172.21.15.22:0/118724215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f40708b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f32f4070e90 tx=0x7f32ec02a560 comp rx=0 tx=0).stop 2024-09-21T05:41:20.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 -- 172.21.15.22:0/118724215 shutdown_connections 2024-09-21T05:41:20.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 --2- 172.21.15.22:0/118724215 >> v2:172.21.15.134:3300/0 conn(0x7f32f415c580 0x7f32f4070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 --2- 172.21.15.22:0/118724215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f40708b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 --2- 172.21.15.22:0/118724215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32f4074420 0x7f32f406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 -- 172.21.15.22:0/118724215 >> 172.21.15.22:0/118724215 conn(0x7f32f406def0 msgr2=0x7f32f406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:20.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 -- 172.21.15.22:0/118724215 shutdown_connections 2024-09-21T05:41:20.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.648+0000 7f32f988b640 1 -- 172.21.15.22:0/118724215 wait complete. 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.650+0000 7f32f988b640 1 Processor -- start 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.650+0000 7f32f988b640 1 -- start start 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f988b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32f4074420 0x7f32f413ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f988b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f41f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f988b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f41e57f0 0x7f32f41e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f988b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f32f415f9b0 con 0x7f32f4155ce0 2024-09-21T05:41:20.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f988b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f32f415f830 con 0x7f32f41e57f0 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f988b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f32f415fb30 con 0x7f32f4074420 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f41f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f41f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55088/0 (socket says 172.21.15.22:55088) 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 -- 172.21.15.22:0/319873897 learned_addr learned my addr 172.21.15.22:0/319873897 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f2ffd640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32f4074420 0x7f32f413ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f37fe640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f41e57f0 0x7f32f41e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 -- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32f4074420 msgr2=0x7f32f413ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32f4074420 0x7f32f413ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 -- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f41e57f0 msgr2=0x7f32f41e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f41e57f0 0x7f32f41e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 -- 172.21.15.22:0/319873897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f41e6140 con 0x7f32f4155ce0 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.652+0000 7f32f27fc640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f41f3f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d740 tx=0x7f32dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:20.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.653+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc017070 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.653+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32dc009d20 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.653+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f413d050 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.653+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0106c0 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.653+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f413d610 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.654+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f415c580 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.654+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f32dc01a070 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.655+0000 7f32ebfff640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f32d4078b90 0x7f32d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.655+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5767+0+0 (secure 0 0 0) 0x7f32dc09af20 con 0x7f32f4155ce0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.655+0000 7f32f2ffd640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f32d4078b90 0x7f32d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:20.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.656+0000 7f32f2ffd640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f32d4078b90 0x7f32d407b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e4007540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:20.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.659+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f32f41e6140 con 0x7f32f4155ce0 2024-09-21T05:41:20.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.886+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f32f415e4f0 con 0x7f32f4155ce0 2024-09-21T05:41:20.889 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:41:20.887+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:41:20.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.889+0000 7f32ebfff640 1 -- 172.21.15.22:0/319873897 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f32dc062c50 con 0x7f32f4155ce0 2024-09-21T05:41:20.891 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.891+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f32d4078b90 msgr2=0x7f32d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.891+0000 7f32f988b640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f32d4078b90 0x7f32d407b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e4007540 comp rx=0 tx=0).stop 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.891+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 msgr2=0x7f32f41f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.891+0000 7f32f988b640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f41f3f30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00d740 tx=0x7f32dc00dc00 comp rx=0 tx=0).stop 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 shutdown_connections 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f32f41e57f0 0x7f32f41e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f32f4155ce0 0x7f32f41f3f30 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f32d4078b90 0x7f32d407b0a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 --2- 172.21.15.22:0/319873897 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f32f4074420 0x7f32f413ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:20.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 >> 172.21.15.22:0/319873897 conn(0x7f32f406def0 msgr2=0x7f32f415db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:20.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 shutdown_connections 2024-09-21T05:41:20.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:20.892+0000 7f32f988b640 1 -- 172.21.15.22:0/319873897 wait complete. 2024-09-21T05:41:20.899 INFO:tasks.ceph.mds.b.smithi134.stderr: -7052> 2024-09-21T05:40:40.297+0000 7fc2c8aab640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:41:20.902 INFO:tasks.ceph.mds.b.smithi134.stderr: -3810> 2024-09-21T05:40:46.505+0000 7fc2c4aa3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 pv4 ap=4 DIRTYPARENT s=0 n(v0 1=1+0)->n(v0 rc2024-09-21T05:40:45.404725+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={5245=0-4194304@1} caps={5245=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=5245 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x56094c4a0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:41:20.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:41:20.957 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:41:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 -- 172.21.15.22:0/1570289281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cfb00 msgr2=0x7f732015bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 --2- 172.21.15.22:0/1570289281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cfb00 0x7f732015bf20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f731c009830 tx=0x7f731c02a510 comp rx=0 tx=0).stop 2024-09-21T05:41:21.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 -- 172.21.15.22:0/1570289281 shutdown_connections 2024-09-21T05:41:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 --2- 172.21.15.22:0/1570289281 >> v2:172.21.15.134:3300/0 conn(0x7f73200cf160 0x7f73200ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 --2- 172.21.15.22:0/1570289281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cfb00 0x7f732015bf20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 -- 172.21.15.22:0/1570289281 >> 172.21.15.22:0/1570289281 conn(0x7f73200b1cb0 msgr2=0x7f73200b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 -- 172.21.15.22:0/1570289281 shutdown_connections 2024-09-21T05:41:21.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.024+0000 7f7328371640 1 -- 172.21.15.22:0/1570289281 wait complete. 2024-09-21T05:41:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.026+0000 7f7328371640 1 Processor -- start 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.026+0000 7f7328371640 1 -- start start 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f7328371640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 0x7f7320150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f7328371640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cf570 0x7f73200b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f7328371640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73200cfb00 0x7f73200b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f7328371640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f73200cb270 con 0x7f73200cf160 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f7328371640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f73200cb0f0 con 0x7f73200cfb00 2024-09-21T05:41:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f7328371640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f73200ccba0 con 0x7f73200cf570 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.027+0000 7f732610d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 0x7f7320150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 0x7f7320150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55094/0 (socket says 172.21.15.22:55094) 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 -- 172.21.15.22:0/3287663371 learned_addr learned my addr 172.21.15.22:0/3287663371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732590c640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cf570 0x7f73200b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732690e640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73200cfb00 0x7f73200b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 -- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cf570 msgr2=0x7f73200b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:21.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cf570 0x7f73200b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 -- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73200cfb00 msgr2=0x7f73200b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:21.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73200cfb00 0x7f73200b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 -- 172.21.15.22:0/3287663371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732015ed80 con 0x7f73200cf160 2024-09-21T05:41:21.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f732610d640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 0x7f7320150610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f731c009950 tx=0x7f731c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:21.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c031d60 con 0x7f73200cf160 2024-09-21T05:41:21.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f731c033e10 con 0x7f73200cf160 2024-09-21T05:41:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732015f060 con 0x7f73200cf160 2024-09-21T05:41:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c005260 con 0x7f73200cf160 2024-09-21T05:41:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.028+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f732009bde0 con 0x7f73200cf160 2024-09-21T05:41:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.029+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73200cd0c0 con 0x7f73200cf160 2024-09-21T05:41:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.030+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f731c005400 con 0x7f73200cf160 2024-09-21T05:41:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.031+0000 7f7316ffd640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7304078bc0 0x7f730407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.031+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7f731c0bbf50 con 0x7f73200cf160 2024-09-21T05:41:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.031+0000 7f732590c640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7304078bc0 0x7f730407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.032+0000 7f732590c640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7304078bc0 0x7f730407b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7320070670 tx=0x7f7310009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.036+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f731c03f030 con 0x7f73200cf160 2024-09-21T05:41:21.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.263+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f73200d0aa0 con 0x7f73200cf160 2024-09-21T05:41:21.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:41:21.310 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:41:21.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.893+0000 7f7316ffd640 1 -- 172.21.15.22:0/3287663371 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f731c084310 con 0x7f73200cf160 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7304078bc0 msgr2=0x7f730407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7304078bc0 0x7f730407b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7320070670 tx=0x7f7310009260 comp rx=0 tx=0).stop 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 msgr2=0x7f7320150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 0x7f7320150610 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f731c009950 tx=0x7f731c033c20 comp rx=0 tx=0).stop 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 shutdown_connections 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f73200cfb00 0x7f73200b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7304078bc0 0x7f730407b0d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f73200cf570 0x7f73200b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 --2- 172.21.15.22:0/3287663371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f73200cf160 0x7f7320150610 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 >> 172.21.15.22:0/3287663371 conn(0x7f73200b1cb0 msgr2=0x7f73200afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 shutdown_connections 2024-09-21T05:41:21.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:21.896+0000 7f7328371640 1 -- 172.21.15.22:0/3287663371 wait complete. 2024-09-21T05:41:21.911 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:41:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.033+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/1284817924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80b38a0 msgr2=0x7fb5f80d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.033+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/1284817924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80b38a0 0x7fb5f80d1470 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009830 tx=0x7fb5ec02a480 comp rx=0 tx=0).stop 2024-09-21T05:41:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.033+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/1284817924 shutdown_connections 2024-09-21T05:41:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.033+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/1284817924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80b38a0 0x7fb5f80d1470 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.033+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/1284817924 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f806ded0 0x7fb5f80d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.033+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/1284817924 >> 172.21.15.22:0/1284817924 conn(0x7fb5f809d820 msgr2=0x7fb5f808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.034+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/1284817924 shutdown_connections 2024-09-21T05:41:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.034+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/1284817924 wait complete. 2024-09-21T05:41:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.035+0000 7fb5fd6f9640 1 Processor -- start 2024-09-21T05:41:22.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.035+0000 7fb5fd6f9640 1 -- start start 2024-09-21T05:41:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5fd6f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f806ded0 0x7fb5f8157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5fd6f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f80b38a0 0x7fb5f81f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5fd6f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 0x7fb5f81f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5fd6f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80b6830 con 0x7fb5f80d3f80 2024-09-21T05:41:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5fd6f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f80b66b0 con 0x7fb5f806ded0 2024-09-21T05:41:22.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5fd6f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5f80b69b0 con 0x7fb5f80b38a0 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 0x7fb5f81f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 0x7fb5f81f9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55112/0 (socket says 172.21.15.22:55112) 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 -- 172.21.15.22:0/3196265413 learned_addr learned my addr 172.21.15.22:0/3196265413 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f67fc640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f80b38a0 0x7fb5f81f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f6ffd640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f806ded0 0x7fb5f8157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 -- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f80b38a0 msgr2=0x7fb5f81f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f80b38a0 0x7fb5f81f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 -- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f806ded0 msgr2=0x7fb5f8157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f806ded0 0x7fb5f8157240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 -- 172.21.15.22:0/3196265413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f8157780 con 0x7fb5f80d3f80 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f67fc640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f80b38a0 0x7fb5f81f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f6ffd640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f806ded0 0x7fb5f8157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:22.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.037+0000 7fb5f77fe640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 0x7fb5f81f9050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000c6c0 tx=0x7fb5f000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.038+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0017070 con 0x7fb5f80d3f80 2024-09-21T05:41:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.038+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5f0003c60 con 0x7fb5f80d3f80 2024-09-21T05:41:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.038+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00052d0 con 0x7fb5f80d3f80 2024-09-21T05:41:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.038+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f81581c0 con 0x7fb5f80d3f80 2024-09-21T05:41:22.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.038+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f815abb0 con 0x7fb5f80d3f80 2024-09-21T05:41:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.039+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f80b5fa0 con 0x7fb5f80d3f80 2024-09-21T05:41:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.039+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5f001a070 con 0x7fb5f80d3f80 2024-09-21T05:41:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.040+0000 7fb5df7fe640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5e4078b90 0x7fb5e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.040+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5853+0+0 (secure 0 0 0) 0x7fb5f009a3d0 con 0x7fb5f80d3f80 2024-09-21T05:41:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.040+0000 7fb5f6ffd640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5e4078b90 0x7fb5e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:22.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.041+0000 7fb5f6ffd640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5e4078b90 0x7fb5e407b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009890 tx=0x7fb5e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:22.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.044+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5f0062810 con 0x7fb5f80d3f80 2024-09-21T05:41:22.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.232+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb5f81f9a40 con 0x7fb5f80d3f80 2024-09-21T05:41:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.895+0000 7fb5df7fe640 1 -- 172.21.15.22:0/3196265413 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) ==== 181+0+0 (secure 0 0 0) 0x7fb5f0062110 con 0x7fb5f80d3f80 2024-09-21T05:41:22.897 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:41:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5e4078b90 msgr2=0x7fb5e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5e4078b90 0x7fb5e407b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009890 tx=0x7fb5e8008040 comp rx=0 tx=0).stop 2024-09-21T05:41:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 msgr2=0x7fb5f81f9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:22.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 0x7fb5f81f9050 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000c6c0 tx=0x7fb5f000cb80 comp rx=0 tx=0).stop 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 shutdown_connections 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5f80d3f80 0x7fb5f81f9050 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5e4078b90 0x7fb5e407b0a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5f80b38a0 0x7fb5f81f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 --2- 172.21.15.22:0/3196265413 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5f806ded0 0x7fb5f8157240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.899+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 >> 172.21.15.22:0/3196265413 conn(0x7fb5f809d820 msgr2=0x7fb5f8161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.900+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 shutdown_connections 2024-09-21T05:41:22.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:22.900+0000 7fb5fd6f9640 1 -- 172.21.15.22:0/3196265413 wait complete. 2024-09-21T05:41:22.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 -- 172.21.15.22:0/3951576601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 msgr2=0x7fa91015d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 --2- 172.21.15.22:0/3951576601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91015d570 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa9100d2170 tx=0x7fa90402a4d0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 -- 172.21.15.22:0/3951576601 shutdown_connections 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 --2- 172.21.15.22:0/3951576601 >> v2:172.21.15.134:3300/0 conn(0x7fa9100ccb50 0x7fa9100ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 --2- 172.21.15.22:0/3951576601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91015d570 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 -- 172.21.15.22:0/3951576601 >> 172.21.15.22:0/3951576601 conn(0x7fa9100b3ea0 msgr2=0x7fa9100b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 -- 172.21.15.22:0/3951576601 shutdown_connections 2024-09-21T05:41:23.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.042+0000 7fa9161f1640 1 -- 172.21.15.22:0/3951576601 wait complete. 2024-09-21T05:41:23.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.044+0000 7fa9161f1640 1 Processor -- start 2024-09-21T05:41:23.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.044+0000 7fa9161f1640 1 -- start start 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.045+0000 7fa9161f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9100ccb50 0x7fa91006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.045+0000 7fa9161f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9100d1d60 0x7fa91006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.045+0000 7fa9161f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.045+0000 7fa9161f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100d0250 con 0x7fa91015c5e0 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.045+0000 7fa9161f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100d00d0 con 0x7fa9100ccb50 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.045+0000 7fa9161f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100d03d0 con 0x7fa9100d1d60 2024-09-21T05:41:23.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91006f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55134/0 (socket says 172.21.15.22:55134) 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 -- 172.21.15.22:0/951076331 learned_addr learned my addr 172.21.15.22:0/951076331 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90f7fe640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9100ccb50 0x7fa91006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90effd640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9100d1d60 0x7fa91006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 -- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9100d1d60 msgr2=0x7fa91006efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9100d1d60 0x7fa91006efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 -- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9100ccb50 msgr2=0x7fa91006ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9100ccb50 0x7fa91006ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 -- 172.21.15.22:0/951076331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100cb930 con 0x7fa91015c5e0 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90effd640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9100d1d60 0x7fa91006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa90ffff640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91006f4f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa90800dac0 tx=0x7fa90800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:23.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908017070 con 0x7fa91015c5e0 2024-09-21T05:41:23.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa908003970 con 0x7fa91015c5e0 2024-09-21T05:41:23.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91006df90 con 0x7fa91015c5e0 2024-09-21T05:41:23.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908003da0 con 0x7fa91015c5e0 2024-09-21T05:41:23.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.046+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91006e550 con 0x7fa91015c5e0 2024-09-21T05:41:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.047+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9100d0880 con 0x7fa91015c5e0 2024-09-21T05:41:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.048+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa908003f40 con 0x7fa91015c5e0 2024-09-21T05:41:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.049+0000 7fa8f7fff640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8fc078b90 0x7fa8fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.049+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5415+0+0 (secure 0 0 0) 0x7fa90809b020 con 0x7fa91015c5e0 2024-09-21T05:41:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.049+0000 7fa90f7fe640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8fc078b90 0x7fa8fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:23.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.050+0000 7fa90f7fe640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8fc078b90 0x7fa8fc07b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa900001720 tx=0x7fa900009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:23.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.054+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa908062f30 con 0x7fa91015c5e0 2024-09-21T05:41:23.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.239+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa9100731d0 con 0x7fa91015c5e0 2024-09-21T05:41:23.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.901+0000 7fa8f7fff640 1 -- 172.21.15.22:0/951076331 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) ==== 169+0+0 (secure 0 0 0) 0x7fa908067960 con 0x7fa91015c5e0 2024-09-21T05:41:23.903 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:41:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.905+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8fc078b90 msgr2=0x7fa8fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.905+0000 7fa9161f1640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8fc078b90 0x7fa8fc07b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa900001720 tx=0x7fa900009040 comp rx=0 tx=0).stop 2024-09-21T05:41:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.905+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 msgr2=0x7fa91006f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.905+0000 7fa9161f1640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91006f4f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa90800dac0 tx=0x7fa90800df80 comp rx=0 tx=0).stop 2024-09-21T05:41:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.905+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 shutdown_connections 2024-09-21T05:41:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.906+0000 7fa9161f1640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa91015c5e0 0x7fa91006f4f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.906+0000 7fa9161f1640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8fc078b90 0x7fa8fc07b0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.906+0000 7fa9161f1640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa9100d1d60 0x7fa91006efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.906+0000 7fa9161f1640 1 --2- 172.21.15.22:0/951076331 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa9100ccb50 0x7fa91006ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.906+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 >> 172.21.15.22:0/951076331 conn(0x7fa9100b3ea0 msgr2=0x7fa9100b71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.906+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 shutdown_connections 2024-09-21T05:41:23.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:23.907+0000 7fa9161f1640 1 -- 172.21.15.22:0/951076331 wait complete. 2024-09-21T05:41:23.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:41:24.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 -- 172.21.15.22:0/1365294535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 msgr2=0x7f9ca00d4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/1365294535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 0x7f9ca00d4510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009830 tx=0x7f9c9402a480 comp rx=0 tx=0).stop 2024-09-21T05:41:24.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 -- 172.21.15.22:0/1365294535 shutdown_connections 2024-09-21T05:41:24.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/1365294535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d4a50 0x7f9ca00b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/1365294535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 0x7f9ca00d4510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 -- 172.21.15.22:0/1365294535 >> 172.21.15.22:0/1365294535 conn(0x7f9ca008bfc0 msgr2=0x7f9ca009d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:24.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 -- 172.21.15.22:0/1365294535 shutdown_connections 2024-09-21T05:41:24.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.052+0000 7f9ca68b2640 1 -- 172.21.15.22:0/1365294535 wait complete. 2024-09-21T05:41:24.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.055+0000 7f9ca68b2640 1 Processor -- start 2024-09-21T05:41:24.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.055+0000 7f9ca68b2640 1 -- start start 2024-09-21T05:41:24.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca68b2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 0x7f9ca01519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca68b2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d0ce0 0x7f9ca01541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca68b2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 0x7f9ca01546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca68b2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00b4a90 con 0x7f9ca00d4a50 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca68b2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca00b4910 con 0x7f9ca00b9be0 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca68b2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca00b4c10 con 0x7f9ca00d0ce0 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca4e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 0x7f9ca01546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca4e4f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 0x7f9ca01546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55140/0 (socket says 172.21.15.22:55140) 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9ca4e4f640 1 -- 172.21.15.22:0/4393744 learned_addr learned my addr 172.21.15.22:0/4393744 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9c9ffff640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 0x7f9ca01519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.057+0000 7f9c9f7fe640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d0ce0 0x7f9ca01541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca4e4f640 1 -- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d0ce0 msgr2=0x7f9ca01541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca4e4f640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d0ce0 0x7f9ca01541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca4e4f640 1 -- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 msgr2=0x7f9ca01519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca4e4f640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 0x7f9ca01519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca4e4f640 1 -- 172.21.15.22:0/4393744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca00cdf70 con 0x7f9ca00d4a50 2024-09-21T05:41:24.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9c9f7fe640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d0ce0 0x7f9ca01541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca4e4f640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 0x7f9ca01546f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800efc0 tx=0x7f9c9800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98003970 con 0x7f9ca00d4a50 2024-09-21T05:41:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c98003c20 con 0x7f9ca00d4a50 2024-09-21T05:41:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca01521c0 con 0x7f9ca00d4a50 2024-09-21T05:41:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004280 con 0x7f9ca00d4a50 2024-09-21T05:41:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.058+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca0152780 con 0x7f9ca00d4a50 2024-09-21T05:41:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.059+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca00b50c0 con 0x7f9ca00d4a50 2024-09-21T05:41:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.060+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c98004420 con 0x7f9ca00d4a50 2024-09-21T05:41:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.060+0000 7f9c9cff9640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c8c078bc0 0x7f9c8c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.060+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f9c9809b2e0 con 0x7f9ca00d4a50 2024-09-21T05:41:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.061+0000 7f9c9ffff640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c8c078bc0 0x7f9c8c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.061+0000 7f9c9ffff640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c8c078bc0 0x7f9c8c07b0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000f60 tx=0x7f9c90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:24.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.065+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9c98063ac0 con 0x7f9ca00d4a50 2024-09-21T05:41:24.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.260+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9ca006f150 con 0x7f9ca00d4a50 2024-09-21T05:41:24.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.263+0000 7f9c9cff9640 1 -- 172.21.15.22:0/4393744 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f9c980633c0 con 0x7f9ca00d4a50 2024-09-21T05:41:24.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.265+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c8c078bc0 msgr2=0x7f9c8c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.265+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c8c078bc0 0x7f9c8c07b0d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9c90000f60 tx=0x7f9c90009040 comp rx=0 tx=0).stop 2024-09-21T05:41:24.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.265+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 msgr2=0x7f9ca01546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.265+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 0x7f9ca01546f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800efc0 tx=0x7f9c9800c800 comp rx=0 tx=0).stop 2024-09-21T05:41:24.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 shutdown_connections 2024-09-21T05:41:24.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ca00d4a50 0x7f9ca01546f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c8c078bc0 0x7f9c8c07b0d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ca00d0ce0 0x7f9ca01541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 --2- 172.21.15.22:0/4393744 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ca00b9be0 0x7f9ca01519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 >> 172.21.15.22:0/4393744 conn(0x7f9ca008bfc0 msgr2=0x7f9ca0161a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 shutdown_connections 2024-09-21T05:41:24.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.266+0000 7f9ca68b2640 1 -- 172.21.15.22:0/4393744 wait complete. 2024-09-21T05:41:24.279 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-09-21T05:41:24.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 -- 172.21.15.22:0/2170078580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0cd360 msgr2=0x7fc21c15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 --2- 172.21.15.22:0/2170078580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0cd360 0x7fc21c15c6a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009830 tx=0x7fc20c02a480 comp rx=0 tx=0).stop 2024-09-21T05:41:24.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 -- 172.21.15.22:0/2170078580 shutdown_connections 2024-09-21T05:41:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 --2- 172.21.15.22:0/2170078580 >> v2:172.21.15.134:3300/0 conn(0x7fc21c0cf440 0x7fc21c0d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 --2- 172.21.15.22:0/2170078580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0cd360 0x7fc21c15c6a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 -- 172.21.15.22:0/2170078580 >> 172.21.15.22:0/2170078580 conn(0x7fc21c0710b0 msgr2=0x7fc21c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 -- 172.21.15.22:0/2170078580 shutdown_connections 2024-09-21T05:41:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.394+0000 7fc2230c5640 1 -- 172.21.15.22:0/2170078580 wait complete. 2024-09-21T05:41:24.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.395+0000 7fc2230c5640 1 Processor -- start 2024-09-21T05:41:24.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.396+0000 7fc2230c5640 1 -- start start 2024-09-21T05:41:24.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc2230c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0b7fe0 0x7fc21c14adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc2230c5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 0x7fc21c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc2230c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21c15e320 0x7fc21c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc2230c5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0d0b70 con 0x7fc21c0cd360 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc2230c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0d09f0 con 0x7fc21c15e320 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc2230c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc21c0d0cf0 con 0x7fc21c0b7fe0 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc21bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 0x7fc21c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc21bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 0x7fc21c0cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36586/0 (socket says 172.21.15.22:36586) 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc21bfff640 1 -- 172.21.15.22:0/109447965 learned_addr learned my addr 172.21.15.22:0/109447965 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc220e61640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0b7fe0 0x7fc21c14adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.397+0000 7fc221662640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21c15e320 0x7fc21c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc21bfff640 1 -- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0b7fe0 msgr2=0x7fc21c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc21bfff640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0b7fe0 0x7fc21c14adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc21bfff640 1 -- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21c15e320 msgr2=0x7fc21c0c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc21bfff640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21c15e320 0x7fc21c0c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc21bfff640 1 -- 172.21.15.22:0/109447965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c0c6340 con 0x7fc21c0cd360 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc21bfff640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 0x7fc21c0cba80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c730 tx=0x7fc20400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204003b70 con 0x7fc21c0cd360 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc204003d30 con 0x7fc21c0cd360 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20400aab0 con 0x7fc21c0cd360 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c0c6620 con 0x7fc21c0cd360 2024-09-21T05:41:24.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.398+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c0c6be0 con 0x7fc21c0cd360 2024-09-21T05:41:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.399+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c0d1210 con 0x7fc21c0cd360 2024-09-21T05:41:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.400+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc20400ac50 con 0x7fc21c0cd360 2024-09-21T05:41:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.400+0000 7fc2197fa640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1fc078b90 0x7fc1fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.400+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fc20409a730 con 0x7fc21c0cd360 2024-09-21T05:41:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.400+0000 7fc220e61640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1fc078b90 0x7fc1fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.401+0000 7fc220e61640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1fc078b90 0x7fc1fc07b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc20c034000 tx=0x7fc20c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:24.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.405+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc204062820 con 0x7fc21c0cd360 2024-09-21T05:41:24.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.666+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fc21c15e730 con 0x7fc21c0cd360 2024-09-21T05:41:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.902+0000 7fc2197fa640 1 -- 172.21.15.22:0/109447965 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v122) ==== 149+0+0 (secure 0 0 0) 0x7fc204067250 con 0x7fc21c0cd360 2024-09-21T05:41:24.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1fc078b90 msgr2=0x7fc1fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1fc078b90 0x7fc1fc07b0a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc20c034000 tx=0x7fc20c004730 comp rx=0 tx=0).stop 2024-09-21T05:41:24.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 msgr2=0x7fc21c0cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:24.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 0x7fc21c0cba80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc20400c730 tx=0x7fc20400cbf0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 shutdown_connections 2024-09-21T05:41:24.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc21c15e320 0x7fc21c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc21c0cd360 0x7fc21c0cba80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1fc078b90 0x7fc1fc07b0a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 --2- 172.21.15.22:0/109447965 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc21c0b7fe0 0x7fc21c14adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 >> 172.21.15.22:0/109447965 conn(0x7fc21c0710b0 msgr2=0x7fc21c15b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 shutdown_connections 2024-09-21T05:41:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:24.905+0000 7fc2230c5640 1 -- 172.21.15.22:0/109447965 wait complete. 2024-09-21T05:41:24.919 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-09-21T05:41:24.919 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-21T05:41:24.920 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:41:24.920 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:41:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 -- 172.21.15.22:0/2566068729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000cf540 msgr2=0x7fad0006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 --2- 172.21.15.22:0/2566068729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000cf540 0x7fad0006eab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7facf8005fd0 tx=0x7facf802a9e0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 -- 172.21.15.22:0/2566068729 shutdown_connections 2024-09-21T05:41:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 --2- 172.21.15.22:0/2566068729 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000cf540 0x7fad0006eab0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 --2- 172.21.15.22:0/2566068729 >> v2:172.21.15.134:3300/0 conn(0x7fad000b66b0 0x7fad000d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 -- 172.21.15.22:0/2566068729 >> 172.21.15.22:0/2566068729 conn(0x7fad0009d7f0 msgr2=0x7fad000d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 -- 172.21.15.22:0/2566068729 shutdown_connections 2024-09-21T05:41:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.035+0000 7fad07a4d640 1 -- 172.21.15.22:0/2566068729 wait complete. 2024-09-21T05:41:25.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.037+0000 7fad07a4d640 1 Processor -- start 2024-09-21T05:41:25.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.037+0000 7fad07a4d640 1 -- start start 2024-09-21T05:41:25.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad07a4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad000b9b60 0x7fad001f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad07a4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000cf540 0x7fad000cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad07a4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 0x7fad000b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad07a4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad0006f600 con 0x7fad000b2040 2024-09-21T05:41:25.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad07a4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad0006f480 con 0x7fad000b9b60 2024-09-21T05:41:25.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad07a4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad0006f780 con 0x7fad000cf540 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 0x7fad000b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 0x7fad000b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36598/0 (socket says 172.21.15.22:36598) 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 -- 172.21.15.22:0/3186497314 learned_addr learned my addr 172.21.15.22:0/3186497314 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad057e9640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad000b9b60 0x7fad001f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad04fe8640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000cf540 0x7fad000cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 -- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000cf540 msgr2=0x7fad000cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000cf540 0x7fad000cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 -- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad000b9b60 msgr2=0x7fad001f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad000b9b60 0x7fad001f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad05fea640 1 -- 172.21.15.22:0/3186497314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad000b2990 con 0x7fad000b2040 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad057e9640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad000b9b60 0x7fad001f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.038+0000 7fad04fe8640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000cf540 0x7fad000cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.039+0000 7fad05fea640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 0x7fad000b2450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7facf802afd0 tx=0x7facf8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.039+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8033050 con 0x7fad000b2040 2024-09-21T05:41:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.039+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facf80044d0 con 0x7fad000b2040 2024-09-21T05:41:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.039+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad001f54c0 con 0x7fad000b2040 2024-09-21T05:41:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.039+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8038450 con 0x7fad000b2040 2024-09-21T05:41:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.039+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad000b46e0 con 0x7fad000b2040 2024-09-21T05:41:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.040+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0006fce0 con 0x7fad000b2040 2024-09-21T05:41:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.041+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7facf8036070 con 0x7fad000b2040 2024-09-21T05:41:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.042+0000 7facf5ffb640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face4078b90 0x7face407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.042+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7facf80bbdf0 con 0x7fad000b2040 2024-09-21T05:41:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.042+0000 7fad057e9640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face4078b90 0x7face407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.043+0000 7fad057e9640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face4078b90 0x7face407b0a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:25.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.046+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad000b2990 con 0x7fad000b2040 2024-09-21T05:41:25.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.236+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad000d3fb0 con 0x7fad000b2040 2024-09-21T05:41:25.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.237+0000 7facf5ffb640 1 -- 172.21.15.22:0/3186497314 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+13020 (secure 0 0 0) 0x7facf80845b0 con 0x7fad000b2040 2024-09-21T05:41:25.239 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:25.239 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":48,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:23.896346+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.134:6837/441099152":"2024-09-22T05:41:20.888195+0000","172.21.15.134:6835/441099152":"2024-09-22T05:41:20.888195+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.239+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face4078b90 msgr2=0x7face407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.239+0000 7fad07a4d640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face4078b90 0x7face407b0a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facfc007fd0 tx=0x7facfc009040 comp rx=0 tx=0).stop 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 msgr2=0x7fad000b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 0x7fad000b2450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7facf802afd0 tx=0x7facf8003f90 comp rx=0 tx=0).stop 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 shutdown_connections 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad000b2040 0x7fad000b2450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face4078b90 0x7face407b0a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad000cf540 0x7fad000cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 --2- 172.21.15.22:0/3186497314 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad000b9b60 0x7fad001f4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 >> 172.21.15.22:0/3186497314 conn(0x7fad0009d7f0 msgr2=0x7fad00161960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 shutdown_connections 2024-09-21T05:41:25.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.240+0000 7fad07a4d640 1 -- 172.21.15.22:0/3186497314 wait complete. 2024-09-21T05:41:25.253 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:41:25.253 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.371+0000 7f45aa972640 1 -- 172.21.15.22:0/1959255978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 msgr2=0x7f45a40d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.371+0000 7f45aa972640 1 --2- 172.21.15.22:0/1959255978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a40d3330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4598009900 tx=0x7f459802a580 comp rx=0 tx=0).stop 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.371+0000 7f45aa972640 1 -- 172.21.15.22:0/1959255978 shutdown_connections 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.372+0000 7f45aa972640 1 --2- 172.21.15.22:0/1959255978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a40d3330 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.372+0000 7f45aa972640 1 --2- 172.21.15.22:0/1959255978 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45a406e170 0x7f45a40b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.372+0000 7f45aa972640 1 -- 172.21.15.22:0/1959255978 >> 172.21.15.22:0/1959255978 conn(0x7f45a408bfc0 msgr2=0x7f45a409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:25.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.372+0000 7f45aa972640 1 -- 172.21.15.22:0/1959255978 shutdown_connections 2024-09-21T05:41:25.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.372+0000 7f45aa972640 1 -- 172.21.15.22:0/1959255978 wait complete. 2024-09-21T05:41:25.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.373+0000 7f45aa972640 1 Processor -- start 2024-09-21T05:41:25.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.374+0000 7f45aa972640 1 -- start start 2024-09-21T05:41:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45aa972640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a409e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45aa972640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45a40d3870 0x7f45a4151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45aa972640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45a415f910 0x7f45a41f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45aa972640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f45a40d0e60 con 0x7f45a406f4b0 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45aa972640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f45a40d0ce0 con 0x7f45a40d3870 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45aa972640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f45a40d0fe0 con 0x7f45a415f910 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a409e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a409e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36624/0 (socket says 172.21.15.22:36624) 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 -- 172.21.15.22:0/2068416987 learned_addr learned my addr 172.21.15.22:0/2068416987 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a37fe640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45a40d3870 0x7f45a4151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a8f0f640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45a415f910 0x7f45a41f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 -- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45a415f910 msgr2=0x7f45a41f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45a415f910 0x7f45a41f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 -- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45a40d3870 msgr2=0x7f45a4151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45a40d3870 0x7f45a4151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.375+0000 7f45a3fff640 1 -- 172.21.15.22:0/2068416987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a41f6290 con 0x7f45a406f4b0 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.376+0000 7f45a3fff640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a409e1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f459400e7c0 tx=0x7f459400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:25.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.376+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459400ccc0 con 0x7f45a406f4b0 2024-09-21T05:41:25.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.376+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4594010430 con 0x7f45a406f4b0 2024-09-21T05:41:25.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.376+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a41f67e0 con 0x7f45a406f4b0 2024-09-21T05:41:25.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.376+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4594010860 con 0x7f45a406f4b0 2024-09-21T05:41:25.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.376+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a41f9bf0 con 0x7f45a406f4b0 2024-09-21T05:41:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.377+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a40b2170 con 0x7f45a406f4b0 2024-09-21T05:41:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.378+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4594010a00 con 0x7f45a406f4b0 2024-09-21T05:41:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.378+0000 7f45a0ff9640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4590078b90 0x7f459007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.378+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f459409bb00 con 0x7f45a406f4b0 2024-09-21T05:41:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.379+0000 7f45a37fe640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4590078b90 0x7f459007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.379+0000 7f45a37fe640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4590078b90 0x7f459007b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4598033000 tx=0x7f4598004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:25.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.382+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45a40b2170 con 0x7f45a406f4b0 2024-09-21T05:41:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.571+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45a40d1500 con 0x7f45a406f4b0 2024-09-21T05:41:25.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.571+0000 7f45a0ff9640 1 -- 172.21.15.22:0/2068416987 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) ==== 93+0+10 (secure 0 0 0) 0x7f4594063cf0 con 0x7f45a406f4b0 2024-09-21T05:41:25.574 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:41:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4590078b90 msgr2=0x7f459007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4590078b90 0x7f459007b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4598033000 tx=0x7f4598004730 comp rx=0 tx=0).stop 2024-09-21T05:41:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 msgr2=0x7f45a409e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a409e1d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f459400e7c0 tx=0x7f459400ec80 comp rx=0 tx=0).stop 2024-09-21T05:41:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 shutdown_connections 2024-09-21T05:41:25.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4590078b90 0x7f459007b0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f45a415f910 0x7f45a41f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f45a40d3870 0x7f45a4151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 --2- 172.21.15.22:0/2068416987 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f45a406f4b0 0x7f45a409e1d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 >> 172.21.15.22:0/2068416987 conn(0x7f45a408bfc0 msgr2=0x7f45a41618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 shutdown_connections 2024-09-21T05:41:25.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.574+0000 7f45aa972640 1 -- 172.21.15.22:0/2068416987 wait complete. 2024-09-21T05:41:25.587 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 -- 172.21.15.22:0/654047342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 msgr2=0x7f827c06f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 --2- 172.21.15.22:0/654047342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 0x7f827c06f6e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8270005fd0 tx=0x7f827002a4b0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 -- 172.21.15.22:0/654047342 shutdown_connections 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 --2- 172.21.15.22:0/654047342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c0d0e50 0x7f827c0d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 --2- 172.21.15.22:0/654047342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 0x7f827c06f6e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 -- 172.21.15.22:0/654047342 >> 172.21.15.22:0/654047342 conn(0x7f827c09d840 msgr2=0x7f827c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 -- 172.21.15.22:0/654047342 shutdown_connections 2024-09-21T05:41:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.702+0000 7f8281fe6640 1 -- 172.21.15.22:0/654047342 wait complete. 2024-09-21T05:41:25.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.704+0000 7f8281fe6640 1 Processor -- start 2024-09-21T05:41:25.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.704+0000 7f8281fe6640 1 -- start start 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f8281fe6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c06e0a0 0x7f827c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f8281fe6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 0x7f827c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f8281fe6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 0x7f827c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f8281fe6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f827c0cfea0 con 0x7f827c0d0e50 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f8281fe6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f827c0cfd20 con 0x7f827c0b97e0 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f8281fe6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f827c0d0020 con 0x7f827c06e0a0 2024-09-21T05:41:25.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 0x7f827c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 0x7f827c0b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36634/0 (socket says 172.21.15.22:36634) 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 -- 172.21.15.22:0/3827389655 learned_addr learned my addr 172.21.15.22:0/3827389655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827affd640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 0x7f827c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827b7fe640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c06e0a0 0x7f827c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 -- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c06e0a0 msgr2=0x7f827c151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c06e0a0 0x7f827c151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 -- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 msgr2=0x7f827c0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 0x7f827c0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 -- 172.21.15.22:0/3827389655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c09f990 con 0x7f827c0d0e50 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827b7fe640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c06e0a0 0x7f827c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.706+0000 7f827bfff640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 0x7f827c0b5420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f827400ea70 tx=0x7f827400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:25.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.707+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827400ce40 con 0x7f827c0d0e50 2024-09-21T05:41:25.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.707+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8274004c10 con 0x7f827c0d0e50 2024-09-21T05:41:25.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.707+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c09fc70 con 0x7f827c0d0e50 2024-09-21T05:41:25.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.707+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8274005020 con 0x7f827c0d0e50 2024-09-21T05:41:25.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.707+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c15d2f0 con 0x7f827c0d0e50 2024-09-21T05:41:25.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.708+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c0d0420 con 0x7f827c0d0e50 2024-09-21T05:41:25.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.708+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f82740051c0 con 0x7f827c0d0e50 2024-09-21T05:41:25.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.709+0000 7f8263fff640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8268078bc0 0x7f826807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:25.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.709+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f827409b840 con 0x7f827c0d0e50 2024-09-21T05:41:25.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.709+0000 7f827b7fe640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8268078bc0 0x7f826807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:25.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.710+0000 7f827b7fe640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8268078bc0 0x7f826807b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f826c0016d0 tx=0x7f826c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:25.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.714+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8274063f70 con 0x7f827c0d0e50 2024-09-21T05:41:25.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:25.976+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f827c06e7b0 con 0x7f827c0d0e50 2024-09-21T05:41:26.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.910+0000 7f8263fff640 1 -- 172.21.15.22:0/3827389655 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v124) ==== 172+0+0 (secure 0 0 0) 0x7f8274063870 con 0x7f827c0d0e50 2024-09-21T05:41:26.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8268078bc0 msgr2=0x7f826807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:26.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8268078bc0 0x7f826807b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f826c0016d0 tx=0x7f826c009040 comp rx=0 tx=0).stop 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 msgr2=0x7f827c0b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 0x7f827c0b5420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f827400ea70 tx=0x7f827400ef30 comp rx=0 tx=0).stop 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 shutdown_connections 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f827c0d0e50 0x7f827c0b5420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f827c0b97e0 0x7f827c0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8268078bc0 0x7f826807b0d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 --2- 172.21.15.22:0/3827389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f827c06e0a0 0x7f827c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 >> 172.21.15.22:0/3827389655 conn(0x7f827c09d840 msgr2=0x7f827c161cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.913+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 shutdown_connections 2024-09-21T05:41:26.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:26.914+0000 7f8281fe6640 1 -- 172.21.15.22:0/3827389655 wait complete. 2024-09-21T05:41:26.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:41:27.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/2440929055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00b66b0 msgr2=0x7fb4a006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/2440929055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00b66b0 0x7fb4a006eab0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb498005fd0 tx=0x7fb49802a510 comp rx=0 tx=0).stop 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/2440929055 shutdown_connections 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/2440929055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00b66b0 0x7fb4a006eab0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/2440929055 >> v2:172.21.15.134:3300/0 conn(0x7fb4a00b9b60 0x7fb4a00d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/2440929055 >> 172.21.15.22:0/2440929055 conn(0x7fb4a009d7f0 msgr2=0x7fb4a00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/2440929055 shutdown_connections 2024-09-21T05:41:27.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.052+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/2440929055 wait complete. 2024-09-21T05:41:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.054+0000 7fb4a4c4a640 1 Processor -- start 2024-09-21T05:41:27.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.054+0000 7fb4a4c4a640 1 -- start start 2024-09-21T05:41:27.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb4a4c4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a00b9b60 0x7fb4a01f9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb4a4c4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 0x7fb4a01f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb4a4c4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a01f5790 0x7fb4a0160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb4a4c4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a006f600 con 0x7fb4a00cf540 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb4a4c4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4a006f480 con 0x7fb4a01f5790 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb4a4c4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4a006f780 con 0x7fb4a00b9b60 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.055+0000 7fb49dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 0x7fb4a01f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 0x7fb4a01f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36646/0 (socket says 172.21.15.22:36646) 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 -- 172.21.15.22:0/3847597093 learned_addr learned my addr 172.21.15.22:0/3847597093 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49e59c640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a00b9b60 0x7fb4a01f9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49ed9d640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a01f5790 0x7fb4a0160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 -- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a00b9b60 msgr2=0x7fb4a01f9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a00b9b60 0x7fb4a01f9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 -- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a01f5790 msgr2=0x7fb4a0160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a01f5790 0x7fb4a0160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 -- 172.21.15.22:0/3847597093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a01f5ea0 con 0x7fb4a00cf540 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb49dd9b640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 0x7fb4a01f5250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c340 tx=0x7fb49400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494017070 con 0x7fb4a00cf540 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4940053c0 con 0x7fb4a00cf540 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a00cc940 con 0x7fb4a00cf540 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494003850 con 0x7fb4a00cf540 2024-09-21T05:41:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.056+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4a00ccf00 con 0x7fb4a00cf540 2024-09-21T05:41:27.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.058+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a01f5c10 con 0x7fb4a00cf540 2024-09-21T05:41:27.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.058+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb49401a070 con 0x7fb4a00cf540 2024-09-21T05:41:27.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.059+0000 7fb48effd640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.059+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fb494099cc0 con 0x7fb4a00cf540 2024-09-21T05:41:27.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.059+0000 7fb49e59c640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.060+0000 7fb49e59c640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb490005fd0 tx=0x7fb490009630 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:27.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.063+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb494061db0 con 0x7fb4a00cf540 2024-09-21T05:41:27.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.253+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb4a006fca0 con 0x7fb4a00cf540 2024-09-21T05:41:27.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.254+0000 7fb48effd640 1 -- 172.21.15.22:0/3847597093 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7fb4940667e0 con 0x7fb4a00cf540 2024-09-21T05:41:27.256 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:41:27.256 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:41:27.256 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:41:27.256 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:41:27.256 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:41:27.257 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:41:27.258 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 msgr2=0x7fb48007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb490005fd0 tx=0x7fb490009630 comp rx=0 tx=0).stop 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 msgr2=0x7fb4a01f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 0x7fb4a01f5250 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb49400c340 tx=0x7fb49400c800 comp rx=0 tx=0).stop 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 shutdown_connections 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4a01f5790 0x7fb4a0160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4a00cf540 0x7fb4a01f5250 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 --2- 172.21.15.22:0/3847597093 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4a00b9b60 0x7fb4a01f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 >> 172.21.15.22:0/3847597093 conn(0x7fb4a009d7f0 msgr2=0x7fb4a00b6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 shutdown_connections 2024-09-21T05:41:27.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.256+0000 7fb4a4c4a640 1 -- 172.21.15.22:0/3847597093 wait complete. 2024-09-21T05:41:27.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:41:27.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1786178193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc0cff70 msgr2=0x7fa0dc0d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1786178193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc0cff70 0x7fa0dc0d1da0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc005fd0 tx=0x7fa0cc02a8b0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1786178193 shutdown_connections 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1786178193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0dc0b3e20 0x7fa0dc0b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1786178193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc0cff70 0x7fa0dc0d1da0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1786178193 >> 172.21.15.22:0/1786178193 conn(0x7fa0dc0cee20 msgr2=0x7fa0dc0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1786178193 shutdown_connections 2024-09-21T05:41:27.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.387+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1786178193 wait complete. 2024-09-21T05:41:27.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.389+0000 7fa0e3cf9640 1 Processor -- start 2024-09-21T05:41:27.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.389+0000 7fa0e3cf9640 1 -- start start 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e3cf9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0dc0b3e20 0x7fa0dc14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e3cf9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0dc0cff70 0x7fa0dc1e93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e3cf9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 0x7fa0dc1e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e3cf9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc0c6470 con 0x7fa0dc15c930 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e3cf9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0dc0c62f0 con 0x7fa0dc0cff70 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e3cf9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa0dc0c65f0 con 0x7fa0dc0b3e20 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e2296640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 0x7fa0dc1e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e2296640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 0x7fa0dc1e98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36656/0 (socket says 172.21.15.22:36656) 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.390+0000 7fa0e2296640 1 -- 172.21.15.22:0/1043409294 learned_addr learned my addr 172.21.15.22:0/1043409294 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:27.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e1a95640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0dc0b3e20 0x7fa0dc14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e1294640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0dc0cff70 0x7fa0dc1e93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e2296640 1 -- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0dc0b3e20 msgr2=0x7fa0dc14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e2296640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0dc0b3e20 0x7fa0dc14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e2296640 1 -- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0dc0cff70 msgr2=0x7fa0dc1e93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e2296640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0dc0cff70 0x7fa0dc1e93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e2296640 1 -- 172.21.15.22:0/1043409294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0dc0b7a30 con 0x7fa0dc15c930 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e2296640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 0x7fa0dc1e98e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d650 tx=0x7fa0d400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017070 con 0x7fa0dc15c930 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0d4005310 con 0x7fa0dc15c930 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0dc0b7f30 con 0x7fa0dc15c930 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40097c0 con 0x7fa0dc15c930 2024-09-21T05:41:27.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.391+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0dc1f5ad0 con 0x7fa0dc15c930 2024-09-21T05:41:27.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.392+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0dc0b7c80 con 0x7fa0dc15c930 2024-09-21T05:41:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.393+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa0d4009960 con 0x7fa0dc15c930 2024-09-21T05:41:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.394+0000 7fa0da7fc640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0c8078bc0 0x7fa0c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.394+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7fa0d409a980 con 0x7fa0dc15c930 2024-09-21T05:41:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.394+0000 7fa0e1a95640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0c8078bc0 0x7fa0c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.394+0000 7fa0e1a95640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0c8078bc0 0x7fa0c807b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc034000 tx=0x7fa0cc004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:27.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.398+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0d4063150 con 0x7fa0dc15c930 2024-09-21T05:41:27.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.584+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa0dc0c6b10 con 0x7fa0dc15c930 2024-09-21T05:41:27.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.588+0000 7fa0da7fc640 1 -- 172.21.15.22:0/1043409294 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fa0d4062a50 con 0x7fa0dc15c930 2024-09-21T05:41:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0c8078bc0 msgr2=0x7fa0c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0c8078bc0 0x7fa0c807b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc034000 tx=0x7fa0cc004ab0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 msgr2=0x7fa0dc1e98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 0x7fa0dc1e98e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d650 tx=0x7fa0d400db10 comp rx=0 tx=0).stop 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 shutdown_connections 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0dc15c930 0x7fa0dc1e98e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0dc0cff70 0x7fa0dc1e93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0c8078bc0 0x7fa0c807b0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 --2- 172.21.15.22:0/1043409294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0dc0b3e20 0x7fa0dc14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 >> 172.21.15.22:0/1043409294 conn(0x7fa0dc0cee20 msgr2=0x7fa0dc0c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 shutdown_connections 2024-09-21T05:41:27.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.590+0000 7fa0e3cf9640 1 -- 172.21.15.22:0/1043409294 wait complete. 2024-09-21T05:41:27.603 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:41:27.603 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:41:27.644 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:41:27.687 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:41:27.687 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:41:27.687 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:27.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:41:27.703 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:41:27.703 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:27.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:41:27.745 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:41:27.745 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:27.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:41:27.762 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:41:27.762 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:27.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:41:27.803 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:41:27.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:27.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 -- 172.21.15.22:0/1430632514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0b9b70 msgr2=0x7f24dc0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 --2- 172.21.15.22:0/1430632514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0b9b70 0x7f24dc0cf090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f24dc08d4e0 tx=0x7f24d002a560 comp rx=0 tx=0).stop 2024-09-21T05:41:27.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 -- 172.21.15.22:0/1430632514 shutdown_connections 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 --2- 172.21.15.22:0/1430632514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24dc0cf5d0 0x7f24dc0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 --2- 172.21.15.22:0/1430632514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0b9b70 0x7f24dc0cf090 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 --2- 172.21.15.22:0/1430632514 >> v2:172.21.15.134:3300/0 conn(0x7f24dc0d2ed0 0x7f24dc0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 -- 172.21.15.22:0/1430632514 >> 172.21.15.22:0/1430632514 conn(0x7f24dc08bfc0 msgr2=0x7f24dc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 -- 172.21.15.22:0/1430632514 shutdown_connections 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.918+0000 7f24e1827640 1 -- 172.21.15.22:0/1430632514 wait complete. 2024-09-21T05:41:27.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.920+0000 7f24e1827640 1 Processor -- start 2024-09-21T05:41:27.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.920+0000 7f24e1827640 1 -- start start 2024-09-21T05:41:27.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24e1827640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24dc0b9b70 0x7f24dc160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24e1827640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24dc0cf5d0 0x7f24dc159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24e1827640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 0x7f24dc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24e1827640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc0d45c0 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24e1827640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f24dc0d4440 con 0x7f24dc0cf5d0 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24e1827640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f24dc0d4740 con 0x7f24dc0b9b70 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 0x7f24dc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24db7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 0x7f24dc0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36674/0 (socket says 172.21.15.22:36674) 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24db7fe640 1 -- 172.21.15.22:0/2653824079 learned_addr learned my addr 172.21.15.22:0/2653824079 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24da7fc640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24dc0cf5d0 0x7f24dc159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.921+0000 7f24daffd640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24dc0b9b70 0x7f24dc160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24db7fe640 1 -- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24dc0b9b70 msgr2=0x7f24dc160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24db7fe640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24dc0b9b70 0x7f24dc160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24db7fe640 1 -- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24dc0cf5d0 msgr2=0x7f24dc159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24db7fe640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24dc0cf5d0 0x7f24dc159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:27.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24db7fe640 1 -- 172.21.15.22:0/2653824079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24dc15e790 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24db7fe640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 0x7f24dc0c9770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24d400ea10 tx=0x7f24d400eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:27.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d400cde0 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24d4004c10 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4005020 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24dc15ea70 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.922+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24dc15e130 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.923+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24dc15dac0 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.924+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f24d40051c0 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.924+0000 7f24c3fff640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f24c8078b90 0x7f24c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.924+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f24d409b770 con 0x7f24dc0d2ed0 2024-09-21T05:41:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.925+0000 7f24daffd640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f24c8078b90 0x7f24c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:27.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.925+0000 7f24daffd640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f24c8078b90 0x7f24c807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009b80 tx=0x7f24cc005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:27.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:27.929+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f24d4063850 con 0x7f24dc0d2ed0 2024-09-21T05:41:28.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.156+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24dc0cf9e0 con 0x7f24dc0d2ed0 2024-09-21T05:41:28.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.156+0000 7f24c3fff640 1 -- 172.21.15.22:0/2653824079 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3914 (secure 0 0 0) 0x7f24d4068280 con 0x7f24dc0d2ed0 2024-09-21T05:41:28.158 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:28.159 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":28,"btime":"2024-09-21T05:41:21:890129+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3727112820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3727112820},{"type":"v1","addr":"172.21.15.134:6836","nonce":3727112820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5105,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1695888289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1695888289},{"type":"v1","addr":"172.21.15.134:6837","nonce":1695888289}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5137,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/279929996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":279929996},{"type":"v1","addr":"172.21.15.22:6835","nonce":279929996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2850693006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2850693006},{"type":"v1","addr":"172.21.15.22:6834","nonce":2850693006}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-21T05:41:28.159 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 28 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f24c8078b90 msgr2=0x7f24c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f24c8078b90 0x7f24c807b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009b80 tx=0x7f24cc005f30 comp rx=0 tx=0).stop 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 msgr2=0x7f24dc0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 0x7f24dc0c9770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24d400ea10 tx=0x7f24d400eed0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 shutdown_connections 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24dc0d2ed0 0x7f24dc0c9770 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.158+0000 7f24e1827640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24dc0cf5d0 0x7f24dc159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.159+0000 7f24e1827640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f24c8078b90 0x7f24c807b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.159+0000 7f24e1827640 1 --2- 172.21.15.22:0/2653824079 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24dc0b9b70 0x7f24dc160cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.159+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 >> 172.21.15.22:0/2653824079 conn(0x7f24dc08bfc0 msgr2=0x7f24dc0b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:28.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.159+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 shutdown_connections 2024-09-21T05:41:28.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.159+0000 7f24e1827640 1 -- 172.21.15.22:0/2653824079 wait complete. 2024-09-21T05:41:28.172 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:41:28.173 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:41:28.173 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:41:28.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:41:28.173 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:41:28.173 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:41:28.173 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:41:28.173 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:41:28.174 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:41:28.174 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:41:28.174 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:41:28.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:41:28.174 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:41:28.174 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:41:28.174 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:41:28.174 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:41:28.183 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:41:28.183 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:41:28.184 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:41:28.184 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:41:28.188 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:41:28.188 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:41:28.189 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:41:28.189 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:41:28.226 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:41:28.226 DEBUG:teuthology.parallel:result is None 2024-09-21T05:41:28.226 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:41:28.227 DEBUG:teuthology.parallel:result is None 2024-09-21T05:41:28.231 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:41:28.232 DEBUG:teuthology.parallel:result is None 2024-09-21T05:41:28.233 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:41:28.233 DEBUG:teuthology.parallel:result is None 2024-09-21T05:41:28.234 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:41:28.314 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:41:28.314 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:41:28.329 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:41:28.330 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:41:28.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 -- 172.21.15.22:0/2376525208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1398003c30 msgr2=0x7f1398004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 --2- 172.21.15.22:0/2376525208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1398003c30 0x7f1398004080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13a4001850 tx=0x7f1398005830 comp rx=0 tx=0).stop 2024-09-21T05:41:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 -- 172.21.15.22:0/2376525208 shutdown_connections 2024-09-21T05:41:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 --2- 172.21.15.22:0/2376525208 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1398003c30 0x7f1398004080 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 -- 172.21.15.22:0/2376525208 >> 172.21.15.22:0/2376525208 conn(0x7f13ac0d12a0 msgr2=0x7f13ac0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 -- 172.21.15.22:0/2376525208 shutdown_connections 2024-09-21T05:41:28.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.355+0000 7f13b0870640 1 -- 172.21.15.22:0/2376525208 wait complete. 2024-09-21T05:41:28.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.357+0000 7f13b0870640 1 Processor -- start 2024-09-21T05:41:28.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.357+0000 7f13b0870640 1 -- start start 2024-09-21T05:41:28.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.358+0000 7f13b0870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1398003c30 0x7f13ac1da080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.358+0000 7f13b0870640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 0x7f13ac1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.358+0000 7f13b0870640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ac0cd400 0x7f13ac1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.358+0000 7f13b0870640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac0b0de0 con 0x7f13ac0ccff0 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.358+0000 7f13b0870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13ac0b20d0 con 0x7f1398003c30 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.358+0000 7f13b0870640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13ac0b0f60 con 0x7f13ac0cd400 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 0x7f13ac1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 0x7f13ac1f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36718/0 (socket says 172.21.15.22:36718) 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 -- 172.21.15.22:0/2017357167 learned_addr learned my addr 172.21.15.22:0/2017357167 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13aad9d640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ac0cd400 0x7f13ac1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13aa59c640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1398003c30 0x7f13ac1da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 -- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ac0cd400 msgr2=0x7f13ac1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ac0cd400 0x7f13ac1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 -- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1398003c30 msgr2=0x7f13ac1da080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1398003c30 0x7f13ac1da080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 -- 172.21.15.22:0/2017357167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ac0cc830 con 0x7f13ac0ccff0 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a9d9b640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 0x7f13ac1f4600 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1394006480 tx=0x7f13940039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394015070 con 0x7f13ac0ccff0 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1394011860 con 0x7f13ac0ccff0 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ac1e3180 con 0x7f13ac0ccff0 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394011cb0 con 0x7f13ac0ccff0 2024-09-21T05:41:28.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.359+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ac1e3740 con 0x7f13ac0ccff0 2024-09-21T05:41:28.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.361+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ac06cde0 con 0x7f13ac0ccff0 2024-09-21T05:41:28.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.361+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1394018070 con 0x7f13ac0ccff0 2024-09-21T05:41:28.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.362+0000 7f13a2ffd640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f138c078bc0 0x7f138c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:28.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.362+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4943+0+0 (secure 0 0 0) 0x7f139409b580 con 0x7f13ac0ccff0 2024-09-21T05:41:28.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.362+0000 7f13aa59c640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f138c078bc0 0x7f138c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:28.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.363+0000 7f13aa59c640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f138c078bc0 0x7f138c07b0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ceda0 tx=0x7f13a40013b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:28.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.367+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13940635e0 con 0x7f13ac0ccff0 2024-09-21T05:41:28.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:28.565+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f13ac0b0780 con 0x7f13ac0ccff0 2024-09-21T05:41:29.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.603+0000 7f13a2ffd640 1 -- 172.21.15.22:0/2017357167 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) ==== 97+0+0 (secure 0 0 0) 0x7f1394068010 con 0x7f13ac0ccff0 2024-09-21T05:41:29.606 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f138c078bc0 msgr2=0x7f138c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f138c078bc0 0x7f138c07b0d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f13ac0ceda0 tx=0x7f13a40013b0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 msgr2=0x7f13ac1f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 0x7f13ac1f4600 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1394006480 tx=0x7f13940039c0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 shutdown_connections 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f138c078bc0 0x7f138c07b0d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13ac0cd400 0x7f13ac1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13ac0ccff0 0x7f13ac1f4600 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 --2- 172.21.15.22:0/2017357167 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1398003c30 0x7f13ac1da080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 >> 172.21.15.22:0/2017357167 conn(0x7f13ac0d12a0 msgr2=0x7f13ac070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 shutdown_connections 2024-09-21T05:41:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.606+0000 7f13b0870640 1 -- 172.21.15.22:0/2017357167 wait complete. 2024-09-21T05:41:29.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 -- 172.21.15.22:0/318172123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41300053f0 msgr2=0x7f413000ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 --2- 172.21.15.22:0/318172123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41300053f0 0x7f413000ba70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4140001d90 tx=0x7f4130005830 comp rx=0 tx=0).stop 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 -- 172.21.15.22:0/318172123 shutdown_connections 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 --2- 172.21.15.22:0/318172123 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41300053f0 0x7f413000ba70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 -- 172.21.15.22:0/318172123 >> 172.21.15.22:0/318172123 conn(0x7f414809d7f0 msgr2=0x7f41480d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 -- 172.21.15.22:0/318172123 shutdown_connections 2024-09-21T05:41:29.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.745+0000 7f414d16e640 1 -- 172.21.15.22:0/318172123 wait complete. 2024-09-21T05:41:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.747+0000 7f414d16e640 1 Processor -- start 2024-09-21T05:41:29.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.748+0000 7f414d16e640 1 -- start start 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414d16e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 0x7f41481540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414d16e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4148160510 0x7f41481e8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414d16e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41481e8fa0 0x7f41481f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414d16e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41480b6120 con 0x7f4148161b40 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414d16e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41480b5fa0 con 0x7f41481e8fa0 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414d16e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f41480b62a0 con 0x7f4148160510 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 0x7f41481540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:29.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 0x7f41481540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36738/0 (socket says 172.21.15.22:36738) 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 -- 172.21.15.22:0/2006647886 learned_addr learned my addr 172.21.15.22:0/2006647886 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414659c640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4148160510 0x7f41481e8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414759e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41481e8fa0 0x7f41481f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 -- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4148160510 msgr2=0x7f41481e8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4148160510 0x7f41481e8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 -- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41481e8fa0 msgr2=0x7f41481f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41481e8fa0 0x7f41481f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 -- 172.21.15.22:0/2006647886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41481e93b0 con 0x7f4148161b40 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f414659c640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4148160510 0x7f41481e8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.749+0000 7f4146d9d640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 0x7f41481540d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4138005f00 tx=0x7f4138011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.750+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413800fc70 con 0x7f4148161b40 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.750+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f413800fe10 con 0x7f4148161b40 2024-09-21T05:41:29.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.750+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004290 con 0x7f4148161b40 2024-09-21T05:41:29.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.750+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4148154610 con 0x7f4148161b40 2024-09-21T05:41:29.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.750+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41480b25b0 con 0x7f4148161b40 2024-09-21T05:41:29.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.751+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41480b67b0 con 0x7f4148161b40 2024-09-21T05:41:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.751+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41380035f0 con 0x7f4148161b40 2024-09-21T05:41:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.752+0000 7f412f7fe640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4134078b90 0x7f413407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.752+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f413809b460 con 0x7f4148161b40 2024-09-21T05:41:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.752+0000 7f414659c640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4134078b90 0x7f413407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:29.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.753+0000 7f414659c640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4134078b90 0x7f413407b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f413c001860 tx=0x7f413c001200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:29.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:29.757+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41481e93b0 con 0x7f4148161b40 2024-09-21T05:41:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.016+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f41480d3fb0 con 0x7f4148161b40 2024-09-21T05:41:30.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.016+0000 7f412f7fe640 1 -- 172.21.15.22:0/2006647886 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f4138063c70 con 0x7f4148161b40 2024-09-21T05:41:30.018 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:30.018 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:30.018 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:41:30.019 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:41:30.020 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:41:30.021 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.022 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.023 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.024 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.024 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.024 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.024 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:41:30.024 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.026 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:41:30.027 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow r", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:41:30.028 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:41:30.029 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.030 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.031 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:41:30.032 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:30.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4134078b90 msgr2=0x7f413407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4134078b90 0x7f413407b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f413c001860 tx=0x7f413c001200 comp rx=0 tx=0).stop 2024-09-21T05:41:30.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 msgr2=0x7f41481540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 0x7f41481540d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4138005f00 tx=0x7f4138011f40 comp rx=0 tx=0).stop 2024-09-21T05:41:30.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 shutdown_connections 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41481e8fa0 0x7f41481f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4134078b90 0x7f413407b0a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4148160510 0x7f41481e8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 --2- 172.21.15.22:0/2006647886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4148161b40 0x7f41481540d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 >> 172.21.15.22:0/2006647886 conn(0x7f414809d7f0 msgr2=0x7f414815bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 shutdown_connections 2024-09-21T05:41:30.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.018+0000 7f414d16e640 1 -- 172.21.15.22:0/2006647886 wait complete. 2024-09-21T05:41:30.034 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:41:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 -- 172.21.15.22:0/546528934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b00b66b0 msgr2=0x7f71b00d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 --2- 172.21.15.22:0/546528934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b00b66b0 0x7f71b00d4d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f71b00d00e0 tx=0x7f71a402a410 comp rx=0 tx=0).stop 2024-09-21T05:41:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 -- 172.21.15.22:0/546528934 shutdown_connections 2024-09-21T05:41:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 --2- 172.21.15.22:0/546528934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b00b66b0 0x7f71b00d4d80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 -- 172.21.15.22:0/546528934 >> 172.21.15.22:0/546528934 conn(0x7f71b009d7f0 msgr2=0x7f71b00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:30.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 -- 172.21.15.22:0/546528934 shutdown_connections 2024-09-21T05:41:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.149+0000 7f71b6120640 1 -- 172.21.15.22:0/546528934 wait complete. 2024-09-21T05:41:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.150+0000 7f71b6120640 1 Processor -- start 2024-09-21T05:41:30.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.151+0000 7f71b6120640 1 -- start start 2024-09-21T05:41:30.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71b6120640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b00b9b60 0x7f71b00c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71b6120640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b00cf540 0x7f71b01f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71b6120640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 0x7f71b0161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71b6120640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71b006f7f0 con 0x7f71b0160e20 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71b6120640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71b006f670 con 0x7f71b00b9b60 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71b6120640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71b006f970 con 0x7f71b00cf540 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 0x7f71b0161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 0x7f71b0161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36754/0 (socket says 172.21.15.22:36754) 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 -- 172.21.15.22:0/1472107650 learned_addr learned my addr 172.21.15.22:0/1472107650 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71af7fe640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b00b9b60 0x7f71b00c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71aeffd640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b00cf540 0x7f71b01f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 -- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b00cf540 msgr2=0x7f71b01f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b00cf540 0x7f71b01f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 -- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b00b9b60 msgr2=0x7f71b00c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b00b9b60 0x7f71b00c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.152+0000 7f71affff640 1 -- 172.21.15.22:0/1472107650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b00b4dd0 con 0x7f71b0160e20 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.153+0000 7f71affff640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 0x7f71b0161230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f71a800cad0 tx=0x7f71a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.153+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8016020 con 0x7f71b0160e20 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.153+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a800ad20 con 0x7f71b0160e20 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.153+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b01f6800 con 0x7f71b0160e20 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.153+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8012680 con 0x7f71b0160e20 2024-09-21T05:41:30.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.153+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b00b6970 con 0x7f71b0160e20 2024-09-21T05:41:30.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.154+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b00d49b0 con 0x7f71b0160e20 2024-09-21T05:41:30.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.155+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71a8019020 con 0x7f71b0160e20 2024-09-21T05:41:30.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.155+0000 7f7197fff640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f719c078b90 0x7f719c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.155+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f71a809b140 con 0x7f71b0160e20 2024-09-21T05:41:30.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.155+0000 7f71af7fe640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f719c078b90 0x7f719c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.156+0000 7f71af7fe640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f719c078b90 0x7f719c07b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b7f0 tx=0x7f71a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:30.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.160+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71b00d49b0 con 0x7f71b0160e20 2024-09-21T05:41:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.347+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f71b00b5650 con 0x7f71b0160e20 2024-09-21T05:41:30.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.347+0000 7f7197fff640 1 -- 172.21.15.22:0/1472107650 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) ==== 84+0+65 (secure 0 0 0) 0x7f71a8020020 con 0x7f71b0160e20 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:30.350 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f719c078b90 msgr2=0x7f719c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f719c078b90 0x7f719c07b0a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f71a000b7f0 tx=0x7f71a0008040 comp rx=0 tx=0).stop 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 msgr2=0x7f71b0161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 0x7f71b0161230 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f71a800cad0 tx=0x7f71a800cf90 comp rx=0 tx=0).stop 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 shutdown_connections 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b0160e20 0x7f71b0161230 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f719c078b90 0x7f719c07b0a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b00cf540 0x7f71b01f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 --2- 172.21.15.22:0/1472107650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b00b9b60 0x7f71b00c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 >> 172.21.15.22:0/1472107650 conn(0x7f71b009d7f0 msgr2=0x7f71b00b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 shutdown_connections 2024-09-21T05:41:30.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.350+0000 7f71b6120640 1 -- 172.21.15.22:0/1472107650 wait complete. 2024-09-21T05:41:30.363 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:41:30.363 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/3643615964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 msgr2=0x7f0dc40cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/3643615964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 0x7f0dc40cf710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc005fd0 tx=0x7f0dbc02a820 comp rx=0 tx=0).stop 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/3643615964 shutdown_connections 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/3643615964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 0x7f0dc40cf710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/3643615964 >> v2:172.21.15.134:3300/0 conn(0x7f0dc4162190 0x7f0dc40d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/3643615964 >> 172.21.15.22:0/3643615964 conn(0x7f0dc409d7c0 msgr2=0x7f0dc408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/3643615964 shutdown_connections 2024-09-21T05:41:30.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.483+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/3643615964 wait complete. 2024-09-21T05:41:30.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.485+0000 7f0dcc4ec640 1 Processor -- start 2024-09-21T05:41:30.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.485+0000 7f0dcc4ec640 1 -- start start 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dcc4ec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 0x7f0dc40caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dcc4ec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 0x7f0dc40a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dcc4ec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc4162190 0x7f0dc40c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dcc4ec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc40cfdd0 con 0x7f0dc40b32a0 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dcc4ec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc40cfc50 con 0x7f0dc4162190 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dcc4ec640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc40cff50 con 0x7f0dc40cf300 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.486+0000 7f0dca288640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 0x7f0dc40caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 0x7f0dc40caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36770/0 (socket says 172.21.15.22:36770) 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 -- 172.21.15.22:0/2295702805 learned_addr learned my addr 172.21.15.22:0/2295702805 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dc9a87640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 0x7f0dc40a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dcaa89640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc4162190 0x7f0dc40c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 -- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 msgr2=0x7f0dc40a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 0x7f0dc40a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 -- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc4162190 msgr2=0x7f0dc40c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc4162190 0x7f0dc40c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 -- 172.21.15.22:0/2295702805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc40a1f00 con 0x7f0dc40b32a0 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dca288640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 0x7f0dc40caa40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c3d0 tx=0x7f0dc000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0011790 con 0x7f0dc40b32a0 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dc0017070 con 0x7f0dc40b32a0 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc40cdbb0 con 0x7f0dc40b32a0 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0011cc0 con 0x7f0dc40b32a0 2024-09-21T05:41:30.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.487+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc40ce170 con 0x7f0dc40b32a0 2024-09-21T05:41:30.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.488+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc40cf710 con 0x7f0dc40b32a0 2024-09-21T05:41:30.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.489+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0dc001c460 con 0x7f0dc40b32a0 2024-09-21T05:41:30.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.490+0000 7f0dbaffd640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0da8078b90 0x7f0da807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:30.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.490+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 4857+0+0 (secure 0 0 0) 0x7f0dc009a660 con 0x7f0dc40b32a0 2024-09-21T05:41:30.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.490+0000 7f0dc9a87640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0da8078b90 0x7f0da807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:30.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.491+0000 7f0dc9a87640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0da8078b90 0x7f0da807b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40d1d00 tx=0x7f0db4005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:30.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.495+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0dc00627a0 con 0x7f0dc40b32a0 2024-09-21T05:41:30.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:30.681+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0dc4160f30 con 0x7f0dc40b32a0 2024-09-21T05:41:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.606+0000 7f0dbaffd640 1 -- 172.21.15.22:0/2295702805 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) ==== 138+0+0 (secure 0 0 0) 0x7f0dc00671d0 con 0x7f0dc40b32a0 2024-09-21T05:41:31.609 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:41:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0da8078b90 msgr2=0x7f0da807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0da8078b90 0x7f0da807b0a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40d1d00 tx=0x7f0db4005f90 comp rx=0 tx=0).stop 2024-09-21T05:41:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 msgr2=0x7f0dc40caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 0x7f0dc40caa40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000c3d0 tx=0x7f0dc000c890 comp rx=0 tx=0).stop 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 shutdown_connections 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0dc4162190 0x7f0dc40c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0da8078b90 0x7f0da807b0a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0dc40cf300 0x7f0dc40a19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 --2- 172.21.15.22:0/2295702805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0dc40b32a0 0x7f0dc40caa40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 >> 172.21.15.22:0/2295702805 conn(0x7f0dc409d7c0 msgr2=0x7f0dc4160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 shutdown_connections 2024-09-21T05:41:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.610+0000 7f0dcc4ec640 1 -- 172.21.15.22:0/2295702805 wait complete. 2024-09-21T05:41:31.630 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.758+0000 7f00cd9cc640 1 -- 172.21.15.22:0/308993676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 msgr2=0x7f00c80cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.758+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/308993676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80cba00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f00c8070dc0 tx=0x7f00b802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.758+0000 7f00cd9cc640 1 -- 172.21.15.22:0/308993676 shutdown_connections 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.758+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/308993676 >> v2:172.21.15.134:3300/0 conn(0x7f00c80706b0 0x7f00c80735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.758+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/308993676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80cba00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.759+0000 7f00cd9cc640 1 -- 172.21.15.22:0/308993676 >> 172.21.15.22:0/308993676 conn(0x7f00c80b1cb0 msgr2=0x7f00c80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.759+0000 7f00cd9cc640 1 -- 172.21.15.22:0/308993676 shutdown_connections 2024-09-21T05:41:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.759+0000 7f00cd9cc640 1 -- 172.21.15.22:0/308993676 wait complete. 2024-09-21T05:41:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.760+0000 7f00cd9cc640 1 Processor -- start 2024-09-21T05:41:31.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.761+0000 7f00cd9cc640 1 -- start start 2024-09-21T05:41:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00cd9cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00cd9cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00c80cad30 0x7f00c814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00cd9cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00c81598f0 0x7f00c80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00cd9cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8073ec0 con 0x7f00c80ca800 2024-09-21T05:41:31.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00cd9cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8073d40 con 0x7f00c80cad30 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00cd9cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00c8074040 con 0x7f00c81598f0 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36782/0 (socket says 172.21.15.22:36782) 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 -- 172.21.15.22:0/3385670702 learned_addr learned my addr 172.21.15.22:0/3385670702 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c67fc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00c80cad30 0x7f00c814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c77fe640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00c81598f0 0x7f00c80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 -- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00c81598f0 msgr2=0x7f00c80ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00c81598f0 0x7f00c80ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 -- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00c80cad30 msgr2=0x7f00c814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00c80cad30 0x7f00c814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.762+0000 7f00c6ffd640 1 -- 172.21.15.22:0/3385670702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c80c9440 con 0x7f00c80ca800 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.763+0000 7f00c6ffd640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80b1580 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009b30 tx=0x7f00b8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:31.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.763+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8031da0 con 0x7f00c80ca800 2024-09-21T05:41:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.763+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00b8003e10 con 0x7f00c80ca800 2024-09-21T05:41:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.763+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c814eac0 con 0x7f00c80ca800 2024-09-21T05:41:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.763+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b80332b0 con 0x7f00c80ca800 2024-09-21T05:41:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.763+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c80c8ef0 con 0x7f00c80ca800 2024-09-21T05:41:31.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.764+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c80745d0 con 0x7f00c80ca800 2024-09-21T05:41:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.765+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f00b8033450 con 0x7f00c80ca800 2024-09-21T05:41:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.765+0000 7f00affff640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f00b4078b90 0x7f00b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.766+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5227+0+0 (secure 0 0 0) 0x7f00b80bc090 con 0x7f00c80ca800 2024-09-21T05:41:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.766+0000 7f00c67fc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f00b4078b90 0x7f00b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:31.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.766+0000 7f00c67fc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f00b4078b90 0x7f00b407b0a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00c81f8940 tx=0x7f00bc007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:31.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.771+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00b803f030 con 0x7f00c80ca800 2024-09-21T05:41:31.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:31.961+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f00c815d6e0 con 0x7f00c80ca800 2024-09-21T05:41:32.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.612+0000 7f00affff640 1 -- 172.21.15.22:0/3385670702 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) ==== 170+0+0 (secure 0 0 0) 0x7f00b8084180 con 0x7f00c80ca800 2024-09-21T05:41:32.615 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f00b4078b90 msgr2=0x7f00b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f00b4078b90 0x7f00b407b0a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f00c81f8940 tx=0x7f00bc007e90 comp rx=0 tx=0).stop 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 msgr2=0x7f00c80b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80b1580 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00b8009b30 tx=0x7f00b8003c20 comp rx=0 tx=0).stop 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 shutdown_connections 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f00b4078b90 0x7f00b407b0a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00c81598f0 0x7f00c80ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00c80cad30 0x7f00c814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 --2- 172.21.15.22:0/3385670702 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00c80ca800 0x7f00c80b1580 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.616+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 >> 172.21.15.22:0/3385670702 conn(0x7f00c80b1cb0 msgr2=0x7f00c815b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.617+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 shutdown_connections 2024-09-21T05:41:32.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.617+0000 7f00cd9cc640 1 -- 172.21.15.22:0/3385670702 wait complete. 2024-09-21T05:41:32.651 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 -- 172.21.15.22:0/1311945996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80d16c0 msgr2=0x7f72b80cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 --2- 172.21.15.22:0/1311945996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80d16c0 0x7f72b80cf310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f72ac005fd0 tx=0x7f72ac02a540 comp rx=0 tx=0).stop 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 -- 172.21.15.22:0/1311945996 shutdown_connections 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 --2- 172.21.15.22:0/1311945996 >> v2:172.21.15.134:3300/0 conn(0x7f72b80b3a10 0x7f72b80cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 --2- 172.21.15.22:0/1311945996 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80d16c0 0x7f72b80cf310 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 -- 172.21.15.22:0/1311945996 >> 172.21.15.22:0/1311945996 conn(0x7f72b809d8f0 msgr2=0x7f72b80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 -- 172.21.15.22:0/1311945996 shutdown_connections 2024-09-21T05:41:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.774+0000 7f72be45c640 1 -- 172.21.15.22:0/1311945996 wait complete. 2024-09-21T05:41:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.776+0000 7f72be45c640 1 Processor -- start 2024-09-21T05:41:32.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.776+0000 7f72be45c640 1 -- start start 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72be45c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b80b3a10 0x7f72b8154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72be45c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 0x7f72b80c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72be45c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80ca530 0x7f72b80ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72be45c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f72b80d0120 con 0x7f72b80b9b60 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72be45c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f72b80cffa0 con 0x7f72b80b3a10 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72be45c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f72b80d02a0 con 0x7f72b80ca530 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 0x7f72b80c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 0x7f72b80c9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36796/0 (socket says 172.21.15.22:36796) 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 -- 172.21.15.22:0/3074314535 learned_addr learned my addr 172.21.15.22:0/3074314535 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72bc9f9640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80ca530 0x7f72b80ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b7fff640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b80b3a10 0x7f72b8154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 -- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80ca530 msgr2=0x7f72b80ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80ca530 0x7f72b80ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 -- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b80b3a10 msgr2=0x7f72b8154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b80b3a10 0x7f72b8154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 -- 172.21.15.22:0/3074314535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b80c8c10 con 0x7f72b80b9b60 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b77fe640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 0x7f72b80c9ff0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72ac034040 tx=0x7f72ac0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.778+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac03e030 con 0x7f72b80b9b60 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.779+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72ac003ca0 con 0x7f72b80b9b60 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.779+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b80c8ef0 con 0x7f72b80b9b60 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.779+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ac035040 con 0x7f72b80b9b60 2024-09-21T05:41:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.779+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b80c94b0 con 0x7f72b80b9b60 2024-09-21T05:41:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.780+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b80cf310 con 0x7f72b80b9b60 2024-09-21T05:41:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.781+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f72ac041030 con 0x7f72b80b9b60 2024-09-21T05:41:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.781+0000 7f72b4ff9640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f72a4078b90 0x7f72a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.781+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5609+0+0 (secure 0 0 0) 0x7f72ac0bb150 con 0x7f72b80b9b60 2024-09-21T05:41:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.781+0000 7f72b7fff640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f72a4078b90 0x7f72a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:32.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.782+0000 7f72b7fff640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f72a4078b90 0x7f72a407b0a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f72a8007fd0 tx=0x7f72a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:32.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:32.786+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f72ac083030 con 0x7f72b80b9b60 2024-09-21T05:41:33.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.012+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f72b80ba0d0 con 0x7f72b80b9b60 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.614+0000 7f72b4ff9640 1 -- 172.21.15.22:0/3074314535 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) ==== 172+0+0 (secure 0 0 0) 0x7f72ac087a60 con 0x7f72b80b9b60 2024-09-21T05:41:33.616 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 8 and data pool 9 2024-09-21T05:41:33.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.624+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f72a4078b90 msgr2=0x7f72a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.624+0000 7f72be45c640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f72a4078b90 0x7f72a407b0a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f72a8007fd0 tx=0x7f72a8009040 comp rx=0 tx=0).stop 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.624+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 msgr2=0x7f72b80c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.624+0000 7f72be45c640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 0x7f72b80c9ff0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f72ac034040 tx=0x7f72ac0038a0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.641+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 shutdown_connections 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.641+0000 7f72be45c640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f72a4078b90 0x7f72a407b0a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.641+0000 7f72be45c640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f72b80ca530 0x7f72b80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.641+0000 7f72be45c640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f72b80b9b60 0x7f72b80c9ff0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.641+0000 7f72be45c640 1 --2- 172.21.15.22:0/3074314535 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f72b80b3a10 0x7f72b8154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.641+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 >> 172.21.15.22:0/3074314535 conn(0x7f72b809d8f0 msgr2=0x7f72b8168700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:33.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.642+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 shutdown_connections 2024-09-21T05:41:33.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.643+0000 7f72be45c640 1 -- 172.21.15.22:0/3074314535 wait complete. 2024-09-21T05:41:33.665 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:41:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 -- 172.21.15.22:0/849551681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500b5fa0 msgr2=0x7fc0500b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 --2- 172.21.15.22:0/849551681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500b5fa0 0x7fc0500b63b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc05008d160 tx=0x7fc04402a5f0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 -- 172.21.15.22:0/849551681 shutdown_connections 2024-09-21T05:41:33.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 --2- 172.21.15.22:0/849551681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500b5fa0 0x7fc0500b63b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 --2- 172.21.15.22:0/849551681 >> v2:172.21.15.134:3300/0 conn(0x7fc0500b9970 0x7fc0500b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 -- 172.21.15.22:0/849551681 >> 172.21.15.22:0/849551681 conn(0x7fc05009d7f0 msgr2=0x7fc0500d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 -- 172.21.15.22:0/849551681 shutdown_connections 2024-09-21T05:41:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.783+0000 7fc05648c640 1 -- 172.21.15.22:0/849551681 wait complete. 2024-09-21T05:41:33.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.785+0000 7fc05648c640 1 Processor -- start 2024-09-21T05:41:33.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.785+0000 7fc05648c640 1 -- start start 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc05648c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0500b5fa0 0x7fc05015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc05648c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0500b9970 0x7fc050159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc05648c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 0x7fc0501e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:33.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc05648c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc05006f270 con 0x7fc0500ced80 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc05648c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc05006f0f0 con 0x7fc0500b5fa0 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc05648c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc05006f3f0 con 0x7fc0500b9970 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc054a29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 0x7fc0501e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc054a29640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 0x7fc0501e60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36820/0 (socket says 172.21.15.22:36820) 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.786+0000 7fc054a29640 1 -- 172.21.15.22:0/1712751524 learned_addr learned my addr 172.21.15.22:0/1712751524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc04f7fe640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0500b9970 0x7fc050159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc04ffff640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0500b5fa0 0x7fc05015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc054a29640 1 -- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0500b9970 msgr2=0x7fc050159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc054a29640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0500b9970 0x7fc050159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc054a29640 1 -- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0500b5fa0 msgr2=0x7fc05015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc054a29640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0500b5fa0 0x7fc05015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc054a29640 1 -- 172.21.15.22:0/1712751524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0501e5060 con 0x7fc0500ced80 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc054a29640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 0x7fc0501e60c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc04800c6e0 tx=0x7fc04800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048003b70 con 0x7fc0500ced80 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc048003d30 con 0x7fc0500ced80 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05015c390 con 0x7fc0500ced80 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc048005330 con 0x7fc0500ced80 2024-09-21T05:41:33.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.787+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0501e55e0 con 0x7fc0500ced80 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.788+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05015b540 con 0x7fc0500ced80 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.789+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc0480054d0 con 0x7fc0500ced80 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.790+0000 7fc04cff9640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc03c078bc0 0x7fc03c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.790+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fc04809b690 con 0x7fc0500ced80 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.790+0000 7fc04ffff640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc03c078bc0 0x7fc03c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:33.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.790+0000 7fc04ffff640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc03c078bc0 0x7fc03c07b0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc040005fd0 tx=0x7fc040005dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:33.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.793+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc0500b68f0 con 0x7fc0500ced80 2024-09-21T05:41:33.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.982+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0500cfc40 con 0x7fc0500ced80 2024-09-21T05:41:33.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.983+0000 7fc04cff9640 1 -- 172.21.15.22:0/1712751524 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16726 (secure 0 0 0) 0x7fc048063a30 con 0x7fc0500ced80 2024-09-21T05:41:33.985 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:33.985 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":56,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:33.013923+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:41:30.686369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T05:41:31.966340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.985+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc03c078bc0 msgr2=0x7fc03c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.985+0000 7fc05648c640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc03c078bc0 0x7fc03c07b0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc040005fd0 tx=0x7fc040005dc0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.985+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 msgr2=0x7fc0501e60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.985+0000 7fc05648c640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 0x7fc0501e60c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc04800c6e0 tx=0x7fc04800cba0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 shutdown_connections 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc0500ced80 0x7fc0501e60c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc03c078bc0 0x7fc03c07b0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc0500b9970 0x7fc050159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 --2- 172.21.15.22:0/1712751524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc0500b5fa0 0x7fc05015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:33.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 >> 172.21.15.22:0/1712751524 conn(0x7fc05009d7f0 msgr2=0x7fc0500b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:33.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 shutdown_connections 2024-09-21T05:41:33.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:33.986+0000 7fc05648c640 1 -- 172.21.15.22:0/1712751524 wait complete. 2024-09-21T05:41:34.000 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:41:34.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.114+0000 7fb860f89640 1 -- 172.21.15.22:0/3871228110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c0d2630 msgr2=0x7fb85c15d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.114+0000 7fb860f89640 1 --2- 172.21.15.22:0/3871228110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c0d2630 0x7fb85c15d130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb850009830 tx=0x7fb85002a510 comp rx=0 tx=0).stop 2024-09-21T05:41:34.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.115+0000 7fb860f89640 1 -- 172.21.15.22:0/3871228110 shutdown_connections 2024-09-21T05:41:34.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.115+0000 7fb860f89640 1 --2- 172.21.15.22:0/3871228110 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c15d670 0x7fb85c0b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.115+0000 7fb860f89640 1 --2- 172.21.15.22:0/3871228110 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c0d2630 0x7fb85c15d130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.115+0000 7fb860f89640 1 -- 172.21.15.22:0/3871228110 >> 172.21.15.22:0/3871228110 conn(0x7fb85c06d550 msgr2=0x7fb85c071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:34.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.115+0000 7fb860f89640 1 -- 172.21.15.22:0/3871228110 shutdown_connections 2024-09-21T05:41:34.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.115+0000 7fb860f89640 1 -- 172.21.15.22:0/3871228110 wait complete. 2024-09-21T05:41:34.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.117+0000 7fb860f89640 1 Processor -- start 2024-09-21T05:41:34.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.117+0000 7fb860f89640 1 -- start start 2024-09-21T05:41:34.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb860f89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c1570e0 0x7fb85c14b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb860f89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 0x7fb85c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb860f89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c0c98f0 0x7fb85c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb860f89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c0b7d60 con 0x7fb85c15d670 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb860f89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c0b7be0 con 0x7fb85c1570e0 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb860f89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb85c0b7ee0 con 0x7fb85c0c98f0 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 0x7fb85c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb859d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 0x7fb85c0c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36830/0 (socket says 172.21.15.22:36830) 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb859d9b640 1 -- 172.21.15.22:0/3062234802 learned_addr learned my addr 172.21.15.22:0/3062234802 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb85ad9d640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c0c98f0 0x7fb85c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb85a59c640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c1570e0 0x7fb85c14b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.118+0000 7fb859d9b640 1 -- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c0c98f0 msgr2=0x7fb85c0c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb859d9b640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c0c98f0 0x7fb85c0c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb859d9b640 1 -- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c1570e0 msgr2=0x7fb85c14b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb859d9b640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c1570e0 0x7fb85c14b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb859d9b640 1 -- 172.21.15.22:0/3062234802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c0cb790 con 0x7fb85c15d670 2024-09-21T05:41:34.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb85ad9d640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c0c98f0 0x7fb85c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:41:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb859d9b640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 0x7fb85c0c93b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb85002ab00 tx=0x7fb850033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850005c90 con 0x7fb85c15d670 2024-09-21T05:41:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb850031e50 con 0x7fb85c15d670 2024-09-21T05:41:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c0cba70 con 0x7fb85c15d670 2024-09-21T05:41:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850005260 con 0x7fb85c15d670 2024-09-21T05:41:34.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.119+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c0c7900 con 0x7fb85c15d670 2024-09-21T05:41:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.120+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85c0cbcb0 con 0x7fb85c15d670 2024-09-21T05:41:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.121+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb850005400 con 0x7fb85c15d670 2024-09-21T05:41:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.121+0000 7fb84affd640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb83c078bc0 0x7fb83c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.121+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7fb8500bbbc0 con 0x7fb85c15d670 2024-09-21T05:41:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.121+0000 7fb85a59c640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb83c078bc0 0x7fb83c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.122+0000 7fb85a59c640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb83c078bc0 0x7fb83c07b0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb84c001e20 tx=0x7fb84c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:34.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.126+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb85003f030 con 0x7fb85c15d670 2024-09-21T05:41:34.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.313+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb85c0cb520 con 0x7fb85c15d670 2024-09-21T05:41:34.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.314+0000 7fb84affd640 1 -- 172.21.15.22:0/3062234802 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) ==== 74+0+16726 (secure 0 0 0) 0x7fb850083ad0 con 0x7fb85c15d670 2024-09-21T05:41:34.317 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:34.317 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":56,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:33.013923+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:41:30.686369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T05:41:31.966340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb83c078bc0 msgr2=0x7fb83c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb83c078bc0 0x7fb83c07b0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb84c001e20 tx=0x7fb84c009040 comp rx=0 tx=0).stop 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 msgr2=0x7fb85c0c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 0x7fb85c0c93b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb85002ab00 tx=0x7fb850033c20 comp rx=0 tx=0).stop 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 shutdown_connections 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb83c078bc0 0x7fb83c07b0d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb85c0c98f0 0x7fb85c0c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb85c15d670 0x7fb85c0c93b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 --2- 172.21.15.22:0/3062234802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb85c1570e0 0x7fb85c14b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 >> 172.21.15.22:0/3062234802 conn(0x7fb85c06d550 msgr2=0x7fb85c0b15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:34.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 shutdown_connections 2024-09-21T05:41:34.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.317+0000 7fb860f89640 1 -- 172.21.15.22:0/3062234802 wait complete. 2024-09-21T05:41:34.331 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:41:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 -- 172.21.15.22:0/1492202180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400b64e0 msgr2=0x7f1f400ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 --2- 172.21.15.22:0/1492202180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400b64e0 0x7f1f400ced80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f38005fd0 tx=0x7f1f3802aa50 comp rx=0 tx=0).stop 2024-09-21T05:41:34.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 -- 172.21.15.22:0/1492202180 shutdown_connections 2024-09-21T05:41:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 --2- 172.21.15.22:0/1492202180 >> v2:172.21.15.134:3300/0 conn(0x7f1f400b3910 0x7f1f400b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 --2- 172.21.15.22:0/1492202180 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400b64e0 0x7f1f400ced80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 --2- 172.21.15.22:0/1492202180 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400d4c30 0x7f1f400b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 -- 172.21.15.22:0/1492202180 >> 172.21.15.22:0/1492202180 conn(0x7f1f4009d830 msgr2=0x7f1f400d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 -- 172.21.15.22:0/1492202180 shutdown_connections 2024-09-21T05:41:34.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.448+0000 7f1f46307640 1 -- 172.21.15.22:0/1492202180 wait complete. 2024-09-21T05:41:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.450+0000 7f1f46307640 1 Processor -- start 2024-09-21T05:41:34.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.450+0000 7f1f46307640 1 -- start start 2024-09-21T05:41:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f400b3910 0x7f1f4014f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400b64e0 0x7f1f400b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 0x7f1f400b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400cf700 con 0x7f1f400d4c30 2024-09-21T05:41:34.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1f400cf580 con 0x7f1f400b3910 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1f400cf880 con 0x7f1f400b64e0 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 0x7f1f400b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 0x7f1f400b8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45348/0 (socket says 172.21.15.22:45348) 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 -- 172.21.15.22:0/951287464 learned_addr learned my addr 172.21.15.22:0/951287464 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f3f7fe640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400b64e0 0x7f1f400b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f3ffff640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f400b3910 0x7f1f4014f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 -- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400b64e0 msgr2=0x7f1f400b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400b64e0 0x7f1f400b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 -- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f400b3910 msgr2=0x7f1f4014f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:34.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f400b3910 0x7f1f4014f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 -- 172.21.15.22:0/951287464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f401f96a0 con 0x7f1f400d4c30 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f3ffff640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f400b3910 0x7f1f4014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f3f7fe640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400b64e0 0x7f1f400b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f448a4640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 0x7f1f400b8de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006010 tx=0x7f1f38003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38033050 con 0x7f1f400d4c30 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f38004250 con 0x7f1f400d4c30 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.452+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f401f9980 con 0x7f1f400d4c30 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.453+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f38004680 con 0x7f1f400d4c30 2024-09-21T05:41:34.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.453+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f401f9f40 con 0x7f1f400d4c30 2024-09-21T05:41:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.454+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f400ced80 con 0x7f1f400d4c30 2024-09-21T05:41:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.454+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1f38036070 con 0x7f1f400d4c30 2024-09-21T05:41:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.455+0000 7f1f3cff9640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f2c078b90 0x7f1f2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.455+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5709+0+0 (secure 0 0 0) 0x7f1f380bbfa0 con 0x7f1f400d4c30 2024-09-21T05:41:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.455+0000 7f1f3ffff640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f2c078b90 0x7f1f2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:34.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.456+0000 7f1f3ffff640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f2c078b90 0x7f1f2c07b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009a80 tx=0x7f1f30005cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:34.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.459+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f38084350 con 0x7f1f400d4c30 2024-09-21T05:41:34.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:34.686+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1f400d42b0 con 0x7f1f400d4c30 2024-09-21T05:41:35.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.643+0000 7f1f3cff9640 1 -- 172.21.15.22:0/951287464 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7f1f38004820 con 0x7f1f400d4c30 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.645+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f2c078b90 msgr2=0x7f1f2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f2c078b90 0x7f1f2c07b0a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009a80 tx=0x7f1f30005cc0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 msgr2=0x7f1f400b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 0x7f1f400b8de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006010 tx=0x7f1f38003f10 comp rx=0 tx=0).stop 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 shutdown_connections 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1f400d4c30 0x7f1f400b8de0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1f2c078b90 0x7f1f2c07b0a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1f400b64e0 0x7f1f400b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 --2- 172.21.15.22:0/951287464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1f400b3910 0x7f1f4014f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 >> 172.21.15.22:0/951287464 conn(0x7f1f4009d830 msgr2=0x7f1f400b5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 shutdown_connections 2024-09-21T05:41:35.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.646+0000 7f1f46307640 1 -- 172.21.15.22:0/951287464 wait complete. 2024-09-21T05:41:35.662 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:41:35.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:41:35.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 -- 172.21.15.22:0/2632229393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140d0ce0 msgr2=0x7f18140b9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 --2- 172.21.15.22:0/2632229393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140d0ce0 0x7f18140b9be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f181408cf00 tx=0x7f180402a560 comp rx=0 tx=0).stop 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 -- 172.21.15.22:0/2632229393 shutdown_connections 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 --2- 172.21.15.22:0/2632229393 >> v2:172.21.15.134:3300/0 conn(0x7f18140cf0b0 0x7f18140d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 --2- 172.21.15.22:0/2632229393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140d0ce0 0x7f18140b9be0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 -- 172.21.15.22:0/2632229393 >> 172.21.15.22:0/2632229393 conn(0x7f181408bfc0 msgr2=0x7f181409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 -- 172.21.15.22:0/2632229393 shutdown_connections 2024-09-21T05:41:35.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.787+0000 7f181967e640 1 -- 172.21.15.22:0/2632229393 wait complete. 2024-09-21T05:41:35.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.789+0000 7f181967e640 1 Processor -- start 2024-09-21T05:41:35.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.789+0000 7f181967e640 1 -- start start 2024-09-21T05:41:35.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f181967e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 0x7f1814151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:35.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f181967e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18140cf4c0 0x7f18140cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f181967e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18140d0ce0 0x7f1814152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f181967e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18140d4e20 con 0x7f18140cf0b0 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f181967e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f18140d4ca0 con 0x7f18140cf4c0 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f181967e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f18140d2bb0 con 0x7f18140d0ce0 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f1812ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 0x7f1814151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f1812ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 0x7f1814151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45366/0 (socket says 172.21.15.22:45366) 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f1812ffd640 1 -- 172.21.15.22:0/2044860819 learned_addr learned my addr 172.21.15.22:0/2044860819 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f18127fc640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18140cf4c0 0x7f18140cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.790+0000 7f18137fe640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18140d0ce0 0x7f1814152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f1812ffd640 1 -- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18140d0ce0 msgr2=0x7f1814152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f1812ffd640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18140d0ce0 0x7f1814152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f1812ffd640 1 -- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18140cf4c0 msgr2=0x7f18140cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f1812ffd640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18140cf4c0 0x7f18140cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f1812ffd640 1 -- 172.21.15.22:0/2044860819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18140cdff0 con 0x7f18140cf0b0 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f18127fc640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18140cf4c0 0x7f18140cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f18137fe640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18140d0ce0 0x7f1814152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f1812ffd640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 0x7f1814151a60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1804034040 tx=0x7f1804003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180403e030 con 0x7f18140cf0b0 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1804003e00 con 0x7f18140cf0b0 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804035040 con 0x7f18140cf0b0 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140cca10 con 0x7f18140cf0b0 2024-09-21T05:41:35.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.791+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140ccfd0 con 0x7f18140cf0b0 2024-09-21T05:41:35.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.792+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18140d30c0 con 0x7f18140cf0b0 2024-09-21T05:41:35.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.793+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1804041030 con 0x7f18140cf0b0 2024-09-21T05:41:35.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.793+0000 7f17fb7fe640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1800078bc0 0x7f180007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:35.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.794+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f18040bb690 con 0x7f18140cf0b0 2024-09-21T05:41:35.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.794+0000 7f18127fc640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1800078bc0 0x7f180007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:35.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.794+0000 7f18127fc640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1800078bc0 0x7f180007b0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f181406f220 tx=0x7f1808005f10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:35.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:35.798+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18040839b0 con 0x7f18140cf0b0 2024-09-21T05:41:36.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.025+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f18140d07b0 con 0x7f18140cf0b0 2024-09-21T05:41:36.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.649+0000 7f17fb7fe640 1 -- 172.21.15.22:0/2044860819 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f18040832b0 con 0x7f18140cf0b0 2024-09-21T05:41:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1800078bc0 msgr2=0x7f180007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1800078bc0 0x7f180007b0d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f181406f220 tx=0x7f1808005f10 comp rx=0 tx=0).stop 2024-09-21T05:41:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 msgr2=0x7f1814151a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:36.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 0x7f1814151a60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f1804034040 tx=0x7f1804003910 comp rx=0 tx=0).stop 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 shutdown_connections 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1800078bc0 0x7f180007b0d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18140d0ce0 0x7f1814152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f18140cf4c0 0x7f18140cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 --2- 172.21.15.22:0/2044860819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f18140cf0b0 0x7f1814151a60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 >> 172.21.15.22:0/2044860819 conn(0x7f181408bfc0 msgr2=0x7f181406f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 shutdown_connections 2024-09-21T05:41:36.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.652+0000 7f181967e640 1 -- 172.21.15.22:0/2044860819 wait complete. 2024-09-21T05:41:36.668 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:41:36.668 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:36.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 -- 172.21.15.22:0/367382119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b9970 msgr2=0x7f75d00b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 --2- 172.21.15.22:0/367382119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b9970 0x7f75d00b3710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009850 tx=0x7f75c002a530 comp rx=0 tx=0).stop 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 -- 172.21.15.22:0/367382119 shutdown_connections 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 --2- 172.21.15.22:0/367382119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75d006e0e0 0x7f75d006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 --2- 172.21.15.22:0/367382119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b9970 0x7f75d00b3710 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 -- 172.21.15.22:0/367382119 >> 172.21.15.22:0/367382119 conn(0x7f75d009d7f0 msgr2=0x7f75d00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 -- 172.21.15.22:0/367382119 shutdown_connections 2024-09-21T05:41:36.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.790+0000 7f75d66f2640 1 -- 172.21.15.22:0/367382119 wait complete. 2024-09-21T05:41:36.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.792+0000 7f75d66f2640 1 Processor -- start 2024-09-21T05:41:36.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.792+0000 7f75d66f2640 1 -- start start 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.793+0000 7f75d66f2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d006e0e0 0x7f75d01f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.793+0000 7f75d66f2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 0x7f75d0161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.793+0000 7f75d66f2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75d00b9970 0x7f75d00c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.793+0000 7f75d66f2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75d006ea30 con 0x7f75d00b5fa0 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.793+0000 7f75d66f2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75d00d5010 con 0x7f75d006e0e0 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.793+0000 7f75d66f2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f75d006ebb0 con 0x7f75d00b9970 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 0x7f75d0161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 0x7f75d0161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45384/0 (socket says 172.21.15.22:45384) 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 -- 172.21.15.22:0/1230121273 learned_addr learned my addr 172.21.15.22:0/1230121273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cffff640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d006e0e0 0x7f75d01f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:36.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75d4c8f640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75d00b9970 0x7f75d00c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 -- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75d00b9970 msgr2=0x7f75d00c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75d00b9970 0x7f75d00c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 -- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d006e0e0 msgr2=0x7f75d01f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d006e0e0 0x7f75d01f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 -- 172.21.15.22:0/1230121273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d0160ec0 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75cf7fe640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 0x7f75d0161f60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c3c0 tx=0x7f75b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8017070 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75b80035f0 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b8003a40 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.794+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d01611a0 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.795+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d0161760 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.795+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d00b3710 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.796+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f75b801a070 con 0x7f75d00b5fa0 2024-09-21T05:41:36.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.797+0000 7f75ccff9640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75b0078b90 0x7f75b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.797+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f75b809a0b0 con 0x7f75d00b5fa0 2024-09-21T05:41:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.797+0000 7f75cffff640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75b0078b90 0x7f75b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:36.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.797+0000 7f75cffff640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75b0078b90 0x7f75b007b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75c0034000 tx=0x7f75c0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:36.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:36.801+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f75b80624f0 con 0x7f75d00b5fa0 2024-09-21T05:41:37.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.023+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75d015f7d0 con 0x7f75d00b5fa0 2024-09-21T05:41:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.024+0000 7f75ccff9640 1 -- 172.21.15.22:0/1230121273 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5226 (secure 0 0 0) 0x7f75b8061df0 con 0x7f75d00b5fa0 2024-09-21T05:41:37.026 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:37.026 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"btime":"2024-09-21T05:41:36:644246+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:41:36.026286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:41:37.026 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 34 2024-09-21T05:41:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75b0078b90 msgr2=0x7f75b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75b0078b90 0x7f75b007b0a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f75c0034000 tx=0x7f75c0004730 comp rx=0 tx=0).stop 2024-09-21T05:41:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 msgr2=0x7f75d0161f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 0x7f75d0161f60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f75b800c3c0 tx=0x7f75b800c880 comp rx=0 tx=0).stop 2024-09-21T05:41:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 shutdown_connections 2024-09-21T05:41:37.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75b0078b90 0x7f75b007b0a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75d00b9970 0x7f75d00c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75d00b5fa0 0x7f75d0161f60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 --2- 172.21.15.22:0/1230121273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75d006e0e0 0x7f75d01f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 >> 172.21.15.22:0/1230121273 conn(0x7f75d009d7f0 msgr2=0x7f75d015e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 shutdown_connections 2024-09-21T05:41:37.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.026+0000 7f75d66f2640 1 -- 172.21.15.22:0/1230121273 wait complete. 2024-09-21T05:41:37.039 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:41:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 -- 172.21.15.22:0/2909196938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640ccba0 msgr2=0x7f53640ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 --2- 172.21.15.22:0/2909196938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640ccba0 0x7f53640ccfb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f535c005fd0 tx=0x7f535c02aa00 comp rx=0 tx=0).stop 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 -- 172.21.15.22:0/2909196938 shutdown_connections 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 --2- 172.21.15.22:0/2909196938 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640ccba0 0x7f53640ccfb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 --2- 172.21.15.22:0/2909196938 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53640d1d40 0x7f53640b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 -- 172.21.15.22:0/2909196938 >> 172.21.15.22:0/2909196938 conn(0x7f53640b3f10 msgr2=0x7f53640b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 -- 172.21.15.22:0/2909196938 shutdown_connections 2024-09-21T05:41:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.153+0000 7f5369039640 1 -- 172.21.15.22:0/2909196938 wait complete. 2024-09-21T05:41:37.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.155+0000 7f5369039640 1 Processor -- start 2024-09-21T05:41:37.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.155+0000 7f5369039640 1 -- start start 2024-09-21T05:41:37.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5369039640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53640ccba0 0x7f53641f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5369039640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 0x7f5364148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5369039640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536415c6f0 0x7f5364148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5369039640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53640d12e0 con 0x7f53640d1d40 2024-09-21T05:41:37.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5369039640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53640d1160 con 0x7f536415c6f0 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5369039640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53640d1460 con 0x7f53640ccba0 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 0x7f5364148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 0x7f5364148660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45394/0 (socket says 172.21.15.22:45394) 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 -- 172.21.15.22:0/236740497 learned_addr learned my addr 172.21.15.22:0/236740497 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f536359e640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536415c6f0 0x7f5364148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f5362d9d640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53640ccba0 0x7f53641f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 -- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53640ccba0 msgr2=0x7f53641f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53640ccba0 0x7f53641f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 -- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536415c6f0 msgr2=0x7f5364148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536415c6f0 0x7f5364148ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.157+0000 7f535a59c640 1 -- 172.21.15.22:0/236740497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53641e51f0 con 0x7f53640d1d40 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.158+0000 7f535a59c640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 0x7f5364148660 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fd0 tx=0x7f534c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.158+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c015020 con 0x7f53640d1d40 2024-09-21T05:41:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.158+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f534c011920 con 0x7f53640d1d40 2024-09-21T05:41:37.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.158+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536415ee20 con 0x7f53640d1d40 2024-09-21T05:41:37.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.158+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c011d50 con 0x7f53640d1d40 2024-09-21T05:41:37.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.158+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53641469a0 con 0x7f53640d1d40 2024-09-21T05:41:37.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.159+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53640d19c0 con 0x7f53640d1d40 2024-09-21T05:41:37.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.160+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f534c018020 con 0x7f53640d1d40 2024-09-21T05:41:37.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.160+0000 7f535bfff640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5344078bc0 0x7f534407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.161+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f534c09aab0 con 0x7f53640d1d40 2024-09-21T05:41:37.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.161+0000 7f5362d9d640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5344078bc0 0x7f534407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.161+0000 7f5362d9d640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5344078bc0 0x7f534407b0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5354005fd0 tx=0x7f535400a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.165+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f534c01f020 con 0x7f53640d1d40 2024-09-21T05:41:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.354+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53640b6020 con 0x7f53640d1d40 2024-09-21T05:41:37.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.355+0000 7f535bfff640 1 -- 172.21.15.22:0/236740497 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16713 (secure 0 0 0) 0x7f534c062930 con 0x7f53640d1d40 2024-09-21T05:41:37.358 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:37.358 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":57,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:34.632274+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:41:30.686369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T05:41:31.966340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:41:37.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.357+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5344078bc0 msgr2=0x7f534407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.357+0000 7f5369039640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5344078bc0 0x7f534407b0d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5354005fd0 tx=0x7f535400a250 comp rx=0 tx=0).stop 2024-09-21T05:41:37.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.357+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 msgr2=0x7f5364148660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.357+0000 7f5369039640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 0x7f5364148660 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f534c009fd0 tx=0x7f534c00b870 comp rx=0 tx=0).stop 2024-09-21T05:41:37.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 shutdown_connections 2024-09-21T05:41:37.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f536415c6f0 0x7f5364148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53640d1d40 0x7f5364148660 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5344078bc0 0x7f534407b0d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 --2- 172.21.15.22:0/236740497 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53640ccba0 0x7f53641f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 >> 172.21.15.22:0/236740497 conn(0x7f53640b3f10 msgr2=0x7f53640b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 shutdown_connections 2024-09-21T05:41:37.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.358+0000 7f5369039640 1 -- 172.21.15.22:0/236740497 wait complete. 2024-09-21T05:41:37.371 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:41:37.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:41:37.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 -- 172.21.15.22:0/660855005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc06e0a0 msgr2=0x7f83cc0b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 --2- 172.21.15.22:0/660855005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b97e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0098e0 tx=0x7f83bc02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 -- 172.21.15.22:0/660855005 shutdown_connections 2024-09-21T05:41:37.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 --2- 172.21.15.22:0/660855005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 0x7f83cc06e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 --2- 172.21.15.22:0/660855005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b97e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 -- 172.21.15.22:0/660855005 >> 172.21.15.22:0/660855005 conn(0x7f83cc09d840 msgr2=0x7f83cc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 -- 172.21.15.22:0/660855005 shutdown_connections 2024-09-21T05:41:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.486+0000 7f83d1739640 1 -- 172.21.15.22:0/660855005 wait complete. 2024-09-21T05:41:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.487+0000 7f83d1739640 1 Processor -- start 2024-09-21T05:41:37.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.488+0000 7f83d1739640 1 -- start start 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83d1739640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83d1739640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 0x7f83cc0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83d1739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 0x7f83cc1f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83d1739640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc0cfcd0 con 0x7f83cc0d0e50 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83d1739640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f83cc0cfb50 con 0x7f83cc06e0a0 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83d1739640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f83cc0cfe50 con 0x7f83cc06f6e0 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 0x7f83cc1f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 0x7f83cc1f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45416/0 (socket says 172.21.15.22:45416) 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 -- 172.21.15.22:0/379626573 learned_addr learned my addr 172.21.15.22:0/379626573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83c3fff640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 0x7f83cc0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83caffd640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 -- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 msgr2=0x7f83cc0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 0x7f83cc0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 -- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc06e0a0 msgr2=0x7f83cc0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 -- 172.21.15.22:0/379626573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83cc0b6270 con 0x7f83cc0d0e50 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83caffd640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83c3fff640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 0x7f83cc0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.489+0000 7f83cb7fe640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 0x7f83cc1f9010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c730 tx=0x7f83c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.490+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c4017070 con 0x7f83cc0d0e50 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.490+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83c4003c60 con 0x7f83cc0d0e50 2024-09-21T05:41:37.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.490+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83cc0b6530 con 0x7f83cc0d0e50 2024-09-21T05:41:37.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.490+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c40052d0 con 0x7f83cc0d0e50 2024-09-21T05:41:37.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.490+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83cc0b6af0 con 0x7f83cc0d0e50 2024-09-21T05:41:37.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.491+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83cc0d01e0 con 0x7f83cc0d0e50 2024-09-21T05:41:37.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.491+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83c401a070 con 0x7f83cc0d0e50 2024-09-21T05:41:37.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.492+0000 7f83c37fe640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f83ac078b90 0x7f83ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.492+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f83c409a530 con 0x7f83cc0d0e50 2024-09-21T05:41:37.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.492+0000 7f83caffd640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f83ac078b90 0x7f83ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.493+0000 7f83caffd640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f83ac078b90 0x7f83ac07b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f83bc034000 tx=0x7f83bc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:37.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.496+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83c4062970 con 0x7f83cc0d0e50 2024-09-21T05:41:37.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.680+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83cc0b76e0 con 0x7f83ac078b90 2024-09-21T05:41:37.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.685+0000 7f83c37fe640 1 -- 172.21.15.22:0/379626573 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248123 (secure 0 0 0) 0x7f83cc0b76e0 con 0x7f83ac078b90 2024-09-21T05:41:37.690 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:37.691 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:41:37.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.688+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f83ac078b90 msgr2=0x7f83ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.688+0000 7f83d1739640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f83ac078b90 0x7f83ac07b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f83bc034000 tx=0x7f83bc004730 comp rx=0 tx=0).stop 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.688+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 msgr2=0x7f83cc1f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.688+0000 7f83d1739640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 0x7f83cc1f9010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c730 tx=0x7f83c400cbf0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 shutdown_connections 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f83cc0d0e50 0x7f83cc1f9010 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f83ac078b90 0x7f83ac07b0a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f83cc06f6e0 0x7f83cc0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 --2- 172.21.15.22:0/379626573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f83cc06e0a0 0x7f83cc0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 >> 172.21.15.22:0/379626573 conn(0x7f83cc09d840 msgr2=0x7f83cc15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 shutdown_connections 2024-09-21T05:41:37.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.689+0000 7f83d1739640 1 -- 172.21.15.22:0/379626573 wait complete. 2024-09-21T05:41:37.702 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-21T05:41:35.770037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591076,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":123,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":255948,"kb_used_data":6388,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718772,"statfs":{"total":773094113280,"available":772832022528,"internally_reserved":0,"allocated":6541312,"data_stored":3378090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.161677"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697571+0000","last_change":"2024-09-21T05:41:33.648732+0000","last_active":"2024-09-21T05:41:34.697571+0000","last_peered":"2024-09-21T05:41:34.697571+0000","last_clean":"2024-09-21T05:41:34.697571+0000","last_became_active":"2024-09-21T05:41:33.648430+0000","last_became_peered":"2024-09-21T05:41:33.648430+0000","last_unstale":"2024-09-21T05:41:34.697571+0000","last_undegraded":"2024-09-21T05:41:34.697571+0000","last_fullsized":"2024-09-21T05:41:34.697571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.569594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699401+0000","last_change":"2024-09-21T05:41:33.639552+0000","last_active":"2024-09-21T05:41:34.699401+0000","last_peered":"2024-09-21T05:41:34.699401+0000","last_clean":"2024-09-21T05:41:34.699401+0000","last_became_active":"2024-09-21T05:41:33.639162+0000","last_became_peered":"2024-09-21T05:41:33.639162+0000","last_unstale":"2024-09-21T05:41:34.699401+0000","last_undegraded":"2024-09-21T05:41:34.699401+0000","last_fullsized":"2024-09-21T05:41:34.699401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.369174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698111+0000","last_change":"2024-09-21T05:41:33.635230+0000","last_active":"2024-09-21T05:41:34.698111+0000","last_peered":"2024-09-21T05:41:34.698111+0000","last_clean":"2024-09-21T05:41:34.698111+0000","last_became_active":"2024-09-21T05:41:33.634821+0000","last_became_peered":"2024-09-21T05:41:33.634821+0000","last_unstale":"2024-09-21T05:41:34.698111+0000","last_undegraded":"2024-09-21T05:41:34.698111+0000","last_fullsized":"2024-09-21T05:41:34.698111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:44.928717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697251+0000","last_change":"2024-09-21T05:41:33.641105+0000","last_active":"2024-09-21T05:41:34.697251+0000","last_peered":"2024-09-21T05:41:34.697251+0000","last_clean":"2024-09-21T05:41:34.697251+0000","last_became_active":"2024-09-21T05:41:33.640769+0000","last_became_peered":"2024-09-21T05:41:33.640769+0000","last_unstale":"2024-09-21T05:41:34.697251+0000","last_undegraded":"2024-09-21T05:41:34.697251+0000","last_fullsized":"2024-09-21T05:41:34.697251+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:52.338089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697422+0000","last_change":"2024-09-21T05:41:33.643002+0000","last_active":"2024-09-21T05:41:34.697422+0000","last_peered":"2024-09-21T05:41:34.697422+0000","last_clean":"2024-09-21T05:41:34.697422+0000","last_became_active":"2024-09-21T05:41:33.642457+0000","last_became_peered":"2024-09-21T05:41:33.642457+0000","last_unstale":"2024-09-21T05:41:34.697422+0000","last_undegraded":"2024-09-21T05:41:34.697422+0000","last_fullsized":"2024-09-21T05:41:34.697422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:10.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697914+0000","last_change":"2024-09-21T05:41:33.636407+0000","last_active":"2024-09-21T05:41:34.697914+0000","last_peered":"2024-09-21T05:41:34.697914+0000","last_clean":"2024-09-21T05:41:34.697914+0000","last_became_active":"2024-09-21T05:41:33.636081+0000","last_became_peered":"2024-09-21T05:41:33.636081+0000","last_unstale":"2024-09-21T05:41:34.697914+0000","last_undegraded":"2024-09-21T05:41:34.697914+0000","last_fullsized":"2024-09-21T05:41:34.697914+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:28.637017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639819+0000","last_change":"2024-09-21T05:41:33.632622+0000","last_active":"2024-09-21T05:41:34.639819+0000","last_peered":"2024-09-21T05:41:34.639819+0000","last_clean":"2024-09-21T05:41:34.639819+0000","last_became_active":"2024-09-21T05:41:33.632328+0000","last_became_peered":"2024-09-21T05:41:33.632328+0000","last_unstale":"2024-09-21T05:41:34.639819+0000","last_undegraded":"2024-09-21T05:41:34.639819+0000","last_fullsized":"2024-09-21T05:41:34.639819+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:15.022568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639699+0000","last_change":"2024-09-21T05:41:33.644830+0000","last_active":"2024-09-21T05:41:34.639699+0000","last_peered":"2024-09-21T05:41:34.639699+0000","last_clean":"2024-09-21T05:41:34.639699+0000","last_became_active":"2024-09-21T05:41:33.644225+0000","last_became_peered":"2024-09-21T05:41:33.644225+0000","last_unstale":"2024-09-21T05:41:34.639699+0000","last_undegraded":"2024-09-21T05:41:34.639699+0000","last_fullsized":"2024-09-21T05:41:34.639699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:19.195318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698394+0000","last_change":"2024-09-21T05:41:33.638131+0000","last_active":"2024-09-21T05:41:34.698394+0000","last_peered":"2024-09-21T05:41:34.698394+0000","last_clean":"2024-09-21T05:41:34.698394+0000","last_became_active":"2024-09-21T05:41:33.637594+0000","last_became_peered":"2024-09-21T05:41:33.637594+0000","last_unstale":"2024-09-21T05:41:34.698394+0000","last_undegraded":"2024-09-21T05:41:34.698394+0000","last_fullsized":"2024-09-21T05:41:34.698394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:57.994501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640491+0000","last_change":"2024-09-21T05:41:33.648300+0000","last_active":"2024-09-21T05:41:34.640491+0000","last_peered":"2024-09-21T05:41:34.640491+0000","last_clean":"2024-09-21T05:41:34.640491+0000","last_became_active":"2024-09-21T05:41:33.647962+0000","last_became_peered":"2024-09-21T05:41:33.647962+0000","last_unstale":"2024-09-21T05:41:34.640491+0000","last_undegraded":"2024-09-21T05:41:34.640491+0000","last_fullsized":"2024-09-21T05:41:34.640491+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:19.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697880+0000","last_change":"2024-09-21T05:41:33.642382+0000","last_active":"2024-09-21T05:41:34.697880+0000","last_peered":"2024-09-21T05:41:34.697880+0000","last_clean":"2024-09-21T05:41:34.697880+0000","last_became_active":"2024-09-21T05:41:33.642033+0000","last_became_peered":"2024-09-21T05:41:33.642033+0000","last_unstale":"2024-09-21T05:41:34.697880+0000","last_undegraded":"2024-09-21T05:41:34.697880+0000","last_fullsized":"2024-09-21T05:41:34.697880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:56.658021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698133+0000","last_change":"2024-09-21T05:41:33.641030+0000","last_active":"2024-09-21T05:41:34.698133+0000","last_peered":"2024-09-21T05:41:34.698133+0000","last_clean":"2024-09-21T05:41:34.698133+0000","last_became_active":"2024-09-21T05:41:33.640701+0000","last_became_peered":"2024-09-21T05:41:33.640701+0000","last_unstale":"2024-09-21T05:41:34.698133+0000","last_undegraded":"2024-09-21T05:41:34.698133+0000","last_fullsized":"2024-09-21T05:41:34.698133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:53.840007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697313+0000","last_change":"2024-09-21T05:41:33.643493+0000","last_active":"2024-09-21T05:41:34.697313+0000","last_peered":"2024-09-21T05:41:34.697313+0000","last_clean":"2024-09-21T05:41:34.697313+0000","last_became_active":"2024-09-21T05:41:33.643183+0000","last_became_peered":"2024-09-21T05:41:33.643183+0000","last_unstale":"2024-09-21T05:41:34.697313+0000","last_undegraded":"2024-09-21T05:41:34.697313+0000","last_fullsized":"2024-09-21T05:41:34.697313+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:35.522759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698761+0000","last_change":"2024-09-21T05:41:33.640103+0000","last_active":"2024-09-21T05:41:34.698761+0000","last_peered":"2024-09-21T05:41:34.698761+0000","last_clean":"2024-09-21T05:41:34.698761+0000","last_became_active":"2024-09-21T05:41:33.636395+0000","last_became_peered":"2024-09-21T05:41:33.636395+0000","last_unstale":"2024-09-21T05:41:34.698761+0000","last_undegraded":"2024-09-21T05:41:34.698761+0000","last_fullsized":"2024-09-21T05:41:34.698761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:28.536973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698034+0000","last_change":"2024-09-21T05:41:33.645860+0000","last_active":"2024-09-21T05:41:34.698034+0000","last_peered":"2024-09-21T05:41:34.698034+0000","last_clean":"2024-09-21T05:41:34.698034+0000","last_became_active":"2024-09-21T05:41:33.645551+0000","last_became_peered":"2024-09-21T05:41:33.645551+0000","last_unstale":"2024-09-21T05:41:34.698034+0000","last_undegraded":"2024-09-21T05:41:34.698034+0000","last_fullsized":"2024-09-21T05:41:34.698034+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:24.769718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640905+0000","last_change":"2024-09-21T05:41:33.648558+0000","last_active":"2024-09-21T05:41:34.640905+0000","last_peered":"2024-09-21T05:41:34.640905+0000","last_clean":"2024-09-21T05:41:34.640905+0000","last_became_active":"2024-09-21T05:41:33.648206+0000","last_became_peered":"2024-09-21T05:41:33.648206+0000","last_unstale":"2024-09-21T05:41:34.640905+0000","last_undegraded":"2024-09-21T05:41:34.640905+0000","last_fullsized":"2024-09-21T05:41:34.640905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.444641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697785+0000","last_change":"2024-09-21T05:41:33.645467+0000","last_active":"2024-09-21T05:41:34.697785+0000","last_peered":"2024-09-21T05:41:34.697785+0000","last_clean":"2024-09-21T05:41:34.697785+0000","last_became_active":"2024-09-21T05:41:33.645113+0000","last_became_peered":"2024-09-21T05:41:33.645113+0000","last_unstale":"2024-09-21T05:41:34.697785+0000","last_undegraded":"2024-09-21T05:41:34.697785+0000","last_fullsized":"2024-09-21T05:41:34.697785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:46.221950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698655+0000","last_change":"2024-09-21T05:41:33.634591+0000","last_active":"2024-09-21T05:41:34.698655+0000","last_peered":"2024-09-21T05:41:34.698655+0000","last_clean":"2024-09-21T05:41:34.698655+0000","last_became_active":"2024-09-21T05:41:33.634265+0000","last_became_peered":"2024-09-21T05:41:33.634265+0000","last_unstale":"2024-09-21T05:41:34.698655+0000","last_undegraded":"2024-09-21T05:41:34.698655+0000","last_fullsized":"2024-09-21T05:41:34.698655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:33.756146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697951+0000","last_change":"2024-09-21T05:41:33.632933+0000","last_active":"2024-09-21T05:41:34.697951+0000","last_peered":"2024-09-21T05:41:34.697951+0000","last_clean":"2024-09-21T05:41:34.697951+0000","last_became_active":"2024-09-21T05:41:33.632625+0000","last_became_peered":"2024-09-21T05:41:33.632625+0000","last_unstale":"2024-09-21T05:41:34.697951+0000","last_undegraded":"2024-09-21T05:41:34.697951+0000","last_fullsized":"2024-09-21T05:41:34.697951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.047303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697677+0000","last_change":"2024-09-21T05:41:33.641926+0000","last_active":"2024-09-21T05:41:34.697677+0000","last_peered":"2024-09-21T05:41:34.697677+0000","last_clean":"2024-09-21T05:41:34.697677+0000","last_became_active":"2024-09-21T05:41:33.641620+0000","last_became_peered":"2024-09-21T05:41:33.641620+0000","last_unstale":"2024-09-21T05:41:34.697677+0000","last_undegraded":"2024-09-21T05:41:34.697677+0000","last_fullsized":"2024-09-21T05:41:34.697677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:43.487445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699386+0000","last_change":"2024-09-21T05:41:33.639651+0000","last_active":"2024-09-21T05:41:34.699386+0000","last_peered":"2024-09-21T05:41:34.699386+0000","last_clean":"2024-09-21T05:41:34.699386+0000","last_became_active":"2024-09-21T05:41:33.639293+0000","last_became_peered":"2024-09-21T05:41:33.639293+0000","last_unstale":"2024-09-21T05:41:34.699386+0000","last_undegraded":"2024-09-21T05:41:34.699386+0000","last_fullsized":"2024-09-21T05:41:34.699386+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:51.923976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697155+0000","last_change":"2024-09-21T05:41:33.645164+0000","last_active":"2024-09-21T05:41:34.697155+0000","last_peered":"2024-09-21T05:41:34.697155+0000","last_clean":"2024-09-21T05:41:34.697155+0000","last_became_active":"2024-09-21T05:41:33.644826+0000","last_became_peered":"2024-09-21T05:41:33.644826+0000","last_unstale":"2024-09-21T05:41:34.697155+0000","last_undegraded":"2024-09-21T05:41:34.697155+0000","last_fullsized":"2024-09-21T05:41:34.697155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:30.888264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698102+0000","last_change":"2024-09-21T05:41:33.652970+0000","last_active":"2024-09-21T05:41:34.698102+0000","last_peered":"2024-09-21T05:41:34.698102+0000","last_clean":"2024-09-21T05:41:34.698102+0000","last_became_active":"2024-09-21T05:41:33.652536+0000","last_became_peered":"2024-09-21T05:41:33.652536+0000","last_unstale":"2024-09-21T05:41:34.698102+0000","last_undegraded":"2024-09-21T05:41:34.698102+0000","last_fullsized":"2024-09-21T05:41:34.698102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:43.986414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698406+0000","last_change":"2024-09-21T05:41:33.643684+0000","last_active":"2024-09-21T05:41:34.698406+0000","last_peered":"2024-09-21T05:41:34.698406+0000","last_clean":"2024-09-21T05:41:34.698406+0000","last_became_active":"2024-09-21T05:41:33.643363+0000","last_became_peered":"2024-09-21T05:41:33.643363+0000","last_unstale":"2024-09-21T05:41:34.698406+0000","last_undegraded":"2024-09-21T05:41:34.698406+0000","last_fullsized":"2024-09-21T05:41:34.698406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:33.385517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697753+0000","last_change":"2024-09-21T05:41:32.638348+0000","last_active":"2024-09-21T05:41:34.697753+0000","last_peered":"2024-09-21T05:41:34.697753+0000","last_clean":"2024-09-21T05:41:34.697753+0000","last_became_active":"2024-09-21T05:41:32.638033+0000","last_became_peered":"2024-09-21T05:41:32.638033+0000","last_unstale":"2024-09-21T05:41:34.697753+0000","last_undegraded":"2024-09-21T05:41:34.697753+0000","last_fullsized":"2024-09-21T05:41:34.697753+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:40.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639883+0000","last_change":"2024-09-21T05:41:33.634060+0000","last_active":"2024-09-21T05:41:34.639883+0000","last_peered":"2024-09-21T05:41:34.639883+0000","last_clean":"2024-09-21T05:41:34.639883+0000","last_became_active":"2024-09-21T05:41:33.633362+0000","last_became_peered":"2024-09-21T05:41:33.633362+0000","last_unstale":"2024-09-21T05:41:34.639883+0000","last_undegraded":"2024-09-21T05:41:34.639883+0000","last_fullsized":"2024-09-21T05:41:34.639883+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.763484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699332+0000","last_change":"2024-09-21T05:41:32.638551+0000","last_active":"2024-09-21T05:41:34.699332+0000","last_peered":"2024-09-21T05:41:34.699332+0000","last_clean":"2024-09-21T05:41:34.699332+0000","last_became_active":"2024-09-21T05:41:32.638262+0000","last_became_peered":"2024-09-21T05:41:32.638262+0000","last_unstale":"2024-09-21T05:41:34.699332+0000","last_undegraded":"2024-09-21T05:41:34.699332+0000","last_fullsized":"2024-09-21T05:41:34.699332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:53.009886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697558+0000","last_change":"2024-09-21T05:41:33.634059+0000","last_active":"2024-09-21T05:41:34.697558+0000","last_peered":"2024-09-21T05:41:34.697558+0000","last_clean":"2024-09-21T05:41:34.697558+0000","last_became_active":"2024-09-21T05:41:33.633746+0000","last_became_peered":"2024-09-21T05:41:33.633746+0000","last_unstale":"2024-09-21T05:41:34.697558+0000","last_undegraded":"2024-09-21T05:41:34.697558+0000","last_fullsized":"2024-09-21T05:41:34.697558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:21.628475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698529+0000","last_change":"2024-09-21T05:41:32.633871+0000","last_active":"2024-09-21T05:41:34.698529+0000","last_peered":"2024-09-21T05:41:34.698529+0000","last_clean":"2024-09-21T05:41:34.698529+0000","last_became_active":"2024-09-21T05:41:32.633531+0000","last_became_peered":"2024-09-21T05:41:32.633531+0000","last_unstale":"2024-09-21T05:41:34.698529+0000","last_undegraded":"2024-09-21T05:41:34.698529+0000","last_fullsized":"2024-09-21T05:41:34.698529+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:55.239582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698180+0000","last_change":"2024-09-21T05:41:33.631513+0000","last_active":"2024-09-21T05:41:34.698180+0000","last_peered":"2024-09-21T05:41:34.698180+0000","last_clean":"2024-09-21T05:41:34.698180+0000","last_became_active":"2024-09-21T05:41:33.631198+0000","last_became_peered":"2024-09-21T05:41:33.631198+0000","last_unstale":"2024-09-21T05:41:34.698180+0000","last_undegraded":"2024-09-21T05:41:34.698180+0000","last_fullsized":"2024-09-21T05:41:34.698180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.047470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698543+0000","last_change":"2024-09-21T05:41:32.632245+0000","last_active":"2024-09-21T05:41:34.698543+0000","last_peered":"2024-09-21T05:41:34.698543+0000","last_clean":"2024-09-21T05:41:34.698543+0000","last_became_active":"2024-09-21T05:41:32.631526+0000","last_became_peered":"2024-09-21T05:41:32.631526+0000","last_unstale":"2024-09-21T05:41:34.698543+0000","last_undegraded":"2024-09-21T05:41:34.698543+0000","last_fullsized":"2024-09-21T05:41:34.698543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:39.894103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698256+0000","last_change":"2024-09-21T05:41:33.645710+0000","last_active":"2024-09-21T05:41:34.698256+0000","last_peered":"2024-09-21T05:41:34.698256+0000","last_clean":"2024-09-21T05:41:34.698256+0000","last_became_active":"2024-09-21T05:41:33.645134+0000","last_became_peered":"2024-09-21T05:41:33.645134+0000","last_unstale":"2024-09-21T05:41:34.698256+0000","last_undegraded":"2024-09-21T05:41:34.698256+0000","last_fullsized":"2024-09-21T05:41:34.698256+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:08.589704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699301+0000","last_change":"2024-09-21T05:41:33.638399+0000","last_active":"2024-09-21T05:41:34.699301+0000","last_peered":"2024-09-21T05:41:34.699301+0000","last_clean":"2024-09-21T05:41:34.699301+0000","last_became_active":"2024-09-21T05:41:33.638058+0000","last_became_peered":"2024-09-21T05:41:33.638058+0000","last_unstale":"2024-09-21T05:41:34.699301+0000","last_undegraded":"2024-09-21T05:41:34.699301+0000","last_fullsized":"2024-09-21T05:41:34.699301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:49.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697553+0000","last_change":"2024-09-21T05:41:32.637365+0000","last_active":"2024-09-21T05:41:34.697553+0000","last_peered":"2024-09-21T05:41:34.697553+0000","last_clean":"2024-09-21T05:41:34.697553+0000","last_became_active":"2024-09-21T05:41:32.637034+0000","last_became_peered":"2024-09-21T05:41:32.637034+0000","last_unstale":"2024-09-21T05:41:34.697553+0000","last_undegraded":"2024-09-21T05:41:34.697553+0000","last_fullsized":"2024-09-21T05:41:34.697553+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:24.572516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640901+0000","last_change":"2024-09-21T05:41:33.631512+0000","last_active":"2024-09-21T05:41:34.640901+0000","last_peered":"2024-09-21T05:41:34.640901+0000","last_clean":"2024-09-21T05:41:34.640901+0000","last_became_active":"2024-09-21T05:41:33.631171+0000","last_became_peered":"2024-09-21T05:41:33.631171+0000","last_unstale":"2024-09-21T05:41:34.640901+0000","last_undegraded":"2024-09-21T05:41:34.640901+0000","last_fullsized":"2024-09-21T05:41:34.640901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:13.447138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698119+0000","last_change":"2024-09-21T05:41:32.635145+0000","last_active":"2024-09-21T05:41:34.698119+0000","last_peered":"2024-09-21T05:41:34.698119+0000","last_clean":"2024-09-21T05:41:34.698119+0000","last_became_active":"2024-09-21T05:41:32.634786+0000","last_became_peered":"2024-09-21T05:41:32.634786+0000","last_unstale":"2024-09-21T05:41:34.698119+0000","last_undegraded":"2024-09-21T05:41:34.698119+0000","last_fullsized":"2024-09-21T05:41:34.698119+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:35.749308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699104+0000","last_change":"2024-09-21T05:41:33.638892+0000","last_active":"2024-09-21T05:41:34.699104+0000","last_peered":"2024-09-21T05:41:34.699104+0000","last_clean":"2024-09-21T05:41:34.699104+0000","last_became_active":"2024-09-21T05:41:33.638106+0000","last_became_peered":"2024-09-21T05:41:33.638106+0000","last_unstale":"2024-09-21T05:41:34.699104+0000","last_undegraded":"2024-09-21T05:41:34.699104+0000","last_fullsized":"2024-09-21T05:41:34.699104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:23.528753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699431+0000","last_change":"2024-09-21T05:41:32.635026+0000","last_active":"2024-09-21T05:41:34.699431+0000","last_peered":"2024-09-21T05:41:34.699431+0000","last_clean":"2024-09-21T05:41:34.699431+0000","last_became_active":"2024-09-21T05:41:32.634713+0000","last_became_peered":"2024-09-21T05:41:32.634713+0000","last_unstale":"2024-09-21T05:41:34.699431+0000","last_undegraded":"2024-09-21T05:41:34.699431+0000","last_fullsized":"2024-09-21T05:41:34.699431+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:09.076416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639394+0000","last_change":"2024-09-21T05:41:33.636619+0000","last_active":"2024-09-21T05:41:34.639394+0000","last_peered":"2024-09-21T05:41:34.639394+0000","last_clean":"2024-09-21T05:41:34.639394+0000","last_became_active":"2024-09-21T05:41:33.636303+0000","last_became_peered":"2024-09-21T05:41:33.636303+0000","last_unstale":"2024-09-21T05:41:34.639394+0000","last_undegraded":"2024-09-21T05:41:34.639394+0000","last_fullsized":"2024-09-21T05:41:34.639394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:48.982626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697772+0000","last_change":"2024-09-21T05:41:33.635658+0000","last_active":"2024-09-21T05:41:34.697772+0000","last_peered":"2024-09-21T05:41:34.697772+0000","last_clean":"2024-09-21T05:41:34.697772+0000","last_became_active":"2024-09-21T05:41:33.635345+0000","last_became_peered":"2024-09-21T05:41:33.635345+0000","last_unstale":"2024-09-21T05:41:34.697772+0000","last_undegraded":"2024-09-21T05:41:34.697772+0000","last_fullsized":"2024-09-21T05:41:34.697772+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:28.402298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699252+0000","last_change":"2024-09-21T05:41:32.639894+0000","last_active":"2024-09-21T05:41:34.699252+0000","last_peered":"2024-09-21T05:41:34.699252+0000","last_clean":"2024-09-21T05:41:34.699252+0000","last_became_active":"2024-09-21T05:41:32.637282+0000","last_became_peered":"2024-09-21T05:41:32.637282+0000","last_unstale":"2024-09-21T05:41:34.699252+0000","last_undegraded":"2024-09-21T05:41:34.699252+0000","last_fullsized":"2024-09-21T05:41:34.699252+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:55.655727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640142+0000","last_change":"2024-09-21T05:41:32.631769+0000","last_active":"2024-09-21T05:41:34.640142+0000","last_peered":"2024-09-21T05:41:34.640142+0000","last_clean":"2024-09-21T05:41:34.640142+0000","last_became_active":"2024-09-21T05:41:32.631219+0000","last_became_peered":"2024-09-21T05:41:32.631219+0000","last_unstale":"2024-09-21T05:41:34.640142+0000","last_undegraded":"2024-09-21T05:41:34.640142+0000","last_fullsized":"2024-09-21T05:41:34.640142+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:58.341553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698751+0000","last_change":"2024-09-21T05:41:32.632401+0000","last_active":"2024-09-21T05:41:34.698751+0000","last_peered":"2024-09-21T05:41:34.698751+0000","last_clean":"2024-09-21T05:41:34.698751+0000","last_became_active":"2024-09-21T05:41:32.631945+0000","last_became_peered":"2024-09-21T05:41:32.631945+0000","last_unstale":"2024-09-21T05:41:34.698751+0000","last_undegraded":"2024-09-21T05:41:34.698751+0000","last_fullsized":"2024-09-21T05:41:34.698751+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:43.733261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698442+0000","last_change":"2024-09-21T05:41:32.633543+0000","last_active":"2024-09-21T05:41:34.698442+0000","last_peered":"2024-09-21T05:41:34.698442+0000","last_clean":"2024-09-21T05:41:34.698442+0000","last_became_active":"2024-09-21T05:41:32.633229+0000","last_became_peered":"2024-09-21T05:41:32.633229+0000","last_unstale":"2024-09-21T05:41:34.698442+0000","last_undegraded":"2024-09-21T05:41:34.698442+0000","last_fullsized":"2024-09-21T05:41:34.698442+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:04.045009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698414+0000","last_change":"2024-09-21T05:41:33.635421+0000","last_active":"2024-09-21T05:41:34.698414+0000","last_peered":"2024-09-21T05:41:34.698414+0000","last_clean":"2024-09-21T05:41:34.698414+0000","last_became_active":"2024-09-21T05:41:33.635100+0000","last_became_peered":"2024-09-21T05:41:33.635100+0000","last_unstale":"2024-09-21T05:41:34.698414+0000","last_undegraded":"2024-09-21T05:41:34.698414+0000","last_fullsized":"2024-09-21T05:41:34.698414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:23.846337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698332+0000","last_change":"2024-09-21T05:41:32.635937+0000","last_active":"2024-09-21T05:41:34.698332+0000","last_peered":"2024-09-21T05:41:34.698332+0000","last_clean":"2024-09-21T05:41:34.698332+0000","last_became_active":"2024-09-21T05:41:32.635590+0000","last_became_peered":"2024-09-21T05:41:32.635590+0000","last_unstale":"2024-09-21T05:41:34.698332+0000","last_undegraded":"2024-09-21T05:41:34.698332+0000","last_fullsized":"2024-09-21T05:41:34.698332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:04.417273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698987+0000","last_change":"2024-09-21T05:41:33.638288+0000","last_active":"2024-09-21T05:41:34.698987+0000","last_peered":"2024-09-21T05:41:34.698987+0000","last_clean":"2024-09-21T05:41:34.698987+0000","last_became_active":"2024-09-21T05:41:33.637173+0000","last_became_peered":"2024-09-21T05:41:33.637173+0000","last_unstale":"2024-09-21T05:41:34.698987+0000","last_undegraded":"2024-09-21T05:41:34.698987+0000","last_fullsized":"2024-09-21T05:41:34.698987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:39.314026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699057+0000","last_change":"2024-09-21T05:41:32.631706+0000","last_active":"2024-09-21T05:41:34.699057+0000","last_peered":"2024-09-21T05:41:34.699057+0000","last_clean":"2024-09-21T05:41:34.699057+0000","last_became_active":"2024-09-21T05:41:32.631345+0000","last_became_peered":"2024-09-21T05:41:32.631345+0000","last_unstale":"2024-09-21T05:41:34.699057+0000","last_undegraded":"2024-09-21T05:41:34.699057+0000","last_fullsized":"2024-09-21T05:41:34.699057+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:45.182503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697492+0000","last_change":"2024-09-21T05:41:32.635058+0000","last_active":"2024-09-21T05:41:34.697492+0000","last_peered":"2024-09-21T05:41:34.697492+0000","last_clean":"2024-09-21T05:41:34.697492+0000","last_became_active":"2024-09-21T05:41:32.634748+0000","last_became_peered":"2024-09-21T05:41:32.634748+0000","last_unstale":"2024-09-21T05:41:34.697492+0000","last_undegraded":"2024-09-21T05:41:34.697492+0000","last_fullsized":"2024-09-21T05:41:34.697492+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:20.216493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699144+0000","last_change":"2024-09-21T05:41:33.630458+0000","last_active":"2024-09-21T05:41:34.699144+0000","last_peered":"2024-09-21T05:41:34.699144+0000","last_clean":"2024-09-21T05:41:34.699144+0000","last_became_active":"2024-09-21T05:41:33.630058+0000","last_became_peered":"2024-09-21T05:41:33.630058+0000","last_unstale":"2024-09-21T05:41:34.699144+0000","last_undegraded":"2024-09-21T05:41:34.699144+0000","last_fullsized":"2024-09-21T05:41:34.699144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.444704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699426+0000","last_change":"2024-09-21T05:41:32.639401+0000","last_active":"2024-09-21T05:41:34.699426+0000","last_peered":"2024-09-21T05:41:34.699426+0000","last_clean":"2024-09-21T05:41:34.699426+0000","last_became_active":"2024-09-21T05:41:32.639111+0000","last_became_peered":"2024-09-21T05:41:32.639111+0000","last_unstale":"2024-09-21T05:41:34.699426+0000","last_undegraded":"2024-09-21T05:41:34.699426+0000","last_fullsized":"2024-09-21T05:41:34.699426+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:25.720779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698813+0000","last_change":"2024-09-21T05:41:33.642434+0000","last_active":"2024-09-21T05:41:34.698813+0000","last_peered":"2024-09-21T05:41:34.698813+0000","last_clean":"2024-09-21T05:41:34.698813+0000","last_became_active":"2024-09-21T05:41:33.642094+0000","last_became_peered":"2024-09-21T05:41:33.642094+0000","last_unstale":"2024-09-21T05:41:34.698813+0000","last_undegraded":"2024-09-21T05:41:34.698813+0000","last_fullsized":"2024-09-21T05:41:34.698813+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.189588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639233+0000","last_change":"2024-09-21T05:41:32.631531+0000","last_active":"2024-09-21T05:41:34.639233+0000","last_peered":"2024-09-21T05:41:34.639233+0000","last_clean":"2024-09-21T05:41:34.639233+0000","last_became_active":"2024-09-21T05:41:32.631025+0000","last_became_peered":"2024-09-21T05:41:32.631025+0000","last_unstale":"2024-09-21T05:41:34.639233+0000","last_undegraded":"2024-09-21T05:41:34.639233+0000","last_fullsized":"2024-09-21T05:41:34.639233+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:23.285869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698031+0000","last_change":"2024-09-21T05:41:33.623783+0000","last_active":"2024-09-21T05:41:34.698031+0000","last_peered":"2024-09-21T05:41:34.698031+0000","last_clean":"2024-09-21T05:41:34.698031+0000","last_became_active":"2024-09-21T05:41:33.623439+0000","last_became_peered":"2024-09-21T05:41:33.623439+0000","last_unstale":"2024-09-21T05:41:34.698031+0000","last_undegraded":"2024-09-21T05:41:34.698031+0000","last_fullsized":"2024-09-21T05:41:34.698031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:26.723073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697836+0000","last_change":"2024-09-21T05:41:32.631171+0000","last_active":"2024-09-21T05:41:34.697836+0000","last_peered":"2024-09-21T05:41:34.697836+0000","last_clean":"2024-09-21T05:41:34.697836+0000","last_became_active":"2024-09-21T05:41:32.630876+0000","last_became_peered":"2024-09-21T05:41:32.630876+0000","last_unstale":"2024-09-21T05:41:34.697836+0000","last_undegraded":"2024-09-21T05:41:34.697836+0000","last_fullsized":"2024-09-21T05:41:34.697836+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:43.095261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698137+0000","last_change":"2024-09-21T05:41:33.636896+0000","last_active":"2024-09-21T05:41:34.698137+0000","last_peered":"2024-09-21T05:41:34.698137+0000","last_clean":"2024-09-21T05:41:34.698137+0000","last_became_active":"2024-09-21T05:41:33.634526+0000","last_became_peered":"2024-09-21T05:41:33.634526+0000","last_unstale":"2024-09-21T05:41:34.698137+0000","last_undegraded":"2024-09-21T05:41:34.698137+0000","last_fullsized":"2024-09-21T05:41:34.698137+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:21.467807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639825+0000","last_change":"2024-09-21T05:41:32.633802+0000","last_active":"2024-09-21T05:41:34.639825+0000","last_peered":"2024-09-21T05:41:34.639825+0000","last_clean":"2024-09-21T05:41:34.639825+0000","last_became_active":"2024-09-21T05:41:32.633466+0000","last_became_peered":"2024-09-21T05:41:32.633466+0000","last_unstale":"2024-09-21T05:41:34.639825+0000","last_undegraded":"2024-09-21T05:41:34.639825+0000","last_fullsized":"2024-09-21T05:41:34.639825+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:24.054877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698471+0000","last_change":"2024-09-21T05:41:33.638144+0000","last_active":"2024-09-21T05:41:34.698471+0000","last_peered":"2024-09-21T05:41:34.698471+0000","last_clean":"2024-09-21T05:41:34.698471+0000","last_became_active":"2024-09-21T05:41:33.637806+0000","last_became_peered":"2024-09-21T05:41:33.637806+0000","last_unstale":"2024-09-21T05:41:34.698471+0000","last_undegraded":"2024-09-21T05:41:34.698471+0000","last_fullsized":"2024-09-21T05:41:34.698471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:14.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698465+0000","last_change":"2024-09-21T05:41:32.636124+0000","last_active":"2024-09-21T05:41:34.698465+0000","last_peered":"2024-09-21T05:41:34.698465+0000","last_clean":"2024-09-21T05:41:34.698465+0000","last_became_active":"2024-09-21T05:41:32.635756+0000","last_became_peered":"2024-09-21T05:41:32.635756+0000","last_unstale":"2024-09-21T05:41:34.698465+0000","last_undegraded":"2024-09-21T05:41:34.698465+0000","last_fullsized":"2024-09-21T05:41:34.698465+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.430482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639447+0000","last_change":"2024-09-21T05:41:33.634983+0000","last_active":"2024-09-21T05:41:34.639447+0000","last_peered":"2024-09-21T05:41:34.639447+0000","last_clean":"2024-09-21T05:41:34.639447+0000","last_became_active":"2024-09-21T05:41:33.634650+0000","last_became_peered":"2024-09-21T05:41:33.634650+0000","last_unstale":"2024-09-21T05:41:34.639447+0000","last_undegraded":"2024-09-21T05:41:34.639447+0000","last_fullsized":"2024-09-21T05:41:34.639447+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:35.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640744+0000","last_change":"2024-09-21T05:41:32.628780+0000","last_active":"2024-09-21T05:41:34.640744+0000","last_peered":"2024-09-21T05:41:34.640744+0000","last_clean":"2024-09-21T05:41:34.640744+0000","last_became_active":"2024-09-21T05:41:32.627444+0000","last_became_peered":"2024-09-21T05:41:32.627444+0000","last_unstale":"2024-09-21T05:41:34.640744+0000","last_undegraded":"2024-09-21T05:41:34.640744+0000","last_fullsized":"2024-09-21T05:41:34.640744+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:36.131830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698403+0000","last_change":"2024-09-21T05:41:33.629373+0000","last_active":"2024-09-21T05:41:34.698403+0000","last_peered":"2024-09-21T05:41:34.698403+0000","last_clean":"2024-09-21T05:41:34.698403+0000","last_became_active":"2024-09-21T05:41:33.629063+0000","last_became_peered":"2024-09-21T05:41:33.629063+0000","last_unstale":"2024-09-21T05:41:34.698403+0000","last_undegraded":"2024-09-21T05:41:34.698403+0000","last_fullsized":"2024-09-21T05:41:34.698403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:13.577303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698628+0000","last_change":"2024-09-21T05:41:32.639613+0000","last_active":"2024-09-21T05:41:34.698628+0000","last_peered":"2024-09-21T05:41:34.698628+0000","last_clean":"2024-09-21T05:41:34.698628+0000","last_became_active":"2024-09-21T05:41:32.639243+0000","last_became_peered":"2024-09-21T05:41:32.639243+0000","last_unstale":"2024-09-21T05:41:34.698628+0000","last_undegraded":"2024-09-21T05:41:34.698628+0000","last_fullsized":"2024-09-21T05:41:34.698628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:01.439766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698472+0000","last_change":"2024-09-21T05:41:33.639642+0000","last_active":"2024-09-21T05:41:34.698472+0000","last_peered":"2024-09-21T05:41:34.698472+0000","last_clean":"2024-09-21T05:41:34.698472+0000","last_became_active":"2024-09-21T05:41:33.639328+0000","last_became_peered":"2024-09-21T05:41:33.639328+0000","last_unstale":"2024-09-21T05:41:34.698472+0000","last_undegraded":"2024-09-21T05:41:34.698472+0000","last_fullsized":"2024-09-21T05:41:34.698472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:04.614594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639124+0000","last_change":"2024-09-21T05:41:32.627712+0000","last_active":"2024-09-21T05:41:34.639124+0000","last_peered":"2024-09-21T05:41:34.639124+0000","last_clean":"2024-09-21T05:41:34.639124+0000","last_became_active":"2024-09-21T05:41:32.627390+0000","last_became_peered":"2024-09-21T05:41:32.627390+0000","last_unstale":"2024-09-21T05:41:34.639124+0000","last_undegraded":"2024-09-21T05:41:34.639124+0000","last_fullsized":"2024-09-21T05:41:34.639124+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:31.304519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:32.607883+0000","last_change":"2024-09-21T05:41:32.607883+0000","last_active":"2024-09-21T05:41:32.607883+0000","last_peered":"2024-09-21T05:41:32.607883+0000","last_clean":"2024-09-21T05:41:32.607883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:32.607883+0000","last_undegraded":"2024-09-21T05:41:32.607883+0000","last_fullsized":"2024-09-21T05:41:32.607883+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:41:31.605589+0000","last_change":"2024-09-21T05:41:31.605589+0000","last_active":"2024-09-21T05:41:31.605589+0000","last_peered":"2024-09-21T05:41:31.605589+0000","last_clean":"2024-09-21T05:41:31.605589+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:41:31.605589+0000","last_undegraded":"2024-09-21T05:41:31.605589+0000","last_fullsized":"2024-09-21T05:41:31.605589+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639147+0000","last_change":"2024-09-21T05:41:33.635249+0000","last_active":"2024-09-21T05:41:34.639147+0000","last_peered":"2024-09-21T05:41:34.639147+0000","last_clean":"2024-09-21T05:41:34.639147+0000","last_became_active":"2024-09-21T05:41:33.634871+0000","last_became_peered":"2024-09-21T05:41:33.634871+0000","last_unstale":"2024-09-21T05:41:34.639147+0000","last_undegraded":"2024-09-21T05:41:34.639147+0000","last_fullsized":"2024-09-21T05:41:34.639147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:08.079028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639113+0000","last_change":"2024-09-21T05:41:32.638968+0000","last_active":"2024-09-21T05:41:34.639113+0000","last_peered":"2024-09-21T05:41:34.639113+0000","last_clean":"2024-09-21T05:41:34.639113+0000","last_became_active":"2024-09-21T05:41:32.638616+0000","last_became_peered":"2024-09-21T05:41:32.638616+0000","last_unstale":"2024-09-21T05:41:34.639113+0000","last_undegraded":"2024-09-21T05:41:34.639113+0000","last_fullsized":"2024-09-21T05:41:34.639113+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:11.983425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699075+0000","last_change":"2024-09-21T05:41:33.644686+0000","last_active":"2024-09-21T05:41:34.699075+0000","last_peered":"2024-09-21T05:41:34.699075+0000","last_clean":"2024-09-21T05:41:34.699075+0000","last_became_active":"2024-09-21T05:41:33.638504+0000","last_became_peered":"2024-09-21T05:41:33.638504+0000","last_unstale":"2024-09-21T05:41:34.699075+0000","last_undegraded":"2024-09-21T05:41:34.699075+0000","last_fullsized":"2024-09-21T05:41:34.699075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:31.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698266+0000","last_change":"2024-09-21T05:41:32.631328+0000","last_active":"2024-09-21T05:41:34.698266+0000","last_peered":"2024-09-21T05:41:34.698266+0000","last_clean":"2024-09-21T05:41:34.698266+0000","last_became_active":"2024-09-21T05:41:32.630972+0000","last_became_peered":"2024-09-21T05:41:32.630972+0000","last_unstale":"2024-09-21T05:41:34.698266+0000","last_undegraded":"2024-09-21T05:41:34.698266+0000","last_fullsized":"2024-09-21T05:41:34.698266+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:20.626181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698142+0000","last_change":"2024-09-21T05:41:33.646541+0000","last_active":"2024-09-21T05:41:34.698142+0000","last_peered":"2024-09-21T05:41:34.698142+0000","last_clean":"2024-09-21T05:41:34.698142+0000","last_became_active":"2024-09-21T05:41:33.646222+0000","last_became_peered":"2024-09-21T05:41:33.646222+0000","last_unstale":"2024-09-21T05:41:34.698142+0000","last_undegraded":"2024-09-21T05:41:34.698142+0000","last_fullsized":"2024-09-21T05:41:34.698142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:37.177614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640164+0000","last_change":"2024-09-21T05:41:32.627052+0000","last_active":"2024-09-21T05:41:34.640164+0000","last_peered":"2024-09-21T05:41:34.640164+0000","last_clean":"2024-09-21T05:41:34.640164+0000","last_became_active":"2024-09-21T05:41:32.626729+0000","last_became_peered":"2024-09-21T05:41:32.626729+0000","last_unstale":"2024-09-21T05:41:34.640164+0000","last_undegraded":"2024-09-21T05:41:34.640164+0000","last_fullsized":"2024-09-21T05:41:34.640164+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:27.112139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-21T05:41:29.942490+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:41:29.942490+0000","last_peered":"2024-09-21T05:41:29.942490+0000","last_clean":"2024-09-21T05:41:29.942490+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:41:29.942490+0000","last_undegraded":"2024-09-21T05:41:29.942490+0000","last_fullsized":"2024-09-21T05:41:29.942490+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1220608,"data_stored":845787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31816,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":626688,"data_stored":255419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":675840,"data_stored":280578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":684032,"data_stored":281230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":280634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":280634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36016,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335824,"statfs":{"total":96636764160,"available":96599883776,"internally_reserved":0,"allocated":679936,"data_stored":280668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":873140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:41:37.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:41:37.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 -- 172.21.15.22:0/2384427009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 msgr2=0x7f081c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 --2- 172.21.15.22:0/2384427009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f081c004cd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0834070050 tx=0x7f081c005830 comp rx=0 tx=0).stop 2024-09-21T05:41:37.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 -- 172.21.15.22:0/2384427009 shutdown_connections 2024-09-21T05:41:37.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 --2- 172.21.15.22:0/2384427009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f081c004cd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 -- 172.21.15.22:0/2384427009 >> 172.21.15.22:0/2384427009 conn(0x7f083409d7f0 msgr2=0x7f08340d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:37.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 -- 172.21.15.22:0/2384427009 shutdown_connections 2024-09-21T05:41:37.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.857+0000 7f0839e15640 1 -- 172.21.15.22:0/2384427009 wait complete. 2024-09-21T05:41:37.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.858+0000 7f0839e15640 1 Processor -- start 2024-09-21T05:41:37.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.859+0000 7f0839e15640 1 -- start start 2024-09-21T05:41:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0839e15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f0834158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0839e15640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0834158ed0 0x7f08340b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0839e15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08340b48a0 0x7f08340b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0839e15640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08340b6120 con 0x7f081c004800 2024-09-21T05:41:37.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0839e15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08340b5fa0 con 0x7f08340b48a0 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0839e15640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08340b62a0 con 0x7f0834158ed0 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f0834158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f0834158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45444/0 (socket says 172.21.15.22:45444) 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 -- 172.21.15.22:0/2355246548 learned_addr learned my addr 172.21.15.22:0/2355246548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0832ffd640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0834158ed0 0x7f08340b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f0833fff640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08340b48a0 0x7f08340b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 -- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0834158ed0 msgr2=0x7f08340b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0834158ed0 0x7f08340b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 -- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08340b48a0 msgr2=0x7f08340b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08340b48a0 0x7f08340b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.860+0000 7f08337fe640 1 -- 172.21.15.22:0/2355246548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0834158780 con 0x7f081c004800 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.861+0000 7f08337fe640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f0834158240 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0824004d90 tx=0x7f08240017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:37.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.861+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824017070 con 0x7f081c004800 2024-09-21T05:41:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.861+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0824010900 con 0x7f081c004800 2024-09-21T05:41:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.861+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824010d50 con 0x7f081c004800 2024-09-21T05:41:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.861+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08340b4cb0 con 0x7f081c004800 2024-09-21T05:41:37.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.861+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08340cae20 con 0x7f081c004800 2024-09-21T05:41:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.862+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08340b67b0 con 0x7f081c004800 2024-09-21T05:41:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.863+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f082401a360 con 0x7f081c004800 2024-09-21T05:41:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.863+0000 7f081bfff640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078bc0 0x7f082007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.863+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f082409aa50 con 0x7f081c004800 2024-09-21T05:41:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.863+0000 7f0832ffd640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078bc0 0x7f082007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:37.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.864+0000 7f0832ffd640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078bc0 0x7f082007b0d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0828035000 tx=0x7f08280046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:37.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:37.868+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f08340b4cb0 con 0x7f081c004800 2024-09-21T05:41:38.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.050+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08340d3fb0 con 0x7f0820078bc0 2024-09-21T05:41:38.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.055+0000 7f081bfff640 1 -- 172.21.15.22:0/2355246548 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251943 (secure 0 0 0) 0x7f08340d3fb0 con 0x7f0820078bc0 2024-09-21T05:41:38.060 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:38.061 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:41:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078bc0 msgr2=0x7f082007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078bc0 0x7f082007b0d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0828035000 tx=0x7f08280046c0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 msgr2=0x7f0834158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f0834158240 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0824004d90 tx=0x7f08240017f0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 shutdown_connections 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f08340b48a0 0x7f08340b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078bc0 0x7f082007b0d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0834158ed0 0x7f08340b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 --2- 172.21.15.22:0/2355246548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c004800 0x7f0834158240 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 >> 172.21.15.22:0/2355246548 conn(0x7f083409d7f0 msgr2=0x7f08341617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.058+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 shutdown_connections 2024-09-21T05:41:38.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.059+0000 7f0839e15640 1 -- 172.21.15.22:0/2355246548 wait complete. 2024-09-21T05:41:38.071 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-21T05:41:37.770569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256040,"kb_used_data":6504,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754718680,"statfs":{"total":773094113280,"available":772831928320,"internally_reserved":0,"allocated":6660096,"data_stored":3430180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001357"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697571+0000","last_change":"2024-09-21T05:41:33.648732+0000","last_active":"2024-09-21T05:41:34.697571+0000","last_peered":"2024-09-21T05:41:34.697571+0000","last_clean":"2024-09-21T05:41:34.697571+0000","last_became_active":"2024-09-21T05:41:33.648430+0000","last_became_peered":"2024-09-21T05:41:33.648430+0000","last_unstale":"2024-09-21T05:41:34.697571+0000","last_undegraded":"2024-09-21T05:41:34.697571+0000","last_fullsized":"2024-09-21T05:41:34.697571+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:09.569594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699401+0000","last_change":"2024-09-21T05:41:33.639552+0000","last_active":"2024-09-21T05:41:34.699401+0000","last_peered":"2024-09-21T05:41:34.699401+0000","last_clean":"2024-09-21T05:41:34.699401+0000","last_became_active":"2024-09-21T05:41:33.639162+0000","last_became_peered":"2024-09-21T05:41:33.639162+0000","last_unstale":"2024-09-21T05:41:34.699401+0000","last_undegraded":"2024-09-21T05:41:34.699401+0000","last_fullsized":"2024-09-21T05:41:34.699401+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:22.369174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698010+0000","last_change":"2024-09-21T05:41:33.632608+0000","last_active":"2024-09-21T05:41:34.698010+0000","last_peered":"2024-09-21T05:41:34.698010+0000","last_clean":"2024-09-21T05:41:34.698010+0000","last_became_active":"2024-09-21T05:41:33.632262+0000","last_became_peered":"2024-09-21T05:41:33.632262+0000","last_unstale":"2024-09-21T05:41:34.698010+0000","last_undegraded":"2024-09-21T05:41:34.698010+0000","last_fullsized":"2024-09-21T05:41:34.698010+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:49.337617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698111+0000","last_change":"2024-09-21T05:41:33.635230+0000","last_active":"2024-09-21T05:41:34.698111+0000","last_peered":"2024-09-21T05:41:34.698111+0000","last_clean":"2024-09-21T05:41:34.698111+0000","last_became_active":"2024-09-21T05:41:33.634821+0000","last_became_peered":"2024-09-21T05:41:33.634821+0000","last_unstale":"2024-09-21T05:41:34.698111+0000","last_undegraded":"2024-09-21T05:41:34.698111+0000","last_fullsized":"2024-09-21T05:41:34.698111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:44.928717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697251+0000","last_change":"2024-09-21T05:41:33.641105+0000","last_active":"2024-09-21T05:41:34.697251+0000","last_peered":"2024-09-21T05:41:34.697251+0000","last_clean":"2024-09-21T05:41:34.697251+0000","last_became_active":"2024-09-21T05:41:33.640769+0000","last_became_peered":"2024-09-21T05:41:33.640769+0000","last_unstale":"2024-09-21T05:41:34.697251+0000","last_undegraded":"2024-09-21T05:41:34.697251+0000","last_fullsized":"2024-09-21T05:41:34.697251+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:52.338089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697422+0000","last_change":"2024-09-21T05:41:33.643002+0000","last_active":"2024-09-21T05:41:34.697422+0000","last_peered":"2024-09-21T05:41:34.697422+0000","last_clean":"2024-09-21T05:41:34.697422+0000","last_became_active":"2024-09-21T05:41:33.642457+0000","last_became_peered":"2024-09-21T05:41:33.642457+0000","last_unstale":"2024-09-21T05:41:34.697422+0000","last_undegraded":"2024-09-21T05:41:34.697422+0000","last_fullsized":"2024-09-21T05:41:34.697422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:10.078710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.641140+0000","last_change":"2024-09-21T05:41:33.640412+0000","last_active":"2024-09-21T05:41:34.641140+0000","last_peered":"2024-09-21T05:41:34.641140+0000","last_clean":"2024-09-21T05:41:34.641140+0000","last_became_active":"2024-09-21T05:41:33.640059+0000","last_became_peered":"2024-09-21T05:41:33.640059+0000","last_unstale":"2024-09-21T05:41:34.641140+0000","last_undegraded":"2024-09-21T05:41:34.641140+0000","last_fullsized":"2024-09-21T05:41:34.641140+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:00.819617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697914+0000","last_change":"2024-09-21T05:41:33.636407+0000","last_active":"2024-09-21T05:41:34.697914+0000","last_peered":"2024-09-21T05:41:34.697914+0000","last_clean":"2024-09-21T05:41:34.697914+0000","last_became_active":"2024-09-21T05:41:33.636081+0000","last_became_peered":"2024-09-21T05:41:33.636081+0000","last_unstale":"2024-09-21T05:41:34.697914+0000","last_undegraded":"2024-09-21T05:41:34.697914+0000","last_fullsized":"2024-09-21T05:41:34.697914+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:25:28.637017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.641298+0000","last_change":"2024-09-21T05:41:33.635833+0000","last_active":"2024-09-21T05:41:34.641298+0000","last_peered":"2024-09-21T05:41:34.641298+0000","last_clean":"2024-09-21T05:41:34.641298+0000","last_became_active":"2024-09-21T05:41:33.635527+0000","last_became_peered":"2024-09-21T05:41:33.635527+0000","last_unstale":"2024-09-21T05:41:34.641298+0000","last_undegraded":"2024-09-21T05:41:34.641298+0000","last_fullsized":"2024-09-21T05:41:34.641298+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:41.300665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639819+0000","last_change":"2024-09-21T05:41:33.632622+0000","last_active":"2024-09-21T05:41:34.639819+0000","last_peered":"2024-09-21T05:41:34.639819+0000","last_clean":"2024-09-21T05:41:34.639819+0000","last_became_active":"2024-09-21T05:41:33.632328+0000","last_became_peered":"2024-09-21T05:41:33.632328+0000","last_unstale":"2024-09-21T05:41:34.639819+0000","last_undegraded":"2024-09-21T05:41:34.639819+0000","last_fullsized":"2024-09-21T05:41:34.639819+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:17:15.022568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639699+0000","last_change":"2024-09-21T05:41:33.644830+0000","last_active":"2024-09-21T05:41:34.639699+0000","last_peered":"2024-09-21T05:41:34.639699+0000","last_clean":"2024-09-21T05:41:34.639699+0000","last_became_active":"2024-09-21T05:41:33.644225+0000","last_became_peered":"2024-09-21T05:41:33.644225+0000","last_unstale":"2024-09-21T05:41:34.639699+0000","last_undegraded":"2024-09-21T05:41:34.639699+0000","last_fullsized":"2024-09-21T05:41:34.639699+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:39:19.195318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698394+0000","last_change":"2024-09-21T05:41:33.638131+0000","last_active":"2024-09-21T05:41:34.698394+0000","last_peered":"2024-09-21T05:41:34.698394+0000","last_clean":"2024-09-21T05:41:34.698394+0000","last_became_active":"2024-09-21T05:41:33.637594+0000","last_became_peered":"2024-09-21T05:41:33.637594+0000","last_unstale":"2024-09-21T05:41:34.698394+0000","last_undegraded":"2024-09-21T05:41:34.698394+0000","last_fullsized":"2024-09-21T05:41:34.698394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:57.994501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640491+0000","last_change":"2024-09-21T05:41:33.648300+0000","last_active":"2024-09-21T05:41:34.640491+0000","last_peered":"2024-09-21T05:41:34.640491+0000","last_clean":"2024-09-21T05:41:34.640491+0000","last_became_active":"2024-09-21T05:41:33.647962+0000","last_became_peered":"2024-09-21T05:41:33.647962+0000","last_unstale":"2024-09-21T05:41:34.640491+0000","last_undegraded":"2024-09-21T05:41:34.640491+0000","last_fullsized":"2024-09-21T05:41:34.640491+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:19.289057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640873+0000","last_change":"2024-09-21T05:41:33.650754+0000","last_active":"2024-09-21T05:41:34.640873+0000","last_peered":"2024-09-21T05:41:34.640873+0000","last_clean":"2024-09-21T05:41:34.640873+0000","last_became_active":"2024-09-21T05:41:33.650410+0000","last_became_peered":"2024-09-21T05:41:33.650410+0000","last_unstale":"2024-09-21T05:41:34.640873+0000","last_undegraded":"2024-09-21T05:41:34.640873+0000","last_fullsized":"2024-09-21T05:41:34.640873+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:30.257592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640846+0000","last_change":"2024-09-21T05:41:33.632787+0000","last_active":"2024-09-21T05:41:34.640846+0000","last_peered":"2024-09-21T05:41:34.640846+0000","last_clean":"2024-09-21T05:41:34.640846+0000","last_became_active":"2024-09-21T05:41:33.632440+0000","last_became_peered":"2024-09-21T05:41:33.632440+0000","last_unstale":"2024-09-21T05:41:34.640846+0000","last_undegraded":"2024-09-21T05:41:34.640846+0000","last_fullsized":"2024-09-21T05:41:34.640846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:37.504675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697880+0000","last_change":"2024-09-21T05:41:33.642382+0000","last_active":"2024-09-21T05:41:34.697880+0000","last_peered":"2024-09-21T05:41:34.697880+0000","last_clean":"2024-09-21T05:41:34.697880+0000","last_became_active":"2024-09-21T05:41:33.642033+0000","last_became_peered":"2024-09-21T05:41:33.642033+0000","last_unstale":"2024-09-21T05:41:34.697880+0000","last_undegraded":"2024-09-21T05:41:34.697880+0000","last_fullsized":"2024-09-21T05:41:34.697880+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:56.658021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698133+0000","last_change":"2024-09-21T05:41:33.641030+0000","last_active":"2024-09-21T05:41:34.698133+0000","last_peered":"2024-09-21T05:41:34.698133+0000","last_clean":"2024-09-21T05:41:34.698133+0000","last_became_active":"2024-09-21T05:41:33.640701+0000","last_became_peered":"2024-09-21T05:41:33.640701+0000","last_unstale":"2024-09-21T05:41:34.698133+0000","last_undegraded":"2024-09-21T05:41:34.698133+0000","last_fullsized":"2024-09-21T05:41:34.698133+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:53.840007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698176+0000","last_change":"2024-09-21T05:41:33.626701+0000","last_active":"2024-09-21T05:41:34.698176+0000","last_peered":"2024-09-21T05:41:34.698176+0000","last_clean":"2024-09-21T05:41:34.698176+0000","last_became_active":"2024-09-21T05:41:33.626320+0000","last_became_peered":"2024-09-21T05:41:33.626320+0000","last_unstale":"2024-09-21T05:41:34.698176+0000","last_undegraded":"2024-09-21T05:41:34.698176+0000","last_fullsized":"2024-09-21T05:41:34.698176+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:06:33.875654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697313+0000","last_change":"2024-09-21T05:41:33.643493+0000","last_active":"2024-09-21T05:41:34.697313+0000","last_peered":"2024-09-21T05:41:34.697313+0000","last_clean":"2024-09-21T05:41:34.697313+0000","last_became_active":"2024-09-21T05:41:33.643183+0000","last_became_peered":"2024-09-21T05:41:33.643183+0000","last_unstale":"2024-09-21T05:41:34.697313+0000","last_undegraded":"2024-09-21T05:41:34.697313+0000","last_fullsized":"2024-09-21T05:41:34.697313+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:35.522759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698761+0000","last_change":"2024-09-21T05:41:33.640103+0000","last_active":"2024-09-21T05:41:34.698761+0000","last_peered":"2024-09-21T05:41:34.698761+0000","last_clean":"2024-09-21T05:41:34.698761+0000","last_became_active":"2024-09-21T05:41:33.636395+0000","last_became_peered":"2024-09-21T05:41:33.636395+0000","last_unstale":"2024-09-21T05:41:34.698761+0000","last_undegraded":"2024-09-21T05:41:34.698761+0000","last_fullsized":"2024-09-21T05:41:34.698761+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:28.536973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697972+0000","last_change":"2024-09-21T05:41:33.626832+0000","last_active":"2024-09-21T05:41:34.697972+0000","last_peered":"2024-09-21T05:41:34.697972+0000","last_clean":"2024-09-21T05:41:34.697972+0000","last_became_active":"2024-09-21T05:41:33.626506+0000","last_became_peered":"2024-09-21T05:41:33.626506+0000","last_unstale":"2024-09-21T05:41:34.697972+0000","last_undegraded":"2024-09-21T05:41:34.697972+0000","last_fullsized":"2024-09-21T05:41:34.697972+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:51.195606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698034+0000","last_change":"2024-09-21T05:41:33.645860+0000","last_active":"2024-09-21T05:41:34.698034+0000","last_peered":"2024-09-21T05:41:34.698034+0000","last_clean":"2024-09-21T05:41:34.698034+0000","last_became_active":"2024-09-21T05:41:33.645551+0000","last_became_peered":"2024-09-21T05:41:33.645551+0000","last_unstale":"2024-09-21T05:41:34.698034+0000","last_undegraded":"2024-09-21T05:41:34.698034+0000","last_fullsized":"2024-09-21T05:41:34.698034+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:24.769718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698548+0000","last_change":"2024-09-21T05:41:33.646921+0000","last_active":"2024-09-21T05:41:34.698548+0000","last_peered":"2024-09-21T05:41:34.698548+0000","last_clean":"2024-09-21T05:41:34.698548+0000","last_became_active":"2024-09-21T05:41:33.646470+0000","last_became_peered":"2024-09-21T05:41:33.646470+0000","last_unstale":"2024-09-21T05:41:34.698548+0000","last_undegraded":"2024-09-21T05:41:34.698548+0000","last_fullsized":"2024-09-21T05:41:34.698548+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:04:41.797762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640905+0000","last_change":"2024-09-21T05:41:33.648558+0000","last_active":"2024-09-21T05:41:34.640905+0000","last_peered":"2024-09-21T05:41:34.640905+0000","last_clean":"2024-09-21T05:41:34.640905+0000","last_became_active":"2024-09-21T05:41:33.648206+0000","last_became_peered":"2024-09-21T05:41:33.648206+0000","last_unstale":"2024-09-21T05:41:34.640905+0000","last_undegraded":"2024-09-21T05:41:34.640905+0000","last_fullsized":"2024-09-21T05:41:34.640905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:13.444641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697785+0000","last_change":"2024-09-21T05:41:33.645467+0000","last_active":"2024-09-21T05:41:34.697785+0000","last_peered":"2024-09-21T05:41:34.697785+0000","last_clean":"2024-09-21T05:41:34.697785+0000","last_became_active":"2024-09-21T05:41:33.645113+0000","last_became_peered":"2024-09-21T05:41:33.645113+0000","last_unstale":"2024-09-21T05:41:34.697785+0000","last_undegraded":"2024-09-21T05:41:34.697785+0000","last_fullsized":"2024-09-21T05:41:34.697785+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:46.221950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698655+0000","last_change":"2024-09-21T05:41:33.634591+0000","last_active":"2024-09-21T05:41:34.698655+0000","last_peered":"2024-09-21T05:41:34.698655+0000","last_clean":"2024-09-21T05:41:34.698655+0000","last_became_active":"2024-09-21T05:41:33.634265+0000","last_became_peered":"2024-09-21T05:41:33.634265+0000","last_unstale":"2024-09-21T05:41:34.698655+0000","last_undegraded":"2024-09-21T05:41:34.698655+0000","last_fullsized":"2024-09-21T05:41:34.698655+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:33.756146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697951+0000","last_change":"2024-09-21T05:41:33.632933+0000","last_active":"2024-09-21T05:41:34.697951+0000","last_peered":"2024-09-21T05:41:34.697951+0000","last_clean":"2024-09-21T05:41:34.697951+0000","last_became_active":"2024-09-21T05:41:33.632625+0000","last_became_peered":"2024-09-21T05:41:33.632625+0000","last_unstale":"2024-09-21T05:41:34.697951+0000","last_undegraded":"2024-09-21T05:41:34.697951+0000","last_fullsized":"2024-09-21T05:41:34.697951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:05.047303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697677+0000","last_change":"2024-09-21T05:41:33.641926+0000","last_active":"2024-09-21T05:41:34.697677+0000","last_peered":"2024-09-21T05:41:34.697677+0000","last_clean":"2024-09-21T05:41:34.697677+0000","last_became_active":"2024-09-21T05:41:33.641620+0000","last_became_peered":"2024-09-21T05:41:33.641620+0000","last_unstale":"2024-09-21T05:41:34.697677+0000","last_undegraded":"2024-09-21T05:41:34.697677+0000","last_fullsized":"2024-09-21T05:41:34.697677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:43.487445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699386+0000","last_change":"2024-09-21T05:41:33.639651+0000","last_active":"2024-09-21T05:41:34.699386+0000","last_peered":"2024-09-21T05:41:34.699386+0000","last_clean":"2024-09-21T05:41:34.699386+0000","last_became_active":"2024-09-21T05:41:33.639293+0000","last_became_peered":"2024-09-21T05:41:33.639293+0000","last_unstale":"2024-09-21T05:41:34.699386+0000","last_undegraded":"2024-09-21T05:41:34.699386+0000","last_fullsized":"2024-09-21T05:41:34.699386+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:51.923976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697906+0000","last_change":"2024-09-21T05:41:33.634478+0000","last_active":"2024-09-21T05:41:34.697906+0000","last_peered":"2024-09-21T05:41:34.697906+0000","last_clean":"2024-09-21T05:41:34.697906+0000","last_became_active":"2024-09-21T05:41:33.634143+0000","last_became_peered":"2024-09-21T05:41:33.634143+0000","last_unstale":"2024-09-21T05:41:34.697906+0000","last_undegraded":"2024-09-21T05:41:34.697906+0000","last_fullsized":"2024-09-21T05:41:34.697906+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:36.866973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697155+0000","last_change":"2024-09-21T05:41:33.645164+0000","last_active":"2024-09-21T05:41:34.697155+0000","last_peered":"2024-09-21T05:41:34.697155+0000","last_clean":"2024-09-21T05:41:34.697155+0000","last_became_active":"2024-09-21T05:41:33.644826+0000","last_became_peered":"2024-09-21T05:41:33.644826+0000","last_unstale":"2024-09-21T05:41:34.697155+0000","last_undegraded":"2024-09-21T05:41:34.697155+0000","last_fullsized":"2024-09-21T05:41:34.697155+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:30.888264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698102+0000","last_change":"2024-09-21T05:41:33.652970+0000","last_active":"2024-09-21T05:41:34.698102+0000","last_peered":"2024-09-21T05:41:34.698102+0000","last_clean":"2024-09-21T05:41:34.698102+0000","last_became_active":"2024-09-21T05:41:33.652536+0000","last_became_peered":"2024-09-21T05:41:33.652536+0000","last_unstale":"2024-09-21T05:41:34.698102+0000","last_undegraded":"2024-09-21T05:41:34.698102+0000","last_fullsized":"2024-09-21T05:41:34.698102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:43.986414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698406+0000","last_change":"2024-09-21T05:41:33.643684+0000","last_active":"2024-09-21T05:41:34.698406+0000","last_peered":"2024-09-21T05:41:34.698406+0000","last_clean":"2024-09-21T05:41:34.698406+0000","last_became_active":"2024-09-21T05:41:33.643363+0000","last_became_peered":"2024-09-21T05:41:33.643363+0000","last_unstale":"2024-09-21T05:41:34.698406+0000","last_undegraded":"2024-09-21T05:41:34.698406+0000","last_fullsized":"2024-09-21T05:41:34.698406+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:33.385517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"56'4","reported_seq":19,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697753+0000","last_change":"2024-09-21T05:41:32.638348+0000","last_active":"2024-09-21T05:41:34.697753+0000","last_peered":"2024-09-21T05:41:34.697753+0000","last_clean":"2024-09-21T05:41:34.697753+0000","last_became_active":"2024-09-21T05:41:32.638033+0000","last_became_peered":"2024-09-21T05:41:32.638033+0000","last_unstale":"2024-09-21T05:41:34.697753+0000","last_undegraded":"2024-09-21T05:41:34.697753+0000","last_fullsized":"2024-09-21T05:41:34.697753+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:40.689053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639883+0000","last_change":"2024-09-21T05:41:33.634060+0000","last_active":"2024-09-21T05:41:34.639883+0000","last_peered":"2024-09-21T05:41:34.639883+0000","last_clean":"2024-09-21T05:41:34.639883+0000","last_became_active":"2024-09-21T05:41:33.633362+0000","last_became_peered":"2024-09-21T05:41:33.633362+0000","last_unstale":"2024-09-21T05:41:34.639883+0000","last_undegraded":"2024-09-21T05:41:34.639883+0000","last_fullsized":"2024-09-21T05:41:34.639883+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:02.763484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699332+0000","last_change":"2024-09-21T05:41:32.638551+0000","last_active":"2024-09-21T05:41:34.699332+0000","last_peered":"2024-09-21T05:41:34.699332+0000","last_clean":"2024-09-21T05:41:34.699332+0000","last_became_active":"2024-09-21T05:41:32.638262+0000","last_became_peered":"2024-09-21T05:41:32.638262+0000","last_unstale":"2024-09-21T05:41:34.699332+0000","last_undegraded":"2024-09-21T05:41:34.699332+0000","last_fullsized":"2024-09-21T05:41:34.699332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:53.009886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697558+0000","last_change":"2024-09-21T05:41:33.634059+0000","last_active":"2024-09-21T05:41:34.697558+0000","last_peered":"2024-09-21T05:41:34.697558+0000","last_clean":"2024-09-21T05:41:34.697558+0000","last_became_active":"2024-09-21T05:41:33.633746+0000","last_became_peered":"2024-09-21T05:41:33.633746+0000","last_unstale":"2024-09-21T05:41:34.697558+0000","last_undegraded":"2024-09-21T05:41:34.697558+0000","last_fullsized":"2024-09-21T05:41:34.697558+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:21.628475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698529+0000","last_change":"2024-09-21T05:41:32.633871+0000","last_active":"2024-09-21T05:41:34.698529+0000","last_peered":"2024-09-21T05:41:34.698529+0000","last_clean":"2024-09-21T05:41:34.698529+0000","last_became_active":"2024-09-21T05:41:32.633531+0000","last_became_peered":"2024-09-21T05:41:32.633531+0000","last_unstale":"2024-09-21T05:41:34.698529+0000","last_undegraded":"2024-09-21T05:41:34.698529+0000","last_fullsized":"2024-09-21T05:41:34.698529+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:48:55.239582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698180+0000","last_change":"2024-09-21T05:41:33.631513+0000","last_active":"2024-09-21T05:41:34.698180+0000","last_peered":"2024-09-21T05:41:34.698180+0000","last_clean":"2024-09-21T05:41:34.698180+0000","last_became_active":"2024-09-21T05:41:33.631198+0000","last_became_peered":"2024-09-21T05:41:33.631198+0000","last_unstale":"2024-09-21T05:41:34.698180+0000","last_undegraded":"2024-09-21T05:41:34.698180+0000","last_fullsized":"2024-09-21T05:41:34.698180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:27:37.047470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698543+0000","last_change":"2024-09-21T05:41:32.632245+0000","last_active":"2024-09-21T05:41:34.698543+0000","last_peered":"2024-09-21T05:41:34.698543+0000","last_clean":"2024-09-21T05:41:34.698543+0000","last_became_active":"2024-09-21T05:41:32.631526+0000","last_became_peered":"2024-09-21T05:41:32.631526+0000","last_unstale":"2024-09-21T05:41:34.698543+0000","last_undegraded":"2024-09-21T05:41:34.698543+0000","last_fullsized":"2024-09-21T05:41:34.698543+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:39.894103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698256+0000","last_change":"2024-09-21T05:41:33.645710+0000","last_active":"2024-09-21T05:41:34.698256+0000","last_peered":"2024-09-21T05:41:34.698256+0000","last_clean":"2024-09-21T05:41:34.698256+0000","last_became_active":"2024-09-21T05:41:33.645134+0000","last_became_peered":"2024-09-21T05:41:33.645134+0000","last_unstale":"2024-09-21T05:41:34.698256+0000","last_undegraded":"2024-09-21T05:41:34.698256+0000","last_fullsized":"2024-09-21T05:41:34.698256+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:08.589704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640572+0000","last_change":"2024-09-21T05:41:32.634482+0000","last_active":"2024-09-21T05:41:34.640572+0000","last_peered":"2024-09-21T05:41:34.640572+0000","last_clean":"2024-09-21T05:41:34.640572+0000","last_became_active":"2024-09-21T05:41:32.634176+0000","last_became_peered":"2024-09-21T05:41:32.634176+0000","last_unstale":"2024-09-21T05:41:34.640572+0000","last_undegraded":"2024-09-21T05:41:34.640572+0000","last_fullsized":"2024-09-21T05:41:34.640572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:52:59.721529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699301+0000","last_change":"2024-09-21T05:41:33.638399+0000","last_active":"2024-09-21T05:41:34.699301+0000","last_peered":"2024-09-21T05:41:34.699301+0000","last_clean":"2024-09-21T05:41:34.699301+0000","last_became_active":"2024-09-21T05:41:33.638058+0000","last_became_peered":"2024-09-21T05:41:33.638058+0000","last_unstale":"2024-09-21T05:41:34.699301+0000","last_undegraded":"2024-09-21T05:41:34.699301+0000","last_fullsized":"2024-09-21T05:41:34.699301+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:57:49.640995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697553+0000","last_change":"2024-09-21T05:41:32.637365+0000","last_active":"2024-09-21T05:41:34.697553+0000","last_peered":"2024-09-21T05:41:34.697553+0000","last_clean":"2024-09-21T05:41:34.697553+0000","last_became_active":"2024-09-21T05:41:32.637034+0000","last_became_peered":"2024-09-21T05:41:32.637034+0000","last_unstale":"2024-09-21T05:41:34.697553+0000","last_undegraded":"2024-09-21T05:41:34.697553+0000","last_fullsized":"2024-09-21T05:41:34.697553+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:24.572516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640901+0000","last_change":"2024-09-21T05:41:33.631512+0000","last_active":"2024-09-21T05:41:34.640901+0000","last_peered":"2024-09-21T05:41:34.640901+0000","last_clean":"2024-09-21T05:41:34.640901+0000","last_became_active":"2024-09-21T05:41:33.631171+0000","last_became_peered":"2024-09-21T05:41:33.631171+0000","last_unstale":"2024-09-21T05:41:34.640901+0000","last_undegraded":"2024-09-21T05:41:34.640901+0000","last_fullsized":"2024-09-21T05:41:34.640901+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:13.447138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698119+0000","last_change":"2024-09-21T05:41:32.635145+0000","last_active":"2024-09-21T05:41:34.698119+0000","last_peered":"2024-09-21T05:41:34.698119+0000","last_clean":"2024-09-21T05:41:34.698119+0000","last_became_active":"2024-09-21T05:41:32.634786+0000","last_became_peered":"2024-09-21T05:41:32.634786+0000","last_unstale":"2024-09-21T05:41:34.698119+0000","last_undegraded":"2024-09-21T05:41:34.698119+0000","last_fullsized":"2024-09-21T05:41:34.698119+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:35.749308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699104+0000","last_change":"2024-09-21T05:41:33.638892+0000","last_active":"2024-09-21T05:41:34.699104+0000","last_peered":"2024-09-21T05:41:34.699104+0000","last_clean":"2024-09-21T05:41:34.699104+0000","last_became_active":"2024-09-21T05:41:33.638106+0000","last_became_peered":"2024-09-21T05:41:33.638106+0000","last_unstale":"2024-09-21T05:41:34.699104+0000","last_undegraded":"2024-09-21T05:41:34.699104+0000","last_fullsized":"2024-09-21T05:41:34.699104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:33:23.528753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.641273+0000","last_change":"2024-09-21T05:41:32.633948+0000","last_active":"2024-09-21T05:41:34.641273+0000","last_peered":"2024-09-21T05:41:34.641273+0000","last_clean":"2024-09-21T05:41:34.641273+0000","last_became_active":"2024-09-21T05:41:32.633569+0000","last_became_peered":"2024-09-21T05:41:32.633569+0000","last_unstale":"2024-09-21T05:41:34.641273+0000","last_undegraded":"2024-09-21T05:41:34.641273+0000","last_fullsized":"2024-09-21T05:41:34.641273+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:01.761189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.641472+0000","last_change":"2024-09-21T05:41:33.645062+0000","last_active":"2024-09-21T05:41:34.641472+0000","last_peered":"2024-09-21T05:41:34.641472+0000","last_clean":"2024-09-21T05:41:34.641472+0000","last_became_active":"2024-09-21T05:41:33.644765+0000","last_became_peered":"2024-09-21T05:41:33.644765+0000","last_unstale":"2024-09-21T05:41:34.641472+0000","last_undegraded":"2024-09-21T05:41:34.641472+0000","last_fullsized":"2024-09-21T05:41:34.641472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:25.783888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699027+0000","last_change":"2024-09-21T05:41:32.638548+0000","last_active":"2024-09-21T05:41:34.699027+0000","last_peered":"2024-09-21T05:41:34.699027+0000","last_clean":"2024-09-21T05:41:34.699027+0000","last_became_active":"2024-09-21T05:41:32.638160+0000","last_became_peered":"2024-09-21T05:41:32.638160+0000","last_unstale":"2024-09-21T05:41:34.699027+0000","last_undegraded":"2024-09-21T05:41:34.699027+0000","last_fullsized":"2024-09-21T05:41:34.699027+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:35:54.282398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699126+0000","last_change":"2024-09-21T05:41:33.632238+0000","last_active":"2024-09-21T05:41:34.699126+0000","last_peered":"2024-09-21T05:41:34.699126+0000","last_clean":"2024-09-21T05:41:34.699126+0000","last_became_active":"2024-09-21T05:41:33.631910+0000","last_became_peered":"2024-09-21T05:41:33.631910+0000","last_unstale":"2024-09-21T05:41:34.699126+0000","last_undegraded":"2024-09-21T05:41:34.699126+0000","last_fullsized":"2024-09-21T05:41:34.699126+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:17.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699431+0000","last_change":"2024-09-21T05:41:32.635026+0000","last_active":"2024-09-21T05:41:34.699431+0000","last_peered":"2024-09-21T05:41:34.699431+0000","last_clean":"2024-09-21T05:41:34.699431+0000","last_became_active":"2024-09-21T05:41:32.634713+0000","last_became_peered":"2024-09-21T05:41:32.634713+0000","last_unstale":"2024-09-21T05:41:34.699431+0000","last_undegraded":"2024-09-21T05:41:34.699431+0000","last_fullsized":"2024-09-21T05:41:34.699431+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:09.076416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639394+0000","last_change":"2024-09-21T05:41:33.636619+0000","last_active":"2024-09-21T05:41:34.639394+0000","last_peered":"2024-09-21T05:41:34.639394+0000","last_clean":"2024-09-21T05:41:34.639394+0000","last_became_active":"2024-09-21T05:41:33.636303+0000","last_became_peered":"2024-09-21T05:41:33.636303+0000","last_unstale":"2024-09-21T05:41:34.639394+0000","last_undegraded":"2024-09-21T05:41:34.639394+0000","last_fullsized":"2024-09-21T05:41:34.639394+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:48.982626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"56'3","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699156+0000","last_change":"2024-09-21T05:41:32.637770+0000","last_active":"2024-09-21T05:41:34.699156+0000","last_peered":"2024-09-21T05:41:34.699156+0000","last_clean":"2024-09-21T05:41:34.699156+0000","last_became_active":"2024-09-21T05:41:32.637457+0000","last_became_peered":"2024-09-21T05:41:32.637457+0000","last_unstale":"2024-09-21T05:41:34.699156+0000","last_undegraded":"2024-09-21T05:41:34.699156+0000","last_fullsized":"2024-09-21T05:41:34.699156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:41:10.646548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698504+0000","last_change":"2024-09-21T05:41:33.648957+0000","last_active":"2024-09-21T05:41:34.698504+0000","last_peered":"2024-09-21T05:41:34.698504+0000","last_clean":"2024-09-21T05:41:34.698504+0000","last_became_active":"2024-09-21T05:41:33.648417+0000","last_became_peered":"2024-09-21T05:41:33.648417+0000","last_unstale":"2024-09-21T05:41:34.698504+0000","last_undegraded":"2024-09-21T05:41:34.698504+0000","last_fullsized":"2024-09-21T05:41:34.698504+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:00:50.207425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"56'6","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.641327+0000","last_change":"2024-09-21T05:41:32.641014+0000","last_active":"2024-09-21T05:41:34.641327+0000","last_peered":"2024-09-21T05:41:34.641327+0000","last_clean":"2024-09-21T05:41:34.641327+0000","last_became_active":"2024-09-21T05:41:32.640700+0000","last_became_peered":"2024-09-21T05:41:32.640700+0000","last_unstale":"2024-09-21T05:41:34.641327+0000","last_undegraded":"2024-09-21T05:41:34.641327+0000","last_fullsized":"2024-09-21T05:41:34.641327+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:25.280060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697772+0000","last_change":"2024-09-21T05:41:33.635658+0000","last_active":"2024-09-21T05:41:34.697772+0000","last_peered":"2024-09-21T05:41:34.697772+0000","last_clean":"2024-09-21T05:41:34.697772+0000","last_became_active":"2024-09-21T05:41:33.635345+0000","last_became_peered":"2024-09-21T05:41:33.635345+0000","last_unstale":"2024-09-21T05:41:34.697772+0000","last_undegraded":"2024-09-21T05:41:34.697772+0000","last_fullsized":"2024-09-21T05:41:34.697772+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:28.402298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699252+0000","last_change":"2024-09-21T05:41:32.639894+0000","last_active":"2024-09-21T05:41:34.699252+0000","last_peered":"2024-09-21T05:41:34.699252+0000","last_clean":"2024-09-21T05:41:34.699252+0000","last_became_active":"2024-09-21T05:41:32.637282+0000","last_became_peered":"2024-09-21T05:41:32.637282+0000","last_unstale":"2024-09-21T05:41:34.699252+0000","last_undegraded":"2024-09-21T05:41:34.699252+0000","last_fullsized":"2024-09-21T05:41:34.699252+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:55.655727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698429+0000","last_change":"2024-09-21T05:41:33.633455+0000","last_active":"2024-09-21T05:41:34.698429+0000","last_peered":"2024-09-21T05:41:34.698429+0000","last_clean":"2024-09-21T05:41:34.698429+0000","last_became_active":"2024-09-21T05:41:33.633107+0000","last_became_peered":"2024-09-21T05:41:33.633107+0000","last_unstale":"2024-09-21T05:41:34.698429+0000","last_undegraded":"2024-09-21T05:41:34.698429+0000","last_fullsized":"2024-09-21T05:41:34.698429+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:12.222527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640142+0000","last_change":"2024-09-21T05:41:32.631769+0000","last_active":"2024-09-21T05:41:34.640142+0000","last_peered":"2024-09-21T05:41:34.640142+0000","last_clean":"2024-09-21T05:41:34.640142+0000","last_became_active":"2024-09-21T05:41:32.631219+0000","last_became_peered":"2024-09-21T05:41:32.631219+0000","last_unstale":"2024-09-21T05:41:34.640142+0000","last_undegraded":"2024-09-21T05:41:34.640142+0000","last_fullsized":"2024-09-21T05:41:34.640142+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:58.341553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.641402+0000","last_change":"2024-09-21T05:41:33.635965+0000","last_active":"2024-09-21T05:41:34.641402+0000","last_peered":"2024-09-21T05:41:34.641402+0000","last_clean":"2024-09-21T05:41:34.641402+0000","last_became_active":"2024-09-21T05:41:33.635600+0000","last_became_peered":"2024-09-21T05:41:33.635600+0000","last_unstale":"2024-09-21T05:41:34.641402+0000","last_undegraded":"2024-09-21T05:41:34.641402+0000","last_fullsized":"2024-09-21T05:41:34.641402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:40.038034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698751+0000","last_change":"2024-09-21T05:41:32.632401+0000","last_active":"2024-09-21T05:41:34.698751+0000","last_peered":"2024-09-21T05:41:34.698751+0000","last_clean":"2024-09-21T05:41:34.698751+0000","last_became_active":"2024-09-21T05:41:32.631945+0000","last_became_peered":"2024-09-21T05:41:32.631945+0000","last_unstale":"2024-09-21T05:41:34.698751+0000","last_undegraded":"2024-09-21T05:41:34.698751+0000","last_fullsized":"2024-09-21T05:41:34.698751+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:43.733261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698859+0000","last_change":"2024-09-21T05:41:33.646836+0000","last_active":"2024-09-21T05:41:34.698859+0000","last_peered":"2024-09-21T05:41:34.698859+0000","last_clean":"2024-09-21T05:41:34.698859+0000","last_became_active":"2024-09-21T05:41:33.646531+0000","last_became_peered":"2024-09-21T05:41:33.646531+0000","last_unstale":"2024-09-21T05:41:34.698859+0000","last_undegraded":"2024-09-21T05:41:34.698859+0000","last_fullsized":"2024-09-21T05:41:34.698859+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:34.908765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"56'8","reported_seq":23,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698430+0000","last_change":"2024-09-21T05:41:32.639411+0000","last_active":"2024-09-21T05:41:34.698430+0000","last_peered":"2024-09-21T05:41:34.698430+0000","last_clean":"2024-09-21T05:41:34.698430+0000","last_became_active":"2024-09-21T05:41:32.639061+0000","last_became_peered":"2024-09-21T05:41:32.639061+0000","last_unstale":"2024-09-21T05:41:34.698430+0000","last_undegraded":"2024-09-21T05:41:34.698430+0000","last_fullsized":"2024-09-21T05:41:34.698430+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:42.275615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698253+0000","last_change":"2024-09-21T05:41:33.632608+0000","last_active":"2024-09-21T05:41:34.698253+0000","last_peered":"2024-09-21T05:41:34.698253+0000","last_clean":"2024-09-21T05:41:34.698253+0000","last_became_active":"2024-09-21T05:41:33.632279+0000","last_became_peered":"2024-09-21T05:41:33.632279+0000","last_unstale":"2024-09-21T05:41:34.698253+0000","last_undegraded":"2024-09-21T05:41:34.698253+0000","last_fullsized":"2024-09-21T05:41:34.698253+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:12.231743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698442+0000","last_change":"2024-09-21T05:41:32.633543+0000","last_active":"2024-09-21T05:41:34.698442+0000","last_peered":"2024-09-21T05:41:34.698442+0000","last_clean":"2024-09-21T05:41:34.698442+0000","last_became_active":"2024-09-21T05:41:32.633229+0000","last_became_peered":"2024-09-21T05:41:32.633229+0000","last_unstale":"2024-09-21T05:41:34.698442+0000","last_undegraded":"2024-09-21T05:41:34.698442+0000","last_fullsized":"2024-09-21T05:41:34.698442+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:04.045009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698414+0000","last_change":"2024-09-21T05:41:33.635421+0000","last_active":"2024-09-21T05:41:34.698414+0000","last_peered":"2024-09-21T05:41:34.698414+0000","last_clean":"2024-09-21T05:41:34.698414+0000","last_became_active":"2024-09-21T05:41:33.635100+0000","last_became_peered":"2024-09-21T05:41:33.635100+0000","last_unstale":"2024-09-21T05:41:34.698414+0000","last_undegraded":"2024-09-21T05:41:34.698414+0000","last_fullsized":"2024-09-21T05:41:34.698414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:23.846337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698332+0000","last_change":"2024-09-21T05:41:32.635937+0000","last_active":"2024-09-21T05:41:34.698332+0000","last_peered":"2024-09-21T05:41:34.698332+0000","last_clean":"2024-09-21T05:41:34.698332+0000","last_became_active":"2024-09-21T05:41:32.635590+0000","last_became_peered":"2024-09-21T05:41:32.635590+0000","last_unstale":"2024-09-21T05:41:34.698332+0000","last_undegraded":"2024-09-21T05:41:34.698332+0000","last_fullsized":"2024-09-21T05:41:34.698332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:04.417273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698987+0000","last_change":"2024-09-21T05:41:33.638288+0000","last_active":"2024-09-21T05:41:34.698987+0000","last_peered":"2024-09-21T05:41:34.698987+0000","last_clean":"2024-09-21T05:41:34.698987+0000","last_became_active":"2024-09-21T05:41:33.637173+0000","last_became_peered":"2024-09-21T05:41:33.637173+0000","last_unstale":"2024-09-21T05:41:34.698987+0000","last_undegraded":"2024-09-21T05:41:34.698987+0000","last_fullsized":"2024-09-21T05:41:34.698987+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:39.314026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"56'1","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699057+0000","last_change":"2024-09-21T05:41:32.631706+0000","last_active":"2024-09-21T05:41:34.699057+0000","last_peered":"2024-09-21T05:41:34.699057+0000","last_clean":"2024-09-21T05:41:34.699057+0000","last_became_active":"2024-09-21T05:41:32.631345+0000","last_became_peered":"2024-09-21T05:41:32.631345+0000","last_unstale":"2024-09-21T05:41:34.699057+0000","last_undegraded":"2024-09-21T05:41:34.699057+0000","last_fullsized":"2024-09-21T05:41:34.699057+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:01:45.182503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698752+0000","last_change":"2024-09-21T05:41:33.649048+0000","last_active":"2024-09-21T05:41:34.698752+0000","last_peered":"2024-09-21T05:41:34.698752+0000","last_clean":"2024-09-21T05:41:34.698752+0000","last_became_active":"2024-09-21T05:41:33.648505+0000","last_became_peered":"2024-09-21T05:41:33.648505+0000","last_unstale":"2024-09-21T05:41:34.698752+0000","last_undegraded":"2024-09-21T05:41:34.698752+0000","last_fullsized":"2024-09-21T05:41:34.698752+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:29.132079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697492+0000","last_change":"2024-09-21T05:41:32.635058+0000","last_active":"2024-09-21T05:41:34.697492+0000","last_peered":"2024-09-21T05:41:34.697492+0000","last_clean":"2024-09-21T05:41:34.697492+0000","last_became_active":"2024-09-21T05:41:32.634748+0000","last_became_peered":"2024-09-21T05:41:32.634748+0000","last_unstale":"2024-09-21T05:41:34.697492+0000","last_undegraded":"2024-09-21T05:41:34.697492+0000","last_fullsized":"2024-09-21T05:41:34.697492+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:20.216493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699144+0000","last_change":"2024-09-21T05:41:33.630458+0000","last_active":"2024-09-21T05:41:34.699144+0000","last_peered":"2024-09-21T05:41:34.699144+0000","last_clean":"2024-09-21T05:41:34.699144+0000","last_became_active":"2024-09-21T05:41:33.630058+0000","last_became_peered":"2024-09-21T05:41:33.630058+0000","last_unstale":"2024-09-21T05:41:34.699144+0000","last_undegraded":"2024-09-21T05:41:34.699144+0000","last_fullsized":"2024-09-21T05:41:34.699144+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.444704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699426+0000","last_change":"2024-09-21T05:41:32.639401+0000","last_active":"2024-09-21T05:41:34.699426+0000","last_peered":"2024-09-21T05:41:34.699426+0000","last_clean":"2024-09-21T05:41:34.699426+0000","last_became_active":"2024-09-21T05:41:32.639111+0000","last_became_peered":"2024-09-21T05:41:32.639111+0000","last_unstale":"2024-09-21T05:41:34.699426+0000","last_undegraded":"2024-09-21T05:41:34.699426+0000","last_fullsized":"2024-09-21T05:41:34.699426+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:25.720779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698813+0000","last_change":"2024-09-21T05:41:33.642434+0000","last_active":"2024-09-21T05:41:34.698813+0000","last_peered":"2024-09-21T05:41:34.698813+0000","last_clean":"2024-09-21T05:41:34.698813+0000","last_became_active":"2024-09-21T05:41:33.642094+0000","last_became_peered":"2024-09-21T05:41:33.642094+0000","last_unstale":"2024-09-21T05:41:34.698813+0000","last_undegraded":"2024-09-21T05:41:34.698813+0000","last_fullsized":"2024-09-21T05:41:34.698813+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:23.189588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639233+0000","last_change":"2024-09-21T05:41:32.631531+0000","last_active":"2024-09-21T05:41:34.639233+0000","last_peered":"2024-09-21T05:41:34.639233+0000","last_clean":"2024-09-21T05:41:34.639233+0000","last_became_active":"2024-09-21T05:41:32.631025+0000","last_became_peered":"2024-09-21T05:41:32.631025+0000","last_unstale":"2024-09-21T05:41:34.639233+0000","last_undegraded":"2024-09-21T05:41:34.639233+0000","last_fullsized":"2024-09-21T05:41:34.639233+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:23.285869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698031+0000","last_change":"2024-09-21T05:41:33.623783+0000","last_active":"2024-09-21T05:41:34.698031+0000","last_peered":"2024-09-21T05:41:34.698031+0000","last_clean":"2024-09-21T05:41:34.698031+0000","last_became_active":"2024-09-21T05:41:33.623439+0000","last_became_peered":"2024-09-21T05:41:33.623439+0000","last_unstale":"2024-09-21T05:41:34.698031+0000","last_undegraded":"2024-09-21T05:41:34.698031+0000","last_fullsized":"2024-09-21T05:41:34.698031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:26.723073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.697836+0000","last_change":"2024-09-21T05:41:32.631171+0000","last_active":"2024-09-21T05:41:34.697836+0000","last_peered":"2024-09-21T05:41:34.697836+0000","last_clean":"2024-09-21T05:41:34.697836+0000","last_became_active":"2024-09-21T05:41:32.630876+0000","last_became_peered":"2024-09-21T05:41:32.630876+0000","last_unstale":"2024-09-21T05:41:34.697836+0000","last_undegraded":"2024-09-21T05:41:34.697836+0000","last_fullsized":"2024-09-21T05:41:34.697836+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:43.095261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698137+0000","last_change":"2024-09-21T05:41:33.636896+0000","last_active":"2024-09-21T05:41:34.698137+0000","last_peered":"2024-09-21T05:41:34.698137+0000","last_clean":"2024-09-21T05:41:34.698137+0000","last_became_active":"2024-09-21T05:41:33.634526+0000","last_became_peered":"2024-09-21T05:41:33.634526+0000","last_unstale":"2024-09-21T05:41:34.698137+0000","last_undegraded":"2024-09-21T05:41:34.698137+0000","last_fullsized":"2024-09-21T05:41:34.698137+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:02:21.467807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"56'2","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639825+0000","last_change":"2024-09-21T05:41:32.633802+0000","last_active":"2024-09-21T05:41:34.639825+0000","last_peered":"2024-09-21T05:41:34.639825+0000","last_clean":"2024-09-21T05:41:34.639825+0000","last_became_active":"2024-09-21T05:41:32.633466+0000","last_became_peered":"2024-09-21T05:41:32.633466+0000","last_unstale":"2024-09-21T05:41:34.639825+0000","last_undegraded":"2024-09-21T05:41:34.639825+0000","last_fullsized":"2024-09-21T05:41:34.639825+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:24.054877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698471+0000","last_change":"2024-09-21T05:41:33.638144+0000","last_active":"2024-09-21T05:41:34.698471+0000","last_peered":"2024-09-21T05:41:34.698471+0000","last_clean":"2024-09-21T05:41:34.698471+0000","last_became_active":"2024-09-21T05:41:33.637806+0000","last_became_peered":"2024-09-21T05:41:33.637806+0000","last_unstale":"2024-09-21T05:41:34.698471+0000","last_undegraded":"2024-09-21T05:41:34.698471+0000","last_fullsized":"2024-09-21T05:41:34.698471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:14.222429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698465+0000","last_change":"2024-09-21T05:41:32.636124+0000","last_active":"2024-09-21T05:41:34.698465+0000","last_peered":"2024-09-21T05:41:34.698465+0000","last_clean":"2024-09-21T05:41:34.698465+0000","last_became_active":"2024-09-21T05:41:32.635756+0000","last_became_peered":"2024-09-21T05:41:32.635756+0000","last_unstale":"2024-09-21T05:41:34.698465+0000","last_undegraded":"2024-09-21T05:41:34.698465+0000","last_fullsized":"2024-09-21T05:41:34.698465+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:22.430482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639447+0000","last_change":"2024-09-21T05:41:33.634983+0000","last_active":"2024-09-21T05:41:34.639447+0000","last_peered":"2024-09-21T05:41:34.639447+0000","last_clean":"2024-09-21T05:41:34.639447+0000","last_became_active":"2024-09-21T05:41:33.634650+0000","last_became_peered":"2024-09-21T05:41:33.634650+0000","last_unstale":"2024-09-21T05:41:34.639447+0000","last_undegraded":"2024-09-21T05:41:34.639447+0000","last_fullsized":"2024-09-21T05:41:34.639447+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:35.920327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640744+0000","last_change":"2024-09-21T05:41:32.628780+0000","last_active":"2024-09-21T05:41:34.640744+0000","last_peered":"2024-09-21T05:41:34.640744+0000","last_clean":"2024-09-21T05:41:34.640744+0000","last_became_active":"2024-09-21T05:41:32.627444+0000","last_became_peered":"2024-09-21T05:41:32.627444+0000","last_unstale":"2024-09-21T05:41:34.640744+0000","last_undegraded":"2024-09-21T05:41:34.640744+0000","last_fullsized":"2024-09-21T05:41:34.640744+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:36.131830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698403+0000","last_change":"2024-09-21T05:41:33.629373+0000","last_active":"2024-09-21T05:41:34.698403+0000","last_peered":"2024-09-21T05:41:34.698403+0000","last_clean":"2024-09-21T05:41:34.698403+0000","last_became_active":"2024-09-21T05:41:33.629063+0000","last_became_peered":"2024-09-21T05:41:33.629063+0000","last_unstale":"2024-09-21T05:41:34.698403+0000","last_undegraded":"2024-09-21T05:41:34.698403+0000","last_fullsized":"2024-09-21T05:41:34.698403+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:13.577303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698628+0000","last_change":"2024-09-21T05:41:32.639613+0000","last_active":"2024-09-21T05:41:34.698628+0000","last_peered":"2024-09-21T05:41:34.698628+0000","last_clean":"2024-09-21T05:41:34.698628+0000","last_became_active":"2024-09-21T05:41:32.639243+0000","last_became_peered":"2024-09-21T05:41:32.639243+0000","last_unstale":"2024-09-21T05:41:34.698628+0000","last_undegraded":"2024-09-21T05:41:34.698628+0000","last_fullsized":"2024-09-21T05:41:34.698628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:01.439766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698472+0000","last_change":"2024-09-21T05:41:33.639642+0000","last_active":"2024-09-21T05:41:34.698472+0000","last_peered":"2024-09-21T05:41:34.698472+0000","last_clean":"2024-09-21T05:41:34.698472+0000","last_became_active":"2024-09-21T05:41:33.639328+0000","last_became_peered":"2024-09-21T05:41:33.639328+0000","last_unstale":"2024-09-21T05:41:34.698472+0000","last_undegraded":"2024-09-21T05:41:34.698472+0000","last_fullsized":"2024-09-21T05:41:34.698472+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:04.614594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639124+0000","last_change":"2024-09-21T05:41:32.627712+0000","last_active":"2024-09-21T05:41:34.639124+0000","last_peered":"2024-09-21T05:41:34.639124+0000","last_clean":"2024-09-21T05:41:34.639124+0000","last_became_active":"2024-09-21T05:41:32.627390+0000","last_became_peered":"2024-09-21T05:41:32.627390+0000","last_unstale":"2024-09-21T05:41:34.639124+0000","last_undegraded":"2024-09-21T05:41:34.639124+0000","last_fullsized":"2024-09-21T05:41:34.639124+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:31.304519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698689+0000","last_change":"2024-09-21T05:41:33.646806+0000","last_active":"2024-09-21T05:41:34.698689+0000","last_peered":"2024-09-21T05:41:34.698689+0000","last_clean":"2024-09-21T05:41:34.698689+0000","last_became_active":"2024-09-21T05:41:33.646446+0000","last_became_peered":"2024-09-21T05:41:33.646446+0000","last_unstale":"2024-09-21T05:41:34.698689+0000","last_undegraded":"2024-09-21T05:41:34.698689+0000","last_fullsized":"2024-09-21T05:41:34.698689+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:46.544965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640988+0000","last_change":"2024-09-21T05:41:32.637243+0000","last_active":"2024-09-21T05:41:34.640988+0000","last_peered":"2024-09-21T05:41:34.640988+0000","last_clean":"2024-09-21T05:41:34.640988+0000","last_became_active":"2024-09-21T05:41:32.636940+0000","last_became_peered":"2024-09-21T05:41:32.636940+0000","last_unstale":"2024-09-21T05:41:34.640988+0000","last_undegraded":"2024-09-21T05:41:34.640988+0000","last_fullsized":"2024-09-21T05:41:34.640988+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:42.275615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639147+0000","last_change":"2024-09-21T05:41:33.635249+0000","last_active":"2024-09-21T05:41:34.639147+0000","last_peered":"2024-09-21T05:41:34.639147+0000","last_clean":"2024-09-21T05:41:34.639147+0000","last_became_active":"2024-09-21T05:41:33.634871+0000","last_became_peered":"2024-09-21T05:41:33.634871+0000","last_unstale":"2024-09-21T05:41:34.639147+0000","last_undegraded":"2024-09-21T05:41:34.639147+0000","last_fullsized":"2024-09-21T05:41:34.639147+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:08.079028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.639113+0000","last_change":"2024-09-21T05:41:32.638968+0000","last_active":"2024-09-21T05:41:34.639113+0000","last_peered":"2024-09-21T05:41:34.639113+0000","last_clean":"2024-09-21T05:41:34.639113+0000","last_became_active":"2024-09-21T05:41:32.638616+0000","last_became_peered":"2024-09-21T05:41:32.638616+0000","last_unstale":"2024-09-21T05:41:34.639113+0000","last_undegraded":"2024-09-21T05:41:34.639113+0000","last_fullsized":"2024-09-21T05:41:34.639113+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:11.983425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.699075+0000","last_change":"2024-09-21T05:41:33.644686+0000","last_active":"2024-09-21T05:41:34.699075+0000","last_peered":"2024-09-21T05:41:34.699075+0000","last_clean":"2024-09-21T05:41:34.699075+0000","last_became_active":"2024-09-21T05:41:33.638504+0000","last_became_peered":"2024-09-21T05:41:33.638504+0000","last_unstale":"2024-09-21T05:41:34.699075+0000","last_undegraded":"2024-09-21T05:41:34.699075+0000","last_fullsized":"2024-09-21T05:41:34.699075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:31.775199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698266+0000","last_change":"2024-09-21T05:41:32.631328+0000","last_active":"2024-09-21T05:41:34.698266+0000","last_peered":"2024-09-21T05:41:34.698266+0000","last_clean":"2024-09-21T05:41:34.698266+0000","last_became_active":"2024-09-21T05:41:32.630972+0000","last_became_peered":"2024-09-21T05:41:32.630972+0000","last_unstale":"2024-09-21T05:41:34.698266+0000","last_undegraded":"2024-09-21T05:41:34.698266+0000","last_fullsized":"2024-09-21T05:41:34.698266+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:43:20.626181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698142+0000","last_change":"2024-09-21T05:41:33.646541+0000","last_active":"2024-09-21T05:41:34.698142+0000","last_peered":"2024-09-21T05:41:34.698142+0000","last_clean":"2024-09-21T05:41:34.698142+0000","last_became_active":"2024-09-21T05:41:33.646222+0000","last_became_peered":"2024-09-21T05:41:33.646222+0000","last_unstale":"2024-09-21T05:41:34.698142+0000","last_undegraded":"2024-09-21T05:41:34.698142+0000","last_fullsized":"2024-09-21T05:41:34.698142+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:32.607883+0000","last_clean_scrub_stamp":"2024-09-21T05:41:32.607883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:37.177614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.640164+0000","last_change":"2024-09-21T05:41:32.627052+0000","last_active":"2024-09-21T05:41:34.640164+0000","last_peered":"2024-09-21T05:41:34.640164+0000","last_clean":"2024-09-21T05:41:34.640164+0000","last_became_active":"2024-09-21T05:41:32.626729+0000","last_became_peered":"2024-09-21T05:41:32.626729+0000","last_unstale":"2024-09-21T05:41:34.640164+0000","last_undegraded":"2024-09-21T05:41:34.640164+0000","last_fullsized":"2024-09-21T05:41:34.640164+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:41:31.605589+0000","last_clean_scrub_stamp":"2024-09-21T05:41:31.605589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:27.112139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":174,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-21T05:41:34.698831+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:41:34.698831+0000","last_peered":"2024-09-21T05:41:34.698831+0000","last_clean":"2024-09-21T05:41:34.698831+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:41:34.698831+0000","last_undegraded":"2024-09-21T05:41:34.698831+0000","last_fullsized":"2024-09-21T05:41:34.698831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":871106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":282190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542176,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":675840,"data_stored":280578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":684032,"data_stored":281230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":280634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":684032,"data_stored":280634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36016,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335824,"statfs":{"total":96636764160,"available":96599883776,"internally_reserved":0,"allocated":679936,"data_stored":280668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36612,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335228,"statfs":{"total":96636764160,"available":96599273472,"internally_reserved":0,"allocated":1286144,"data_stored":873140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:41:38.075 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:41:38.075 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:41:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 -- 172.21.15.22:0/3026779317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 msgr2=0x7f6c400b9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3026779317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c400b9900 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009910 tx=0x7f6c3002a5f0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 -- 172.21.15.22:0/3026779317 shutdown_connections 2024-09-21T05:41:38.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3026779317 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c400b3910 0x7f6c400b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3026779317 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c400b9900 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 -- 172.21.15.22:0/3026779317 >> 172.21.15.22:0/3026779317 conn(0x7f6c4009d830 msgr2=0x7f6c400d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:38.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 -- 172.21.15.22:0/3026779317 shutdown_connections 2024-09-21T05:41:38.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.234+0000 7f6c460ce640 1 -- 172.21.15.22:0/3026779317 wait complete. 2024-09-21T05:41:38.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.236+0000 7f6c460ce640 1 Processor -- start 2024-09-21T05:41:38.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.236+0000 7f6c460ce640 1 -- start start 2024-09-21T05:41:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c460ce640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c400b3910 0x7f6c401f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c460ce640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c400b64e0 0x7f6c401f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c460ce640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c401f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c460ce640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400cf610 con 0x7f6c400d4c30 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c460ce640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400cf490 con 0x7f6c400b3910 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c460ce640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c400cf790 con 0x7f6c400b64e0 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c401f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c400b3910 0x7f6c401f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c401f94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45464/0 (socket says 172.21.15.22:45464) 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c3effd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c400b64e0 0x7f6c401f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.237+0000 7f6c3ffff640 1 -- 172.21.15.22:0/3797248554 learned_addr learned my addr 172.21.15.22:0/3797248554 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c3ffff640 1 -- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c400b64e0 msgr2=0x7f6c401f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c3ffff640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c400b64e0 0x7f6c401f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c3ffff640 1 -- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c400b3910 msgr2=0x7f6c401f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c3ffff640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c400b3910 0x7f6c401f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c3ffff640 1 -- 172.21.15.22:0/3797248554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4015d1b0 con 0x7f6c400d4c30 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c3ffff640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c401f94c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6c38002230 tx=0x7f6c3800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:38.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38003b70 con 0x7f6c400d4c30 2024-09-21T05:41:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c38003d10 con 0x7f6c400d4c30 2024-09-21T05:41:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c400cc080 con 0x7f6c400d4c30 2024-09-21T05:41:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38005330 con 0x7f6c400d4c30 2024-09-21T05:41:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.238+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4015bc10 con 0x7f6c400d4c30 2024-09-21T05:41:38.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.239+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c400cc5a0 con 0x7f6c400d4c30 2024-09-21T05:41:38.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.240+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c380054d0 con 0x7f6c400d4c30 2024-09-21T05:41:38.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.240+0000 7f6c27fff640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c2c078b90 0x7f6c2c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.240+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f6c3809aa50 con 0x7f6c400d4c30 2024-09-21T05:41:38.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.240+0000 7f6c3f7fe640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c2c078b90 0x7f6c2c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.241+0000 7f6c3f7fe640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c2c078b90 0x7f6c2c07b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c30034000 tx=0x7f6c30004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:38.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.245+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c400cc5a0 con 0x7f6c400d4c30 2024-09-21T05:41:38.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.505+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6c4015a370 con 0x7f6c400d4c30 2024-09-21T05:41:38.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.508+0000 7f6c27fff640 1 -- 172.21.15.22:0/3797248554 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f6c4015d1b0 con 0x7f6c400d4c30 2024-09-21T05:41:38.510 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:41:38.510 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:41:38.510 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:41:38.510 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:41:38.510 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:41:38.511 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:41:38.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c2c078b90 msgr2=0x7f6c2c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c2c078b90 0x7f6c2c07b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6c30034000 tx=0x7f6c30004730 comp rx=0 tx=0).stop 2024-09-21T05:41:38.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 msgr2=0x7f6c401f94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c401f94c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6c38002230 tx=0x7f6c3800cb90 comp rx=0 tx=0).stop 2024-09-21T05:41:38.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 shutdown_connections 2024-09-21T05:41:38.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c400d4c30 0x7f6c401f94c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c2c078b90 0x7f6c2c07b0a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c400b64e0 0x7f6c401f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 --2- 172.21.15.22:0/3797248554 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c400b3910 0x7f6c401f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 >> 172.21.15.22:0/3797248554 conn(0x7f6c4009d830 msgr2=0x7f6c400b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:38.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 shutdown_connections 2024-09-21T05:41:38.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.511+0000 7f6c460ce640 1 -- 172.21.15.22:0/3797248554 wait complete. 2024-09-21T05:41:38.525 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 -- 172.21.15.22:0/13714253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 msgr2=0x7f97840cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 --2- 172.21.15.22:0/13714253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97840cebf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f976c009830 tx=0x7f976c02a510 comp rx=0 tx=0).stop 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 -- 172.21.15.22:0/13714253 shutdown_connections 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 --2- 172.21.15.22:0/13714253 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97840cebf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 --2- 172.21.15.22:0/13714253 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97840b51c0 0x7f978415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 -- 172.21.15.22:0/13714253 >> 172.21.15.22:0/13714253 conn(0x7f9784072ae0 msgr2=0x7f9784072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 -- 172.21.15.22:0/13714253 shutdown_connections 2024-09-21T05:41:38.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.638+0000 7f978b120640 1 -- 172.21.15.22:0/13714253 wait complete. 2024-09-21T05:41:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.640+0000 7f978b120640 1 Processor -- start 2024-09-21T05:41:38.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.640+0000 7f978b120640 1 -- start start 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.641+0000 7f978b120640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97840b51c0 0x7f97840ab170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.641+0000 7f978b120640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97840ccbd0 0x7f97840ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.641+0000 7f978b120640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97841f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.641+0000 7f978b120640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f97840cf2b0 con 0x7f97840d1490 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.641+0000 7f978b120640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f97840cf130 con 0x7f97840ccbd0 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.641+0000 7f978b120640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f97840cf430 con 0x7f97840b51c0 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97841f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97841f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45490/0 (socket says 172.21.15.22:45490) 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 -- 172.21.15.22:0/2148935316 learned_addr learned my addr 172.21.15.22:0/2148935316 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f9788ebc640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97840b51c0 0x7f97840ab170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f9783fff640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97840ccbd0 0x7f97840ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 -- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97840b51c0 msgr2=0x7f97840ab170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97840b51c0 0x7f97840ab170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 -- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97840ccbd0 msgr2=0x7f97840ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97840ccbd0 0x7f97840ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 -- 172.21.15.22:0/2148935316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97840abf30 con 0x7f97840d1490 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97896bd640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97841f6f90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f976c009950 tx=0x7f976c033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c031d60 con 0x7f97840d1490 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976c005020 con 0x7f97840d1490 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97840ac210 con 0x7f97840d1490 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c005450 con 0x7f97840d1490 2024-09-21T05:41:38.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.642+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97840ac7d0 con 0x7f97840d1490 2024-09-21T05:41:38.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.643+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97840abca0 con 0x7f97840d1490 2024-09-21T05:41:38.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.644+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f976c0047a0 con 0x7f97840d1490 2024-09-21T05:41:38.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.645+0000 7f97817fa640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9764078b90 0x7f976407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:38.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.645+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f976c0bbee0 con 0x7f97840d1490 2024-09-21T05:41:38.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.645+0000 7f9788ebc640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9764078b90 0x7f976407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:38.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.646+0000 7f9788ebc640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9764078b90 0x7f976407b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f977c00a490 tx=0x7f977c00a420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:38.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.649+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f976c03f030 con 0x7f97840d1490 2024-09-21T05:41:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.874+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9784068d00 con 0x7f97840d1490 2024-09-21T05:41:38.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.874+0000 7f97817fa640 1 -- 172.21.15.22:0/2148935316 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5226 (secure 0 0 0) 0x7f9784068d00 con 0x7f97840d1490 2024-09-21T05:41:38.877 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:38.877 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"btime":"2024-09-21T05:41:36:644246+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:41:36.026286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:41:38.877 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 34 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9764078b90 msgr2=0x7f976407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9764078b90 0x7f976407b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f977c00a490 tx=0x7f977c00a420 comp rx=0 tx=0).stop 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 msgr2=0x7f97841f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97841f6f90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f976c009950 tx=0x7f976c033ce0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 shutdown_connections 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f97840d1490 0x7f97841f6f90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f97840ccbd0 0x7f97840ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9764078b90 0x7f976407b0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 --2- 172.21.15.22:0/2148935316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f97840b51c0 0x7f97840ab170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:38.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 >> 172.21.15.22:0/2148935316 conn(0x7f9784072ae0 msgr2=0x7f978415b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:38.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 shutdown_connections 2024-09-21T05:41:38.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:38.877+0000 7f978b120640 1 -- 172.21.15.22:0/2148935316 wait complete. 2024-09-21T05:41:38.922 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:41:38.920+0000 7f13b0349640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:41:39.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:41:33.013872+0000', 'modified': '2024-09-21T05:41:36.026286+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5679': {'gid': 5679, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6835/913296056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 913296056}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 913296056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5679, 'qdb_cluster': [5679]} 2024-09-21T05:41:39.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:41:39.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.412+0000 7f1279920640 1 -- 172.21.15.22:0/3734200523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 msgr2=0x7f12740d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.412+0000 7f1279920640 1 --2- 172.21.15.22:0/3734200523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f12740d4d80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f12740d20f0 tx=0x7f126402a860 comp rx=0 tx=0).stop 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 -- 172.21.15.22:0/3734200523 shutdown_connections 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 --2- 172.21.15.22:0/3734200523 >> v2:172.21.15.134:3300/0 conn(0x7f12740cf540 0x7f12740b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 --2- 172.21.15.22:0/3734200523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740b64e0 0x7f12740d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 --2- 172.21.15.22:0/3734200523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f12740d4d80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 -- 172.21.15.22:0/3734200523 >> 172.21.15.22:0/3734200523 conn(0x7f127409d7f0 msgr2=0x7f12740d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 -- 172.21.15.22:0/3734200523 shutdown_connections 2024-09-21T05:41:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.413+0000 7f1279920640 1 -- 172.21.15.22:0/3734200523 wait complete. 2024-09-21T05:41:39.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.414+0000 7f1279920640 1 Processor -- start 2024-09-21T05:41:39.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.415+0000 7f1279920640 1 -- start start 2024-09-21T05:41:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1279920640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12740b64e0 0x7f12741f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1279920640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f127409e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1279920640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740cf540 0x7f12740a0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1279920640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12740d3020 con 0x7f12740b9b60 2024-09-21T05:41:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1279920640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12740d2ea0 con 0x7f12740b64e0 2024-09-21T05:41:39.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1279920640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12740d31a0 con 0x7f12740cf540 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f127409e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f127409e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45498/0 (socket says 172.21.15.22:45498) 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 -- 172.21.15.22:0/2646969954 learned_addr learned my addr 172.21.15.22:0/2646969954 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f1272ffd640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12740b64e0 0x7f12741f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12737fe640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740cf540 0x7f12740a0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 -- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740cf540 msgr2=0x7f12740a0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740cf540 0x7f12740a0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 -- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12740b64e0 msgr2=0x7f12741f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12740b64e0 0x7f12741f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.416+0000 7f12727fc640 1 -- 172.21.15.22:0/2646969954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12741f9990 con 0x7f12740b9b60 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.417+0000 7f12727fc640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f127409e580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f126800d6a0 tx=0x7f126800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.417+0000 7f125bfff640 1 -- 172.21.15.22:0/2646969954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268017070 con 0x7f12740b9b60 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.417+0000 7f125bfff640 1 -- 172.21.15.22:0/2646969954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1268009b10 con 0x7f12740b9b60 2024-09-21T05:41:39.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.417+0000 7f125bfff640 1 -- 172.21.15.22:0/2646969954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268010460 con 0x7f12740b9b60 2024-09-21T05:41:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.417+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12741f9c70 con 0x7f12740b9b60 2024-09-21T05:41:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.417+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12740a1630 con 0x7f12740b9b60 2024-09-21T05:41:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.419+0000 7f125bfff640 1 -- 172.21.15.22:0/2646969954 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f126801a070 con 0x7f12740b9b60 2024-09-21T05:41:39.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.419+0000 7f125bfff640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1260078bc0 0x7f126007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.419+0000 7f125bfff640 1 -- 172.21.15.22:0/2646969954 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f126809ae80 con 0x7f12740b9b60 2024-09-21T05:41:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.420+0000 7f1272ffd640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1260078bc0 0x7f126007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.420+0000 7f1280893740 1 Processor -- start 2024-09-21T05:41:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.420+0000 7f1280893740 1 -- start start 2024-09-21T05:41:39.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.420+0000 7f1272ffd640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1260078bc0 0x7f126007b0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1264035000 tx=0x7f1264004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1280893740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 0x55923703afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x55923703b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370583e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1280893740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559236f83820 con 0x559236c69270 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559236f72c50 con 0x559236c48590 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559236f76a60 con 0x559236c403b0 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 0x55923703afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 0x55923703afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45506/0 (socket says 172.21.15.22:45506) 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 -- 172.21.15.22:0/3999528864 learned_addr learned my addr 172.21.15.22:0/3999528864 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:39.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f12727fc640 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x55923703b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f12737fe640 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 -- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 msgr2=0x55923703b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x55923703b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 -- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 msgr2=0x5592370583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 -- 172.21.15.22:0/3999528864 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559236cd4850 con 0x559236c69270 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f1272ffd640 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 0x55923703afa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1264005fd0 tx=0x7f126403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e8ba5240a1e3c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f12597fa640 1 -- 172.21.15.22:0/3999528864 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1264003cf0 con 0x559236c69270 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.422+0000 7f12597fa640 1 -- 172.21.15.22:0/3999528864 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1264036040 con 0x559236c69270 2024-09-21T05:41:39.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f12597fa640 1 -- 172.21.15.22:0/3999528864 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12640042a0 con 0x559236c69270 2024-09-21T05:41:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 -- 172.21.15.22:0/3999528864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 msgr2=0x55923703afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 0x55923703afa0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1264005fd0 tx=0x7f126403f040 comp rx=0 tx=0).stop 2024-09-21T05:41:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 -- 172.21.15.22:0/3999528864 shutdown_connections 2024-09-21T05:41:39.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x55923703b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 --2- 172.21.15.22:0/3999528864 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c69270 0x55923703afa0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 -- 172.21.15.22:0/3999528864 >> 172.21.15.22:0/3999528864 conn(0x559236c5d3c0 msgr2=0x559237011260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 -- 172.21.15.22:0/3999528864 shutdown_connections 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.423+0000 7f1280893740 1 -- 172.21.15.22:0/3999528864 wait complete. 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.424+0000 7f1280893740 1 -- start start 2024-09-21T05:41:39.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.424+0000 7f1280893740 1 Processor -- start 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x559237065710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 0x559237037e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c69270 0x5592370681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559236ce3620 con 0x559236c48590 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559236f185f0 con 0x559236c403b0 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559237066aa0 con 0x559236c69270 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1280893740 10 client.0 _ll_register_callbacks cb 0x559236f1c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 0x559237037e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 0x559237037e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45516/0 (socket says 172.21.15.22:45516) 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 -- 172.21.15.22:0/1908326858 learned_addr learned my addr 172.21.15.22:0/1908326858 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:39.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f1272ffd640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x559237065710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12737fe640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c69270 0x5592370681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 -- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c69270 msgr2=0x5592370681d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c69270 0x5592370681d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 -- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 msgr2=0x559237065710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x559237065710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 -- 172.21.15.22:0/1908326858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559236d4b6b0 con 0x559236c48590 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12737fe640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c69270 0x5592370681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12727fc640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 0x559237037e70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1268009610 tx=0x7f1268062810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a0769f33164fd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12527fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.426+0000 7f12527fc640 1 -- 172.21.15.22:0/1908326858 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268021040 con 0x559236c48590 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f12527fc640 1 -- 172.21.15.22:0/1908326858 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f1268062030 con 0x559236c48590 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1264036040 con 0x559236c48590 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f12527fc640 1 -- 172.21.15.22:0/1908326858 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f12680673f0 con 0x559236c48590 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 10 client.5808 fetch_fsmap learned FSMap version 34 2024-09-21T05:41:39.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x559237064be0 con 0x559236c48590 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f12527fc640 1 -- 172.21.15.22:0/1908326858 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1264036040 con 0x559236c48590 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f12527fc640 1 -- 172.21.15.22:0/1908326858 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7f126805e520 con 0x559236c48590 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 10 client.5808 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 10 client.5808 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 20 client.5808 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236c27e40 0x55923704ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 4 client.5808 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559236f85770 con 0x559236c27e40 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.427+0000 7f1272ffd640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236c27e40 0x55923704ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f12527fc640 10 client.5808 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f1272ffd640 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236c27e40 0x55923704ce90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d8b42542d49cde server_cookie=314e96dbf7eae578 in_seq=0 out_seq=0 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f12527fc640 1 -- 172.21.15.22:0/1908326858 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559236f85770 con 0x559236c27e40 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f12527fc640 10 client.5808 handle_command_reply: tid=0 2024-09-21T05:41:39.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f1280893740 1 client.5808 shutdown 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 msgr2=0x559237037e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f1280893740 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 0x559237037e70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1268009610 tx=0x7f1268062810 comp rx=0 tx=0).stop 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 shutdown_connections 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.428+0000 7f1280893740 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236c27e40 0x55923704ce90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c69270 0x5592370681d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f12527fc640 0 client.5808 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c48590 0x559237037e70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 1 --2- 172.21.15.22:0/1908326858 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x559237065710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 >> 172.21.15.22:0/1908326858 conn(0x559236c5d3c0 msgr2=0x5592370579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 shutdown_connections 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 1 -- 172.21.15.22:0/1908326858 wait complete. 2024-09-21T05:41:39.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.429+0000 7f1280893740 20 client.5808 trim_cache size 0 max 16384 2024-09-21T05:41:39.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.447+0000 7f1280893740 1 Processor -- start 2024-09-21T05:41:39.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.447+0000 7f1280893740 1 -- start start 2024-09-21T05:41:39.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1280893740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x559237078f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x5592370297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c48590 0x559237029d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1280893740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12741f68f0 con 0x559236c27e40 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12741f5c50 con 0x559236c403b0 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12741f62a0 con 0x559236c48590 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x559237078f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x559237078f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45526/0 (socket says 172.21.15.22:45526) 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 -- 172.21.15.22:0/4040092734 learned_addr learned my addr 172.21.15.22:0/4040092734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 -- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c48590 msgr2=0x559237029d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f12727fc640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x5592370297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f12737fe640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c48590 0x559237029d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c48590 0x559237029d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 -- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 msgr2=0x5592370297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x5592370297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 -- 172.21.15.22:0/4040092734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559236ef51e0 con 0x559236c27e40 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f12727fc640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x5592370297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f12737fe640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c48590 0x559237029d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.449+0000 7f1272ffd640 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x559237078f00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f126402ae50 tx=0x7f1264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f571d980355b483 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1258ff9640 1 -- 172.21.15.22:0/4040092734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126403e5b0 con 0x559236c27e40 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1258ff9640 1 -- 172.21.15.22:0/4040092734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f126405f030 con 0x559236c27e40 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1258ff9640 1 -- 172.21.15.22:0/4040092734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126403e9d0 con 0x559236c27e40 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 -- 172.21.15.22:0/4040092734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 msgr2=0x559237078f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x559237078f00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f126402ae50 tx=0x7f1264004a40 comp rx=0 tx=0).stop 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 -- 172.21.15.22:0/4040092734 shutdown_connections 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c48590 0x559237029d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c403b0 0x5592370297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 --2- 172.21.15.22:0/4040092734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x559237078f00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 -- 172.21.15.22:0/4040092734 >> 172.21.15.22:0/4040092734 conn(0x559236c5d3c0 msgr2=0x559237038d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 -- 172.21.15.22:0/4040092734 shutdown_connections 2024-09-21T05:41:39.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.450+0000 7f1280893740 1 -- 172.21.15.22:0/4040092734 wait complete. 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.451+0000 7f1280893740 1 -- start start 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.451+0000 7f1280893740 1 Processor -- start 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x5592370d93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x5592370d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370d5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5592370662d0 con 0x559236c27e40 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559237068760 con 0x559236c48590 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559237039580 con 0x559236c403b0 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1280893740 10 client.0 _ll_register_callbacks cb 0x7f1268062a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x5592370d93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x5592370d93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45540/0 (socket says 172.21.15.22:45540) 2024-09-21T05:41:39.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 -- 172.21.15.22:0/3845390036 learned_addr learned my addr 172.21.15.22:0/3845390036 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f12737fe640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370d5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f12727fc640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x5592370d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 -- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 msgr2=0x5592370d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x5592370d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 -- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 msgr2=0x5592370d5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370d5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 -- 172.21.15.22:0/3845390036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559237078b40 con 0x559236c27e40 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f12737fe640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f1272ffd640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x5592370d93e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1264009be0 tx=0x7f1264030660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e25cb8eae1dde9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.453+0000 7f12527fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12527fc640 1 -- 172.21.15.22:0/3845390036 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126403e870 con 0x559236c27e40 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12527fc640 1 -- 172.21.15.22:0/3845390036 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f126403ea10 con 0x559236c27e40 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f126405f030 con 0x559236c27e40 2024-09-21T05:41:39.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12527fc640 1 -- 172.21.15.22:0/3845390036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1264074050 con 0x559236c27e40 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 10 client.5814 fetch_fsmap learned FSMap version 34 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5592370d5b00 con 0x559236c27e40 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12527fc640 1 -- 172.21.15.22:0/3845390036 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f126405f030 con 0x559236c27e40 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12527fc640 1 -- 172.21.15.22:0/3845390036 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7f1264077020 con 0x559236c27e40 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 10 client.5814 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 10 client.5814 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 20 client.5814 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236bfd280 0x55923700a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 4 client.5814 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "status"}) -- 0x559236f81220 con 0x559236bfd280 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12727fc640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236bfd280 0x55923700a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.454+0000 7f12527fc640 10 client.5814 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:39.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f12727fc640 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236bfd280 0x55923700a420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c4f577608b1eea server_cookie=f6370f6458ceaf3e in_seq=0 out_seq=0 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f12527fc640 1 -- 172.21.15.22:0/3845390036 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x559236f81220 con 0x559236bfd280 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f12527fc640 10 client.5814 handle_command_reply: tid=0 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 client.5814 shutdown 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 msgr2=0x5592370d93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x5592370d93e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1264009be0 tx=0x7f1264030660 comp rx=0 tx=0).stop 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 shutdown_connections 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x559236bfd280 0x55923700a420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559236c48590 0x5592370d5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559236c403b0 0x5592370d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f12527fc640 0 client.5814 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 --2- 172.21.15.22:0/3845390036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559236c27e40 0x5592370d93e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 >> 172.21.15.22:0/3845390036 conn(0x559236c5d3c0 msgr2=0x5592370113b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 shutdown_connections 2024-09-21T05:41:39.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 1 -- 172.21.15.22:0/3845390036 wait complete. 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.455+0000 7f1280893740 20 client.5814 trim_cache size 0 max 16384 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 5679, 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.8400166149999997, 2024-09-21T05:41:39.463 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 34, 2024-09-21T05:41:39.464 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 57, 2024-09-21T05:41:39.464 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 57, 2024-09-21T05:41:39.464 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 11.140918359 2024-09-21T05:41:39.464 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:39.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1260078bc0 msgr2=0x7f126007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1260078bc0 0x7f126007b0d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1264035000 tx=0x7f1264004ab0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 msgr2=0x7f127409e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f127409e580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f126800d6a0 tx=0x7f126800db60 comp rx=0 tx=0).stop 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 shutdown_connections 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1260078bc0 0x7f126007b0d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12740cf540 0x7f12740a0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12740b9b60 0x7f127409e580 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 --2- 172.21.15.22:0/2646969954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12740b64e0 0x7f12741f91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 >> 172.21.15.22:0/2646969954 conn(0x7f127409d7f0 msgr2=0x7f12740b5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 shutdown_connections 2024-09-21T05:41:39.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.456+0000 7f1279920640 1 -- 172.21.15.22:0/2646969954 wait complete. 2024-09-21T05:41:39.469 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:41:39.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13e6069640 1 -- 172.21.15.22:0/2522001477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 msgr2=0x7f13c80051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13e6069640 1 --2- 172.21.15.22:0/2522001477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13c80051f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c610 tx=0x7f13c8005830 comp rx=0 tx=0).stop 2024-09-21T05:41:39.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13dffff640 1 -- 172.21.15.22:0/2522001477 reap_dead start 2024-09-21T05:41:39.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13e6069640 1 -- 172.21.15.22:0/2522001477 shutdown_connections 2024-09-21T05:41:39.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13e6069640 1 -- 172.21.15.22:0/2522001477 >> 172.21.15.22:0/2522001477 conn(0x7f13e00d15f0 msgr2=0x7f13e0073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13e6069640 1 -- 172.21.15.22:0/2522001477 shutdown_connections 2024-09-21T05:41:39.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.584+0000 7f13e6069640 1 -- 172.21.15.22:0/2522001477 wait complete. 2024-09-21T05:41:39.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.586+0000 7f13e6069640 1 Processor -- start 2024-09-21T05:41:39.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.586+0000 7f13e6069640 1 -- start start 2024-09-21T05:41:39.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13e6069640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13e015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13e6069640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e015b200 0x7f13e015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13e6069640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13e015e560 0x7f13e01e8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13e6069640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00d06f0 con 0x7f13c8004d40 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13e6069640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00d0570 con 0x7f13e015b200 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13e6069640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00d0870 con 0x7f13e015e560 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13df7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13e015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13df7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13e015a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45566/0 (socket says 172.21.15.22:45566) 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.587+0000 7f13df7fe640 1 -- 172.21.15.22:0/2260009045 learned_addr learned my addr 172.21.15.22:0/2260009045 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13deffd640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e015b200 0x7f13e015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13df7fe640 1 -- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13e015e560 msgr2=0x7f13e01e8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13dffff640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13e015e560 0x7f13e01e8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13df7fe640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13e015e560 0x7f13e01e8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13df7fe640 1 -- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e015b200 msgr2=0x7f13e015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13df7fe640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e015b200 0x7f13e015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13df7fe640 1 -- 172.21.15.22:0/2260009045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e015e970 con 0x7f13c8004d40 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13deffd640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e015b200 0x7f13e015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13df7fe640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13e015a9c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f13d0011ae0 tx=0x7f13d0009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d001e070 con 0x7f13c8004d40 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d000fe70 con 0x7f13c8004d40 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e00cc7f0 con 0x7f13c8004d40 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004290 con 0x7f13c8004d40 2024-09-21T05:41:39.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.588+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01f5a60 con 0x7f13c8004d40 2024-09-21T05:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.589+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e00cfe60 con 0x7f13c8004d40 2024-09-21T05:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.590+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13d0021070 con 0x7f13c8004d40 2024-09-21T05:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.590+0000 7f13c7fff640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f13cc078b90 0x7f13cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.591+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f13d009b7e0 con 0x7f13c8004d40 2024-09-21T05:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.591+0000 7f13deffd640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f13cc078b90 0x7f13cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:39.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.591+0000 7f13deffd640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f13cc078b90 0x7f13cc07b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d4006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:39.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.595+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f13e015e970 con 0x7f13c8004d40 2024-09-21T05:41:39.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.826+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f13e0074200 con 0x7f13c8004d40 2024-09-21T05:41:39.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.826+0000 7f13c7fff640 1 -- 172.21.15.22:0/2260009045 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f13d0063680 con 0x7f13c8004d40 2024-09-21T05:41:39.829 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.828+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f13cc078b90 msgr2=0x7f13cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f13cc078b90 0x7f13cc07b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f13d4000c00 tx=0x7f13d4006b70 comp rx=0 tx=0).stop 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 msgr2=0x7f13e015a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13e015a9c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f13d0011ae0 tx=0x7f13d0009e90 comp rx=0 tx=0).stop 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 shutdown_connections 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f13cc078b90 0x7f13cc07b0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f13e015e560 0x7f13e01e8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f13e015b200 0x7f13e015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 --2- 172.21.15.22:0/2260009045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f13c8004d40 0x7f13e015a9c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:39.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 >> 172.21.15.22:0/2260009045 conn(0x7f13e00d15f0 msgr2=0x7f13e0142f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:39.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 shutdown_connections 2024-09-21T05:41:39.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:39.829+0000 7f13e6069640 1 -- 172.21.15.22:0/2260009045 wait complete. 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:41:39.842 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:41:39.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:39.843 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:41:39.858 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:41:39.859 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42644sec preferred_lft 42644sec 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:41:39.860 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.862 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:41:39.863 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:41:39.863 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:41:39.863 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:41:39.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:39.863 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:41:39.923 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:41:39.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:39.925 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:41:39.981 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:41:39.981 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:41:39.982 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:41:39.982 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:41:39.982 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:41:39.982 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:41:39.982 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:41:39.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:41:39.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:41:39.982 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:41:39.982 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:41:40.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:41:40.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:41:40.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:40.111 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:41:40.127 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:41:40.127 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:41:40.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:41:40.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:40.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:41:40.193 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:41:40.193 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:41:40.249 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:41:40.249 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:41:40.249 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:41:40.304 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:41:40.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:40.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:41:40.400 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:41:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:41:40.401 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:41:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:41:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:41:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:41:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:41:40.412 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:41:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:41:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:41:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:41:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:41:40.418 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:41:40.419 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:41:40.420 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:41:40.421 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:41:40.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:40.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:41:40.447 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:41:40.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:41:40.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:41:40.554 INFO:teuthology.orchestra.run.smithi022.stdout:client5264 2024-09-21T05:41:40.554 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:41:40.554 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:41:40.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:40.570 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-09-21T05:41:40.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:40.633 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-09-21T05:41:40.691 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:41:40.691 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:41:40.691 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:41:40.691 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:41:40.691 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-09-21T05:41:40.691 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:41:40.798 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:41:40.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:41:40.807 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.962+0000 7fdc1895e640 1 -- 172.21.15.22:0/3095208729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 msgr2=0x7fdc140cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.962+0000 7fdc1895e640 1 --2- 172.21.15.22:0/3095208729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 0x7fdc140cba00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009860 tx=0x7fdc0c02a530 comp rx=0 tx=0).stop 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.962+0000 7fdc1895e640 1 -- 172.21.15.22:0/3095208729 shutdown_connections 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.962+0000 7fdc1895e640 1 --2- 172.21.15.22:0/3095208729 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 0x7fdc140cba00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.962+0000 7fdc1895e640 1 -- 172.21.15.22:0/3095208729 >> 172.21.15.22:0/3095208729 conn(0x7fdc140b1cb0 msgr2=0x7fdc140b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.963+0000 7fdc1895e640 1 -- 172.21.15.22:0/3095208729 shutdown_connections 2024-09-21T05:41:40.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.963+0000 7fdc1895e640 1 -- 172.21.15.22:0/3095208729 wait complete. 2024-09-21T05:41:40.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.964+0000 7fdc1895e640 1 Processor -- start 2024-09-21T05:41:40.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.965+0000 7fdc1895e640 1 -- start start 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1895e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 0x7fdc1415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1895e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 0x7fdc1414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1895e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc140c8430 0x7fdc140ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1895e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdc140741d0 con 0x7fdc140ca800 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1895e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdc14074050 con 0x7fdc140c8430 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1895e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdc14074350 con 0x7fdc140cad30 2024-09-21T05:41:40.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 0x7fdc1415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 0x7fdc1415ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45568/0 (socket says 172.21.15.22:45568) 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 -- 172.21.15.22:0/817914393 learned_addr learned my addr 172.21.15.22:0/817914393 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc12d9d640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc140c8430 0x7fdc140ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc11d9b640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 0x7fdc1414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 -- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 msgr2=0x7fdc1414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 0x7fdc1414c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 -- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc140c8430 msgr2=0x7fdc140ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc140c8430 0x7fdc140ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 -- 172.21.15.22:0/817914393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1414c930 con 0x7fdc140ca800 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc12d9d640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc140c8430 0x7fdc140ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.966+0000 7fdc1259c640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 0x7fdc1415ce70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400c730 tx=0x7fdc0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.967+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc04003b70 con 0x7fdc140ca800 2024-09-21T05:41:40.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.967+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1415d850 con 0x7fdc140ca800 2024-09-21T05:41:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.967+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc04003d30 con 0x7fdc140ca800 2024-09-21T05:41:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.967+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0400aa50 con 0x7fdc140ca800 2024-09-21T05:41:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.967+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1414bbf0 con 0x7fdc140ca800 2024-09-21T05:41:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.968+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc140cb280 con 0x7fdc140ca800 2024-09-21T05:41:40.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.969+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdc0400abf0 con 0x7fdc140ca800 2024-09-21T05:41:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.969+0000 7fdc0affd640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdbf4078bc0 0x7fdbf407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.969+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fdc0409ab60 con 0x7fdc140ca800 2024-09-21T05:41:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.969+0000 7fdc11d9b640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdbf4078bc0 0x7fdbf407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:40.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.970+0000 7fdc11d9b640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdbf4078bc0 0x7fdbf407b0d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc001240 tx=0x7fdbfc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:40.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:40.974+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc140cb280 con 0x7fdc140ca800 2024-09-21T05:41:41.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.196+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc14074800 con 0x7fdc140ca800 2024-09-21T05:41:41.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.197+0000 7fdc0affd640 1 -- 172.21.15.22:0/817914393 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5226 (secure 0 0 0) 0x7fdc04062fb0 con 0x7fdc140ca800 2024-09-21T05:41:41.199 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:41.199 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":34,"btime":"2024-09-21T05:41:36:644246+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:41:36.026286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:41:41.199 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 34 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdbf4078bc0 msgr2=0x7fdbf407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdbf4078bc0 0x7fdbf407b0d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc001240 tx=0x7fdbfc009040 comp rx=0 tx=0).stop 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 msgr2=0x7fdc1415ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 0x7fdc1415ce70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdc0400c730 tx=0x7fdc0400cbf0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 shutdown_connections 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdc140c8430 0x7fdc140ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdbf4078bc0 0x7fdbf407b0d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdc140cad30 0x7fdc1414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 --2- 172.21.15.22:0/817914393 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdc140ca800 0x7fdc1415ce70 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 >> 172.21.15.22:0/817914393 conn(0x7fdc140b1cb0 msgr2=0x7fdc140d0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 shutdown_connections 2024-09-21T05:41:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.199+0000 7fdc1895e640 1 -- 172.21.15.22:0/817914393 wait complete. 2024-09-21T05:41:41.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T05:41:41.352 INFO:teuthology.orchestra.run.smithi022.stdout:{"message":"","return_code":0} 2024-09-21T05:41:41.353 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:41:41.353 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-21T05:41:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.406+0000 7f7d0e418f40 1 Processor -- start 2024-09-21T05:41:41.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.406+0000 7f7d0e418f40 1 -- start start 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d0e418f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf33d0 0x55d82edc4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d0e418f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82ecf54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d0e418f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 0x55d82ee5f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d0e418f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d82eca3430 con 0x55d82ecf6e60 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d0e418f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d82ecb01f0 con 0x55d82ecf33d0 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d0e418f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d82ec44d60 con 0x55d82ecf4ea0 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 0x55d82ee5f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 0x55d82ee5f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45574/0 (socket says 172.21.15.22:45574) 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 -- 172.21.15.22:0/2136039879 learned_addr learned my addr 172.21.15.22:0/2136039879 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d07ed2640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf33d0 0x55d82edc4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d076d1640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82ecf54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 -- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 msgr2=0x55d82ecf54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82ecf54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 -- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf33d0 msgr2=0x55d82edc4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf33d0 0x55d82edc4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d086d3640 1 -- 172.21.15.22:0/2136039879 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82ed223a0 con 0x55d82ecf6e60 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d07ed2640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf33d0 0x55d82edc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.407+0000 7f7d076d1640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82ecf54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.408+0000 7f7d086d3640 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 0x55d82ee5f020 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a4b0 tx=0x7f7d0000a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720cfc3e9cd6a2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.408+0000 7f7d066cf640 1 -- 172.21.15.22:0/2136039879 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00015070 con 0x55d82ecf6e60 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.408+0000 7f7d066cf640 1 -- 172.21.15.22:0/2136039879 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d0000e730 con 0x55d82ecf6e60 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.408+0000 7f7d066cf640 1 -- 172.21.15.22:0/2136039879 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000eb80 con 0x55d82ecf6e60 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.408+0000 7f7d0e418f40 1 -- 172.21.15.22:0/2136039879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 msgr2=0x55d82ee5f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.408+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 0x55d82ee5f020 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000a4b0 tx=0x7f7d0000a970 comp rx=0 tx=0).stop 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 -- 172.21.15.22:0/2136039879 shutdown_connections 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf6e60 0x55d82ee5f020 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82ecf54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/2136039879 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf33d0 0x55d82edc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 -- 172.21.15.22:0/2136039879 >> 172.21.15.22:0/2136039879 conn(0x55d82ec15ac0 msgr2=0x55d82ed1ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 -- 172.21.15.22:0/2136039879 shutdown_connections 2024-09-21T05:41:41.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.409+0000 7f7d0e418f40 1 -- 172.21.15.22:0/2136039879 wait complete. 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.410+0000 7f7d0e418f40 1 Processor -- start 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.410+0000 7f7d0e418f40 1 -- start start 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d0e418f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf33d0 0x55d82edc7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d0e418f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82edcc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d0e418f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf6e60 0x55d82edcca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d0e418f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d82ed3aec0 con 0x55d82ecf33d0 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d0e418f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d82ed3b110 con 0x55d82ecf6e60 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d0e418f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d82edc9c10 con 0x55d82ecf4ea0 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf33d0 0x55d82edc7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf33d0 0x55d82edc7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45582/0 (socket says 172.21.15.22:45582) 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 -- 172.21.15.22:0/1463673846 learned_addr learned my addr 172.21.15.22:0/1463673846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d076d1640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82edcc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d086d3640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf6e60 0x55d82edcca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 -- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 msgr2=0x55d82edcc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d82ecf4ea0 0x55d82edcc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 -- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf6e60 msgr2=0x55d82edcca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d82ecf6e60 0x55d82edcca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.411+0000 7f7d07ed2640 1 -- 172.21.15.22:0/1463673846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d82edc4330 con 0x55d82ecf33d0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.412+0000 7f7d07ed2640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf33d0 0x55d82edc7f00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004f40 tx=0x7f7cf804eff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.412+0000 7f7d056cd640 1 -- 172.21.15.22:0/1463673846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf804d6e0 con 0x55d82ecf33d0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.412+0000 7f7d056cd640 1 -- 172.21.15.22:0/1463673846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf8067070 con 0x55d82ecf33d0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.412+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d82edc3040 con 0x55d82ecf33d0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.412+0000 7f7d056cd640 1 -- 172.21.15.22:0/1463673846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8056690 con 0x55d82ecf33d0 2024-09-21T05:41:41.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.412+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d82edc92c0 con 0x55d82ecf33d0 2024-09-21T05:41:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.414+0000 7f7d056cd640 1 -- 172.21.15.22:0/1463673846 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cf8056830 con 0x55d82ecf33d0 2024-09-21T05:41:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.414+0000 7f7d056cd640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7cf4077570 0x7f7cf4079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.414+0000 7f7d056cd640 1 -- 172.21.15.22:0/1463673846 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf80ddc40 con 0x55d82ecf33d0 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.414+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d82edbd710 0x55d82ee67720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.414+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57) -- 0x55d82edccf70 con 0x55d82edbd710 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.414+0000 7f7d076d1640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7cf4077570 0x7f7cf4079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.415+0000 7f7d086d3640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d82edbd710 0x55d82ee67720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.415+0000 7f7d076d1640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7cf4077570 0x7f7cf4079a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55d82ed228d0 tx=0x7f7cfc0037e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.415+0000 7f7d086d3640 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d82edbd710 0x55d82ee67720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d086d3640 1 -- 172.21.15.22:0/1463673846 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f7d0000e070 con 0x55d82edbd710 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d82edbd710 msgr2=0x55d82ee67720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d82edbd710 0x55d82ee67720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7cf4077570 msgr2=0x7f7cf4079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7cf4077570 0x7f7cf4079a80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55d82ed228d0 tx=0x7f7cfc0037e0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf33d0 msgr2=0x55d82edc7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 --2- 172.21.15.22:0/1463673846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d82ecf33d0 0x55d82edc7f00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004f40 tx=0x7f7cf804eff0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d086d3640 1 -- 172.21.15.22:0/1463673846 reap_dead start 2024-09-21T05:41:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 shutdown_connections 2024-09-21T05:41:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 >> 172.21.15.22:0/1463673846 conn(0x55d82ec15ac0 msgr2=0x55d82ed3b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 shutdown_connections 2024-09-21T05:41:41.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.416+0000 7f7d0e418f40 1 -- 172.21.15.22:0/1463673846 wait complete. 2024-09-21T05:41:41.421 INFO:teuthology.orchestra.run.smithi022.stdout:Xfiledir_test 2024-09-21T05:41:41.422 DEBUG:teuthology.orchestra.run.smithi022:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-21T05:41:41.477 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627777, 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "ancestors": [ 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "dirino": 1099511627776, 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "file", 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 2 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "dirino": 1, 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "dir_test", 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 4 2024-09-21T05:41:41.478 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:41.479 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-21T05:41:41.479 INFO:teuthology.orchestra.run.smithi022.stdout: "pool": 9, 2024-09-21T05:41:41.479 INFO:teuthology.orchestra.run.smithi022.stdout: "old_pools": [] 2024-09-21T05:41:41.479 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:41.479 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:41:41.481 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-09-21T05:41:41.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.575+0000 7fbcba985f40 1 Processor -- start 2024-09-21T05:41:41.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.575+0000 7fbcba985f40 1 -- start start 2024-09-21T05:41:41.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.576+0000 7fbcba985f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509e772c0 0x55d509f13d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.576+0000 7fbcba985f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509f1c300 0x55d509f1a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.576+0000 7fbcba985f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 0x55d509f1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.576+0000 7fbcba985f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d509d97d60 con 0x55d509f236b0 2024-09-21T05:41:41.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.576+0000 7fbcba985f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d509df6430 con 0x55d509e772c0 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.576+0000 7fbcba985f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d509e90840 con 0x55d509f1c300 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 0x55d509f1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 0x55d509f1acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45608/0 (socket says 172.21.15.22:45608) 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 -- 172.21.15.22:0/1726291577 learned_addr learned my addr 172.21.15.22:0/1726291577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb3c3e640 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509f1c300 0x55d509f1a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb443f640 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509e772c0 0x55d509f13d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 -- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509f1c300 msgr2=0x55d509f1a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509f1c300 0x55d509f1a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 -- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509e772c0 msgr2=0x55d509f13d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509e772c0 0x55d509f13d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 -- 172.21.15.22:0/1726291577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d509e49070 con 0x55d509f236b0 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb443f640 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509e772c0 0x55d509f13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 0x55d509f1acf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00b700 tx=0x7fbcac00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a648e0a85c09a051 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb2c3c640 1 -- 172.21.15.22:0/1726291577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0099d0 con 0x55d509f236b0 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb2c3c640 1 -- 172.21.15.22:0/1726291577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcac009c60 con 0x55d509f236b0 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.577+0000 7fbcb2c3c640 1 -- 172.21.15.22:0/1726291577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00e6b0 con 0x55d509f236b0 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 -- 172.21.15.22:0/1726291577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 msgr2=0x55d509f1acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 0x55d509f1acf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00b700 tx=0x7fbcac00bbc0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 -- 172.21.15.22:0/1726291577 shutdown_connections 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f236b0 0x55d509f1acf0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509f1c300 0x55d509f1a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 --2- 172.21.15.22:0/1726291577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509e772c0 0x55d509f13d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 -- 172.21.15.22:0/1726291577 >> 172.21.15.22:0/1726291577 conn(0x55d509d68ac0 msgr2=0x55d509f15f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 -- 172.21.15.22:0/1726291577 shutdown_connections 2024-09-21T05:41:41.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.578+0000 7fbcba985f40 1 -- 172.21.15.22:0/1726291577 wait complete. 2024-09-21T05:41:41.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.579+0000 7fbcba985f40 1 Processor -- start 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.579+0000 7fbcba985f40 1 -- start start 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcba985f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509e772c0 0x55d509f0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcba985f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f1c300 0x55d509f10480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcba985f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509f236b0 0x55d509f109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcba985f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d509e493e0 con 0x55d509f1c300 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcba985f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d509f18cf0 con 0x55d509f236b0 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcba985f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d509f18fd0 con 0x55d509e772c0 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcb3c3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f1c300 0x55d509f10480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcb3c3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f1c300 0x55d509f10480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45624/0 (socket says 172.21.15.22:45624) 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcb3c3e640 1 -- 172.21.15.22:0/3094973010 learned_addr learned my addr 172.21.15.22:0/3094973010 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.580+0000 7fbcb443f640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509e772c0 0x55d509f0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509f236b0 0x55d509f109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb3c3e640 1 -- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509e772c0 msgr2=0x55d509f0d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb3c3e640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d509e772c0 0x55d509f0d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb3c3e640 1 -- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509f236b0 msgr2=0x55d509f109c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb3c3e640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509f236b0 0x55d509f109c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb3c3e640 1 -- 172.21.15.22:0/3094973010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d509f22a90 con 0x55d509f1c300 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d509f236b0 0x55d509f109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb3c3e640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f1c300 0x55d509f10480 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbca8002ee0 tx=0x7fbca800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb1c3a640 1 -- 172.21.15.22:0/3094973010 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800f8d0 con 0x55d509f1c300 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb1c3a640 1 -- 172.21.15.22:0/3094973010 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbca8013070 con 0x55d509f1c300 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d509f22d00 con 0x55d509f1c300 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcb1c3a640 1 -- 172.21.15.22:0/3094973010 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800fda0 con 0x55d509f1c300 2024-09-21T05:41:41.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.581+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d509f0fd10 con 0x55d509f1c300 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.583+0000 7fbcb1c3a640 1 -- 172.21.15.22:0/3094973010 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbca8018470 con 0x55d509f1c300 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.583+0000 7fbcb1c3a640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbca00775a0 0x7fbca0079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.583+0000 7fbcb1c3a640 1 -- 172.21.15.22:0/3094973010 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fbca809a2e0 con 0x55d509f1c300 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.584+0000 7fbcba985f40 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d509e91a60 0x55d509fc14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.584+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e57) -- 0x55d509fc19f0 con 0x55d509e91a60 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.584+0000 7fbcb443f640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbca00775a0 0x7fbca0079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.584+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d509e91a60 0x55d509fc14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.584+0000 7fbcb443f640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbca00775a0 0x7fbca0079ab0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbca4055000 tx=0x7fbca4003020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.584+0000 7fbcb4c40640 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d509e91a60 0x55d509fc14b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcb4c40640 1 -- 172.21.15.22:0/3094973010 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v57'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbcac009c60 con 0x55d509e91a60 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d509e91a60 msgr2=0x55d509fc14b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55d509e91a60 0x55d509fc14b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbca00775a0 msgr2=0x7fbca0079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbca00775a0 0x7fbca0079ab0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbca4055000 tx=0x7fbca4003020 comp rx=0 tx=0).stop 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f1c300 msgr2=0x55d509f10480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 --2- 172.21.15.22:0/3094973010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d509f1c300 0x55d509f10480 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbca8002ee0 tx=0x7fbca800c820 comp rx=0 tx=0).stop 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcb4c40640 1 -- 172.21.15.22:0/3094973010 reap_dead start 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 shutdown_connections 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 >> 172.21.15.22:0/3094973010 conn(0x55d509d68ac0 msgr2=0x55d509f15f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 shutdown_connections 2024-09-21T05:41:41.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.586+0000 7fbcba985f40 1 -- 172.21.15.22:0/3094973010 wait complete. 2024-09-21T05:41:41.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:41.592 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-09-21T05:41:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 -- 172.21.15.22:0/3542774070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 msgr2=0x7f7b300b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 --2- 172.21.15.22:0/3542774070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b300b13a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7b300d3410 tx=0x7f7b2802aa00 comp rx=0 tx=0).stop 2024-09-21T05:41:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 -- 172.21.15.22:0/3542774070 shutdown_connections 2024-09-21T05:41:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 --2- 172.21.15.22:0/3542774070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b300b13a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 --2- 172.21.15.22:0/3542774070 >> v2:172.21.15.134:3300/0 conn(0x7f7b30162400 0x7f7b30162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 -- 172.21.15.22:0/3542774070 >> 172.21.15.22:0/3542774070 conn(0x7f7b300d4380 msgr2=0x7f7b300d4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 -- 172.21.15.22:0/3542774070 shutdown_connections 2024-09-21T05:41:41.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.713+0000 7f7b3699d640 1 -- 172.21.15.22:0/3542774070 wait complete. 2024-09-21T05:41:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.714+0000 7f7b3699d640 1 Processor -- start 2024-09-21T05:41:41.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.714+0000 7f7b3699d640 1 -- start start 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3699d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b3014a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3699d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b300d4ea0 0x7f7b3014a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3699d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30162400 0x7f7b300cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3699d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b300b1a60 con 0x7f7b300b0f90 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3699d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b300b18e0 con 0x7f7b30162400 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3699d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b300b1be0 con 0x7f7b300d4ea0 2024-09-21T05:41:41.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b3014a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b3014a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45632/0 (socket says 172.21.15.22:45632) 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 -- 172.21.15.22:0/2395192798 learned_addr learned my addr 172.21.15.22:0/2395192798 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3619c640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30162400 0x7f7b300cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3519a640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b300d4ea0 0x7f7b3014a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 -- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b300d4ea0 msgr2=0x7f7b3014a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b300d4ea0 0x7f7b3014a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 -- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30162400 msgr2=0x7f7b300cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30162400 0x7f7b300cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 -- 172.21.15.22:0/2395192798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b300cb460 con 0x7f7b300b0f90 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3619c640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30162400 0x7f7b300cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.716+0000 7f7b3599b640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b3014a7c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b3c0 tx=0x7f7b2c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.717+0000 7f7b267fc640 1 -- 172.21.15.22:0/2395192798 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c005920 con 0x7f7b300b0f90 2024-09-21T05:41:41.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.717+0000 7f7b267fc640 1 -- 172.21.15.22:0/2395192798 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b2c005bd0 con 0x7f7b300b0f90 2024-09-21T05:41:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.717+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b300cb740 con 0x7f7b300b0f90 2024-09-21T05:41:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.717+0000 7f7b267fc640 1 -- 172.21.15.22:0/2395192798 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c006020 con 0x7f7b300b0f90 2024-09-21T05:41:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.717+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b300cbd00 con 0x7f7b300b0f90 2024-09-21T05:41:41.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.718+0000 7f7b267fc640 1 -- 172.21.15.22:0/2395192798 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7b2c018070 con 0x7f7b300b0f90 2024-09-21T05:41:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.719+0000 7f7b267fc640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7b14078bc0 0x7f7b1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.719+0000 7f7b267fc640 1 -- 172.21.15.22:0/2395192798 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7b2c099f40 con 0x7f7b300b0f90 2024-09-21T05:41:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.720+0000 7f7b3519a640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7b14078bc0 0x7f7b1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.720+0000 7f7b3ec45740 1 Processor -- start 2024-09-21T05:41:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.720+0000 7f7b3ec45740 1 -- start start 2024-09-21T05:41:41.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.720+0000 7f7b3519a640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7b14078bc0 0x7f7b1407b0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005fd0 tx=0x7f7b20005d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.721+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.721+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.721+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d5b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.721+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55704d4fc890 con 0x55704d1c1660 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.721+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55704d4ebcc0 con 0x55704d1e2340 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.721+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55704d4efad0 con 0x55704d1b9480 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d5b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d5b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45644/0 (socket says 172.21.15.22:45644) 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 -- 172.21.15.22:0/1263753827 learned_addr learned my addr 172.21.15.22:0/1263753827 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3599b640 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3519a640 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 -- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 msgr2=0x55704d5b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 -- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 msgr2=0x55704d5d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 -- 172.21.15.22:0/1263753827 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704d2c67b0 con 0x55704d1c1660 2024-09-21T05:41:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3519a640 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b3619c640 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d5b4260 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7b3008ca60 tx=0x7f7b28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4175dab4bafcdf6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b137fe640 1 -- 172.21.15.22:0/1263753827 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28033030 con 0x55704d1c1660 2024-09-21T05:41:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b137fe640 1 -- 172.21.15.22:0/1263753827 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b28003b50 con 0x55704d1c1660 2024-09-21T05:41:41.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.722+0000 7f7b137fe640 1 -- 172.21.15.22:0/1263753827 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2802f950 con 0x55704d1c1660 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1263753827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 msgr2=0x55704d5b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d5b4260 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f7b3008ca60 tx=0x7f7b28004a40 comp rx=0 tx=0).stop 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1263753827 shutdown_connections 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d5b4260 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1263753827 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1263753827 >> 172.21.15.22:0/1263753827 conn(0x55704d1d6490 msgr2=0x55704d589f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1263753827 shutdown_connections 2024-09-21T05:41:41.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.723+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1263753827 wait complete. 2024-09-21T05:41:41.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.724+0000 7f7b3ec45740 1 -- start start 2024-09-21T05:41:41.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.724+0000 7f7b3ec45740 1 Processor -- start 2024-09-21T05:41:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 0x55704d5a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1c1660 0x55704d5c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55704d25c6f0 con 0x55704d1b9480 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55704d472480 con 0x55704d1e2340 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55704d5c90f0 con 0x55704d1c1660 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 0x55704d5a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3ec45740 10 client.0 _ll_register_callbacks cb 0x55704d4e5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 0x55704d5a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45652/0 (socket says 172.21.15.22:45652) 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 -- 172.21.15.22:0/2248558778 learned_addr learned my addr 172.21.15.22:0/2248558778 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3519a640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1c1660 0x55704d5c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3619c640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 -- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1c1660 msgr2=0x55704d5c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1c1660 0x55704d5c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 -- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 msgr2=0x55704d5cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 -- 172.21.15.22:0/2248558778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704d326f40 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b3599b640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 0x55704d5a1ff0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c05dc80 tx=0x7f7b2c0618c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e21370854d7b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.726+0000 7f7b10ff9640 1 -- 172.21.15.22:0/2248558778 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c066750 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b10ff9640 1 -- 172.21.15.22:0/2248558778 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7b2c05d5c0 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b28003b50 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b10ff9640 1 -- 172.21.15.22:0/2248558778 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7b2c083050 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 10 client.5862 fetch_fsmap learned FSMap version 34 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55704d22b650 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b10ff9640 1 -- 172.21.15.22:0/2248558778 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b28003b50 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b10ff9640 1 -- 172.21.15.22:0/2248558778 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7f7b2c05d7c0 con 0x55704d1b9480 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 10 client.5862 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 10 client.5862 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 20 client.5862 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:41.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d1a1120 0x55704d5e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 4 client.5862 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55704d4fe7e0 con 0x55704d1a1120 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.727+0000 7f7b3519a640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d1a1120 0x55704d5e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.728+0000 7f7b10ff9640 10 client.5862 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.728+0000 7f7b3519a640 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d1a1120 0x55704d5e11a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c348dee2c0482f server_cookie=9e4a88ba65856352 in_seq=0 out_seq=0 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.728+0000 7f7b10ff9640 1 -- 172.21.15.22:0/2248558778 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55704d4fe7e0 con 0x55704d1a1120 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.728+0000 7f7b10ff9640 10 client.5862 handle_command_reply: tid=0 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.728+0000 7f7b3ec45740 1 client.5862 shutdown 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 msgr2=0x55704d5a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 0x55704d5a1ff0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c05dc80 tx=0x7f7b2c0618c0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 shutdown_connections 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d1a1120 0x55704d5e11a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d5cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b10ff9640 0 client.5862 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1c1660 0x55704d5c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/2248558778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1b9480 0x55704d5a1ff0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 >> 172.21.15.22:0/2248558778 conn(0x55704d1d6490 msgr2=0x55704d5832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 shutdown_connections 2024-09-21T05:41:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 1 -- 172.21.15.22:0/2248558778 wait complete. 2024-09-21T05:41:41.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.729+0000 7f7b3ec45740 20 client.5862 trim_cache size 0 max 16384 2024-09-21T05:41:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.747+0000 7f7b3ec45740 1 Processor -- start 2024-09-21T05:41:41.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.747+0000 7f7b3ec45740 1 -- start start 2024-09-21T05:41:41.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1c1660 0x55704d589a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 0x55704d5a2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30201fd0 con 0x55704d1e2340 2024-09-21T05:41:41.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301f67b0 con 0x55704d1c1660 2024-09-21T05:41:41.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7b301f5b10 con 0x55704d1b9480 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3619c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 0x55704d5a2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3619c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 0x55704d5a2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45668/0 (socket says 172.21.15.22:45668) 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.748+0000 7f7b3619c640 1 -- 172.21.15.22:0/1317484757 learned_addr learned my addr 172.21.15.22:0/1317484757 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3599b640 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3519a640 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1c1660 0x55704d589a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3619c640 1 -- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 msgr2=0x55704d5e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3619c640 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3619c640 1 -- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1c1660 msgr2=0x55704d589a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3619c640 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1c1660 0x55704d589a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3619c640 1 -- 172.21.15.22:0/1317484757 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704d5cc130 con 0x55704d1e2340 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3519a640 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1c1660 0x55704d589a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b3619c640 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 0x55704d5a2d50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b28031000 tx=0x7f7b280095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191c4ef0be948013 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b12ffd640 1 -- 172.21.15.22:0/1317484757 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2804c070 con 0x55704d1e2340 2024-09-21T05:41:41.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b12ffd640 1 -- 172.21.15.22:0/1317484757 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b2802fa40 con 0x55704d1e2340 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.749+0000 7f7b12ffd640 1 -- 172.21.15.22:0/1317484757 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28033040 con 0x55704d1e2340 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1317484757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 msgr2=0x55704d5a2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 0x55704d5a2d50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b28031000 tx=0x7f7b280095c0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1317484757 shutdown_connections 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1e2340 0x55704d5a2d50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1c1660 0x55704d589a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/1317484757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1317484757 >> 172.21.15.22:0/1317484757 conn(0x55704d1d6490 msgr2=0x55704d5c81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1317484757 shutdown_connections 2024-09-21T05:41:41.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.750+0000 7f7b3ec45740 1 -- 172.21.15.22:0/1317484757 wait complete. 2024-09-21T05:41:41.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.751+0000 7f7b3ec45740 1 -- start start 2024-09-21T05:41:41.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.751+0000 7f7b3ec45740 1 Processor -- start 2024-09-21T05:41:41.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.752+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.752+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d652e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.752+0000 7f7b3ec45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d645790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.752+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55704d589f50 con 0x55704d1c1660 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.752+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301f6160 con 0x55704d1e2340 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.752+0000 7f7b3ec45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55704d5b3fe0 con 0x55704d1b9480 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3ec45740 10 client.0 _ll_register_callbacks cb 0x7f7b2c061b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d652e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d652e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45678/0 (socket says 172.21.15.22:45678) 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 -- 172.21.15.22:0/71326724 learned_addr learned my addr 172.21.15.22:0/71326724 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3619c640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d645790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3599b640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 -- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 msgr2=0x55704d5e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 -- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 msgr2=0x55704d645790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d645790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 -- 172.21.15.22:0/71326724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704d64ee60 con 0x55704d1c1660 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3619c640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d645790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3519a640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d652e30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7b30161ef0 tx=0x7f7b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577d098dc609cd63 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b10ff9640 1 -- 172.21.15.22:0/71326724 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002b070 con 0x55704d1c1660 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b10ff9640 1 -- 172.21.15.22:0/71326724 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7b20010900 con 0x55704d1c1660 2024-09-21T05:41:41.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2802fa40 con 0x55704d1c1660 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b10ff9640 1 -- 172.21.15.22:0/71326724 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7b20039050 con 0x55704d1c1660 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.753+0000 7f7b3ec45740 10 client.5868 fetch_fsmap learned FSMap version 34 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55704d645d30 con 0x55704d1c1660 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b10ff9640 1 -- 172.21.15.22:0/71326724 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b2802fa40 con 0x55704d1c1660 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b10ff9640 1 -- 172.21.15.22:0/71326724 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7f7b2003c020 con 0x55704d1c1660 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 10 client.5868 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 10 client.5868 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 20 client.5868 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:41.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d58b160 0x55704d5d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 4 client.5868 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55704d4fa290 con 0x55704d58b160 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b3599b640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d58b160 0x55704d5d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.754+0000 7f7b10ff9640 10 client.5868 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.755+0000 7f7b3599b640 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d58b160 0x55704d5d09a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e82bb9d17cce983 server_cookie=f87a9be6113b937f in_seq=0 out_seq=0 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b10ff9640 1 -- 172.21.15.22:0/71326724 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55704d4fa290 con 0x55704d58b160 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b10ff9640 10 client.5868 handle_command_reply: tid=0 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 client.5868 shutdown 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 msgr2=0x55704d652e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d652e30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7b30161ef0 tx=0x7f7b200046c0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 shutdown_connections 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55704d58b160 0x55704d5d09a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55704d1e2340 0x55704d645790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55704d1c1660 0x55704d652e30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b10ff9640 0 client.5868 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 --2- 172.21.15.22:0/71326724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55704d1b9480 0x55704d5e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 >> 172.21.15.22:0/71326724 conn(0x55704d1d6490 msgr2=0x55704d5c81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 shutdown_connections 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 1 -- 172.21.15.22:0/71326724 wait complete. 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.756+0000 7f7b3ec45740 20 client.5868 trim_cache size 0 max 16384 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "eb94107a-f919-4784-b8a0-9ecc5d24cea9", 2024-09-21T05:41:41.764 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:41:41.765 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:41.765 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f13ac341640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 pv4 ap=4 s=0 n(v0 1=1+0)->n(v0 rc2024-09-21T05:41:40.688007+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={5264=0-4194304@1} caps={5264=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=5264 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=0 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x55c929782b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7b14078bc0 msgr2=0x7f7b1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7b14078bc0 0x7f7b1407b0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7b20005fd0 tx=0x7f7b20005d80 comp rx=0 tx=0).stop 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 msgr2=0x7f7b3014a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b3014a7c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b3c0 tx=0x7f7b2c00b880 comp rx=0 tx=0).stop 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 shutdown_connections 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7b30162400 0x7f7b300cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7b14078bc0 0x7f7b1407b0d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7b300d4ea0 0x7f7b3014a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 --2- 172.21.15.22:0/2395192798 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7b300b0f90 0x7f7b3014a7c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 >> 172.21.15.22:0/2395192798 conn(0x7f7b300d4380 msgr2=0x7f7b300d4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 shutdown_connections 2024-09-21T05:41:41.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.757+0000 7f7b3699d640 1 -- 172.21.15.22:0/2395192798 wait complete. 2024-09-21T05:41:41.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:41:41.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 -- 172.21.15.22:0/686340943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0d1a80 msgr2=0x7ff9bc0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 --2- 172.21.15.22:0/686340943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0d1a80 0x7ff9bc0cf090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005fd0 tx=0x7ff9b002a520 comp rx=0 tx=0).stop 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 -- 172.21.15.22:0/686340943 shutdown_connections 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 --2- 172.21.15.22:0/686340943 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 --2- 172.21.15.22:0/686340943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0d1a80 0x7ff9bc0cf090 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 -- 172.21.15.22:0/686340943 >> 172.21.15.22:0/686340943 conn(0x7ff9bc08bfc0 msgr2=0x7ff9bc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 -- 172.21.15.22:0/686340943 shutdown_connections 2024-09-21T05:41:41.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.887+0000 7ff9c18de640 1 -- 172.21.15.22:0/686340943 wait complete. 2024-09-21T05:41:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.889+0000 7ff9c18de640 1 Processor -- start 2024-09-21T05:41:41.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.889+0000 7ff9c18de640 1 -- start start 2024-09-21T05:41:41.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9c18de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9c18de640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0d1a80 0x7ff9bc0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9c18de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9bc0d2ed0 0x7ff9bc0b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9c18de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc162c60 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9c18de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc162ae0 con 0x7ff9bc0d2ed0 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9c18de640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc162de0 con 0x7ff9bc0d1a80 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45702/0 (socket says 172.21.15.22:45702) 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.890+0000 7ff9baffd640 1 -- 172.21.15.22:0/3946057052 learned_addr learned my addr 172.21.15.22:0/3946057052 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9bc0d2ed0 0x7ff9bc0b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0d1a80 0x7ff9bc0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9baffd640 1 -- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0d1a80 msgr2=0x7ff9bc0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0d1a80 0x7ff9bc0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9baffd640 1 -- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9bc0d2ed0 msgr2=0x7ff9bc0b8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9bc0d2ed0 0x7ff9bc0b8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9baffd640 1 -- 172.21.15.22:0/3946057052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9bc0cbb30 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0d1a80 0x7ff9bc0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cb5f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b410 tx=0x7ff9ac00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9a3fff640 1 -- 172.21.15.22:0/3946057052 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac015020 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9a3fff640 1 -- 172.21.15.22:0/3946057052 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9ac011920 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9bc0cc4a0 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9a3fff640 1 -- 172.21.15.22:0/3946057052 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac011d70 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.891+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9bc1f94c0 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.893+0000 7ff9a3fff640 1 -- 172.21.15.22:0/3946057052 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff9ac018020 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.893+0000 7ff9a3fff640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff9a8078bc0 0x7ff9a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.894+0000 7ff9a3fff640 1 -- 172.21.15.22:0/3946057052 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac09a950 con 0x7ff9bc0cf5d0 2024-09-21T05:41:41.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.894+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff9a8078bc0 0x7ff9a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.894+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff9a8078bc0 0x7ff9a807b0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0033000 tx=0x7ff9b0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.895+0000 7ff9c884d740 1 Processor -- start 2024-09-21T05:41:41.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.895+0000 7ff9c884d740 1 -- start start 2024-09-21T05:41:41.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1ce7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce54860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce54da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9c884d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555a1cdada60 con 0x555a1ca726b0 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555a1cd9ce90 con 0x555a1ca6a4d0 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555a1cda0ca0 con 0x555a1ca93390 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce54da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce54da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45704/0 (socket says 172.21.15.22:45704) 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/1246481930 learned_addr learned my addr 172.21.15.22:0/1246481930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9baffd640 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1ce7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.896+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce54860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 msgr2=0x555a1ce7fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1ce7fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 msgr2=0x555a1ce54860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce54860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/1246481930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1cb757d0 con 0x555a1ca726b0 2024-09-21T05:41:41.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce54860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce54da0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400ae80 tx=0x7ff9b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9ae5f3b2757646 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9a17fa640 1 -- 172.21.15.22:0/1246481930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4010840 con 0x555a1ca726b0 2024-09-21T05:41:41.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9a17fa640 1 -- 172.21.15.22:0/1246481930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b4010ad0 con 0x555a1ca726b0 2024-09-21T05:41:41.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.897+0000 7ff9a17fa640 1 -- 172.21.15.22:0/1246481930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4018430 con 0x555a1ca726b0 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 -- 172.21.15.22:0/1246481930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 msgr2=0x555a1ce54da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce54da0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400ae80 tx=0x7ff9b400f790 comp rx=0 tx=0).stop 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 -- 172.21.15.22:0/1246481930 shutdown_connections 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce54da0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce54860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 --2- 172.21.15.22:0/1246481930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1ce7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 -- 172.21.15.22:0/1246481930 >> 172.21.15.22:0/1246481930 conn(0x555a1ca874e0 msgr2=0x555a1ce3ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 -- 172.21.15.22:0/1246481930 shutdown_connections 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.898+0000 7ff9c884d740 1 -- 172.21.15.22:0/1246481930 wait complete. 2024-09-21T05:41:41.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.899+0000 7ff9c884d740 1 -- start start 2024-09-21T05:41:41.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.899+0000 7ff9c884d740 1 Processor -- start 2024-09-21T05:41:41.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1ce536a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca726b0 0x555a1ce77590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 0x555a1ce7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555a1cb0d740 con 0x555a1ca93390 2024-09-21T05:41:41.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555a1cd42970 con 0x555a1ca726b0 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555a1ce78080 con 0x555a1ca6a4d0 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9c884d740 10 client.0 _ll_register_callbacks cb 0x555a1cd51780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 0x555a1ce7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 0x555a1ce7c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45708/0 (socket says 172.21.15.22:45708) 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.900+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/3853240490 learned_addr learned my addr 172.21.15.22:0/3853240490 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1ce536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca726b0 0x555a1ce77590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 msgr2=0x555a1ce536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1ce536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca726b0 msgr2=0x555a1ce77590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca726b0 0x555a1ce77590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9bb7fe640 1 -- 172.21.15.22:0/3853240490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1cafe970 con 0x555a1ca93390 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca726b0 0x555a1ce77590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:41.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 0x555a1ce7c320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b010 tx=0x7ff9b4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b12c1fe2cc7163 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff99a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff99a7fc640 1 -- 172.21.15.22:0/3853240490 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4025020 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff99a7fc640 1 -- 172.21.15.22:0/3853240490 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff9b4004b20 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b4010ad0 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff99a7fc640 1 -- 172.21.15.22:0/3853240490 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9b402f050 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9c884d740 10 client.5880 fetch_fsmap learned FSMap version 34 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x555a1cadc6a0 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.901+0000 7ff99a7fc640 1 -- 172.21.15.22:0/3853240490 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b4010ad0 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff99a7fc640 1 -- 172.21.15.22:0/3853240490 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7ff9b4032020 con 0x555a1ca93390 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9c884d740 10 client.5880 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9c884d740 10 client.5880 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9c884d740 20 client.5880 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ca51f60 0x555a1ce92190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9c884d740 4 client.5880 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a1cdaf9b0 con 0x555a1ca51f60 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ca51f60 0x555a1ce92190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff99a7fc640 10 client.5880 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.902+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ca51f60 0x555a1ce92190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c211dd0540aa95 server_cookie=81c9686214a4a300 in_seq=0 out_seq=0 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff99a7fc640 1 -- 172.21.15.22:0/3853240490 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555a1cdaf9b0 con 0x555a1ca51f60 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff99a7fc640 10 client.5880 handle_command_reply: tid=0 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 client.5880 shutdown 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 msgr2=0x555a1ce7c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 0x555a1ce7c320 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400b010 tx=0x7ff9b4003ba0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 shutdown_connections 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ca51f60 0x555a1ce92190 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca93390 0x555a1ce7c320 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca726b0 0x555a1ce77590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff99a7fc640 0 client.5880 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3853240490 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1ce536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 >> 172.21.15.22:0/3853240490 conn(0x555a1ca874e0 msgr2=0x555a1ce61c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 shutdown_connections 2024-09-21T05:41:41.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 1 -- 172.21.15.22:0/3853240490 wait complete. 2024-09-21T05:41:41.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.903+0000 7ff9c884d740 20 client.5880 trim_cache size 0 max 16384 2024-09-21T05:41:41.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.921+0000 7ff9c884d740 1 Processor -- start 2024-09-21T05:41:41.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.921+0000 7ff9c884d740 1 -- start start 2024-09-21T05:41:41.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce90e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1cea1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9c884d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc2022b0 con 0x555a1ca726b0 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc201440 con 0x555a1ca6a4d0 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff9bc0c8300 con 0x555a1ca93390 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce7a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45714/0 (socket says 172.21.15.22:45714) 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/3976522429 learned_addr learned my addr 172.21.15.22:0/3976522429 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce90e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1cea1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 msgr2=0x555a1cea1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1cea1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 msgr2=0x555a1ce90e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce90e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/3976522429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1ce81070 con 0x555a1ca726b0 2024-09-21T05:41:41.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9baffd640 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce90e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce7a8c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0000c00 tx=0x7ff9b003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f286baf30cc5e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9a0ff9640 1 -- 172.21.15.22:0/3976522429 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b004e070 con 0x555a1ca726b0 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9a0ff9640 1 -- 172.21.15.22:0/3976522429 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b0042030 con 0x555a1ca726b0 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.923+0000 7ff9a0ff9640 1 -- 172.21.15.22:0/3976522429 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0035040 con 0x555a1ca726b0 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 -- 172.21.15.22:0/3976522429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 msgr2=0x555a1ce7a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce7a8c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0000c00 tx=0x7ff9b003c000 comp rx=0 tx=0).stop 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 -- 172.21.15.22:0/3976522429 shutdown_connections 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca93390 0x555a1cea1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1ce7a8c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 --2- 172.21.15.22:0/3976522429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca6a4d0 0x555a1ce90e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 -- 172.21.15.22:0/3976522429 >> 172.21.15.22:0/3976522429 conn(0x555a1ca874e0 msgr2=0x555a1ce7ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 -- 172.21.15.22:0/3976522429 shutdown_connections 2024-09-21T05:41:41.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.924+0000 7ff9c884d740 1 -- 172.21.15.22:0/3976522429 wait complete. 2024-09-21T05:41:41.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.925+0000 7ff9c884d740 1 -- start start 2024-09-21T05:41:41.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.925+0000 7ff9c884d740 1 Processor -- start 2024-09-21T05:41:41.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1cf01bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1cefde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca93390 0x555a1ce7d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555a1ce79ad0 con 0x555a1ca726b0 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff9bc0c8950 con 0x555a1ca93390 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555a1ce914f0 con 0x555a1ca6a4d0 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1cefde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 10 client.0 _ll_register_callbacks cb 0x7ff9ac05ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1cefde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:45730/0 (socket says 172.21.15.22:45730) 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/4109221077 learned_addr learned my addr 172.21.15.22:0/4109221077 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9baffd640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1cf01bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca93390 0x555a1ce7d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 msgr2=0x555a1cf01bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1cf01bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca93390 msgr2=0x555a1ce7d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca93390 0x555a1ce7d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 -- 172.21.15.22:0/4109221077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1ce910f0 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9bb7fe640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca93390 0x555a1ce7d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9ba7fc640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1cefde60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0034040 tx=0x7ff9b0003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0f19b2cc84c69f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff99a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff99a7fc640 1 -- 172.21.15.22:0/4109221077 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b004e040 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff99a7fc640 1 -- 172.21.15.22:0/4109221077 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff9b0004a20 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.927+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b0042030 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff99a7fc640 1 -- 172.21.15.22:0/4109221077 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff9b005b050 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 10 client.5886 fetch_fsmap learned FSMap version 34 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x555a1cf02110 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff99a7fc640 1 -- 172.21.15.22:0/4109221077 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b0042030 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff99a7fc640 1 -- 172.21.15.22:0/4109221077 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 34) ==== 3502+0+0 (secure 0 0 0) 0x7ff9b005e020 con 0x555a1ca726b0 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 10 client.5886 fetch_fsmap finished waiting for FSMap version 34 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 10 client.5886 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:41:41.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 20 client.5886 populate_metadata read hostname 'smithi022' 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ce35730 0x555a1ce35b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 4 client.5886 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555a1ce64090 con 0x555a1ce35730 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9baffd640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ce35730 0x555a1ce35b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff99a7fc640 10 client.5886 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.928+0000 7ff9baffd640 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ce35730 0x555a1ce35b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c482377d700c9cd6 server_cookie=187a7aac402e1620 in_seq=0 out_seq=0 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff99a7fc640 1 -- 172.21.15.22:0/4109221077 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x555a1ce64090 con 0x555a1ce35730 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff99a7fc640 10 client.5886 handle_command_reply: tid=0 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 client.5886 shutdown 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 msgr2=0x555a1cefde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1cefde60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0034040 tx=0x7ff9b0003bc0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 shutdown_connections 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x555a1ce35730 0x555a1ce35b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555a1ca93390 0x555a1ce7d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555a1ca726b0 0x555a1cefde60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff99a7fc640 0 client.5886 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 --2- 172.21.15.22:0/4109221077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555a1ca6a4d0 0x555a1cf01bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 >> 172.21.15.22:0/4109221077 conn(0x555a1ca874e0 msgr2=0x555a1ce7ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 shutdown_connections 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 1 -- 172.21.15.22:0/4109221077 wait complete. 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.929+0000 7ff9c884d740 20 client.5886 trim_cache size 0 max 16384 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:41:41.937 INFO:teuthology.orchestra.run.smithi022.stdout: "eb94107a-f919-4784-b8a0-9ecc5d24cea9": { 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "eb94107a-f919-4784-b8a0-9ecc5d24cea9", 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive" 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "eb94107a-f919-4784-b8a0-9ecc5d24cea9": { 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:41:41.755993+0000", 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:41.938 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:41:41.939 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:41:41.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff9a8078bc0 msgr2=0x7ff9a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff9a8078bc0 0x7ff9a807b0d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0033000 tx=0x7ff9b0004730 comp rx=0 tx=0).stop 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 msgr2=0x7ff9bc0cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cb5f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00b410 tx=0x7ff9ac00b8d0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 shutdown_connections 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff9bc0d2ed0 0x7ff9bc0b8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff9a8078bc0 0x7ff9a807b0d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff9bc0d1a80 0x7ff9bc0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 --2- 172.21.15.22:0/3946057052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff9bc0cf5d0 0x7ff9bc0cb5f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 >> 172.21.15.22:0/3946057052 conn(0x7ff9bc08bfc0 msgr2=0x7ff9bc0b5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 shutdown_connections 2024-09-21T05:41:41.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:41:41.930+0000 7ff9c18de640 1 -- 172.21.15.22:0/3946057052 wait complete. 2024-09-21T05:41:41.944 INFO:tasks.cephfs.filesystem:scrub status for tag:eb94107a-f919-4784-b8a0-9ecc5d24cea9 - {'path': '/', 'tag': 'eb94107a-f919-4784-b8a0-9ecc5d24cea9', 'options': 'recursive'} 2024-09-21T05:41:45.828 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:41:45.825+0000 7fbf61087640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T05:41:51.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:41:51.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:42:11.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:11.945 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 -- 172.21.15.22:0/3727642836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 msgr2=0x7f49100b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 --2- 172.21.15.22:0/3727642836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49100b5210 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4900009830 tx=0x7f490002a510 comp rx=0 tx=0).stop 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 -- 172.21.15.22:0/3727642836 shutdown_connections 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 --2- 172.21.15.22:0/3727642836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49100b5210 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 -- 172.21.15.22:0/3727642836 >> 172.21.15.22:0/3727642836 conn(0x7f49100b3f10 msgr2=0x7f49100b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 -- 172.21.15.22:0/3727642836 shutdown_connections 2024-09-21T05:42:12.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.077+0000 7f4916a94640 1 -- 172.21.15.22:0/3727642836 wait complete. 2024-09-21T05:42:12.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.079+0000 7f4916a94640 1 Processor -- start 2024-09-21T05:42:12.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.079+0000 7f4916a94640 1 -- start start 2024-09-21T05:42:12.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4916a94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49100ccba0 0x7f4910156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4916a94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49101573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4916a94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f491015c6f0 0x7f4910157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4916a94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49100d1440 con 0x7f49100d1d40 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4916a94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49100d12c0 con 0x7f49100ccba0 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4916a94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49100d15c0 con 0x7f491015c6f0 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49101573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49101573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48892/0 (socket says 172.21.15.22:48892) 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 -- 172.21.15.22:0/1792182803 learned_addr learned my addr 172.21.15.22:0/1792182803 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4914830640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49100ccba0 0x7f4910156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4915031640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f491015c6f0 0x7f4910157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 -- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f491015c6f0 msgr2=0x7f4910157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f491015c6f0 0x7f4910157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 -- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49100ccba0 msgr2=0x7f4910156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49100ccba0 0x7f4910156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f490ffff640 1 -- 172.21.15.22:0/1792182803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491015ec10 con 0x7f49100d1d40 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.081+0000 7f4915031640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f491015c6f0 0x7f4910157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.082+0000 7f490ffff640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49101573c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f48f800c3f0 tx=0x7f48f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.082+0000 7f490d7fa640 1 -- 172.21.15.22:0/1792182803 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8017070 con 0x7f49100d1d40 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.082+0000 7f490d7fa640 1 -- 172.21.15.22:0/1792182803 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f80035f0 con 0x7f49100d1d40 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.082+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101560c0 con 0x7f49100d1d40 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.082+0000 7f490d7fa640 1 -- 172.21.15.22:0/1792182803 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8003a40 con 0x7f49100d1d40 2024-09-21T05:42:12.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.082+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4910156680 con 0x7f49100d1d40 2024-09-21T05:42:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.084+0000 7f490d7fa640 1 -- 172.21.15.22:0/1792182803 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f48f801a070 con 0x7f49100d1d40 2024-09-21T05:42:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.084+0000 7f490d7fa640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f48f0078b90 0x7f48f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.084+0000 7f490d7fa640 1 -- 172.21.15.22:0/1792182803 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f48f8099fc0 con 0x7f49100d1d40 2024-09-21T05:42:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.085+0000 7f4914830640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f48f0078b90 0x7f48f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.085+0000 7f491da02740 1 Processor -- start 2024-09-21T05:42:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.085+0000 7f4914830640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f48f0078b90 0x7f48f007b0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4900034000 tx=0x7f4900004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.085+0000 7f491da02740 1 -- start start 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f491da02740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 0x556a3cadd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6e34d0 0x556a3caddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6eb6b0 0x556a3cafab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f491da02740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556a3ca26a60 con 0x556a3c70c390 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a3ca15e90 con 0x556a3c6e34d0 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556a3ca19ca0 con 0x556a3c6eb6b0 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 0x556a3cadd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 0x556a3cadd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48904/0 (socket says 172.21.15.22:48904) 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 -- 172.21.15.22:0/2848685447 learned_addr learned my addr 172.21.15.22:0/2848685447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4915031640 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6eb6b0 0x556a3cafab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.090 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f490ffff640 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6e34d0 0x556a3caddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 -- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6eb6b0 msgr2=0x556a3cafab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6eb6b0 0x556a3cafab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 -- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6e34d0 msgr2=0x556a3caddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6e34d0 0x556a3caddc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 -- 172.21.15.22:0/2848685447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3c7ee7d0 con 0x556a3c70c390 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f4914830640 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 0x556a3cadd740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4900009780 tx=0x7f490003e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67b126119fdaaff server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.087+0000 7f49067fc640 1 -- 172.21.15.22:0/2848685447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900003cb0 con 0x556a3c70c390 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f49067fc640 1 -- 172.21.15.22:0/2848685447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4900003e50 con 0x556a3c70c390 2024-09-21T05:42:12.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f49067fc640 1 -- 172.21.15.22:0/2848685447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900005280 con 0x556a3c70c390 2024-09-21T05:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 -- 172.21.15.22:0/2848685447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 msgr2=0x556a3cadd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 0x556a3cadd740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4900009780 tx=0x7f490003e040 comp rx=0 tx=0).stop 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 -- 172.21.15.22:0/2848685447 shutdown_connections 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6eb6b0 0x556a3cafab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6e34d0 0x556a3caddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 --2- 172.21.15.22:0/2848685447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c70c390 0x556a3cadd740 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 -- 172.21.15.22:0/2848685447 >> 172.21.15.22:0/2848685447 conn(0x556a3c7004e0 msgr2=0x556a3cab3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 -- 172.21.15.22:0/2848685447 shutdown_connections 2024-09-21T05:42:12.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.088+0000 7f491da02740 1 -- 172.21.15.22:0/2848685447 wait complete. 2024-09-21T05:42:12.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.089+0000 7f491da02740 1 -- start start 2024-09-21T05:42:12.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.090+0000 7f491da02740 1 Processor -- start 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 0x556a3caefdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6eb6b0 0x556a3cacbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c70c390 0x556a3cafa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556a3c786740 con 0x556a3c6e34d0 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a3c9bb970 con 0x556a3c6eb6b0 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556a3cade1c0 con 0x556a3c70c390 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 0x556a3caefdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f491da02740 10 client.0 _ll_register_callbacks cb 0x556a3c9bebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 0x556a3caefdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48918/0 (socket says 172.21.15.22:48918) 2024-09-21T05:42:12.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 -- 172.21.15.22:0/1505555070 learned_addr learned my addr 172.21.15.22:0/1505555070 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4915031640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c70c390 0x556a3cafa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f490ffff640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6eb6b0 0x556a3cacbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 -- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c70c390 msgr2=0x556a3cafa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c70c390 0x556a3cafa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 -- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6eb6b0 msgr2=0x556a3cacbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6eb6b0 0x556a3cacbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.091+0000 7f4914830640 1 -- 172.21.15.22:0/1505555070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3c777970 con 0x556a3c6e34d0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f4914830640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 0x556a3caefdd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4900009780 tx=0x7f4900005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6891d2b4d8faf92e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f48ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f48ef7fe640 1 -- 172.21.15.22:0/1505555070 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900005420 con 0x556a3c6e34d0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f48ef7fe640 1 -- 172.21.15.22:0/1505555070 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f49000055c0 con 0x556a3c6e34d0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4900003e50 con 0x556a3c6e34d0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f48ef7fe640 1 -- 172.21.15.22:0/1505555070 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f49000047a0 con 0x556a3c6e34d0 2024-09-21T05:42:12.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 10 client.5901 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556a3c7556a0 con 0x556a3c6e34d0 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f48ef7fe640 1 -- 172.21.15.22:0/1505555070 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4900003e50 con 0x556a3c6e34d0 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f48ef7fe640 1 -- 172.21.15.22:0/1505555070 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f490004e070 con 0x556a3c6e34d0 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 10 client.5901 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 10 client.5901 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 20 client.5901 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3c6caf60 0x556a3cb0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 4 client.5901 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.092+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a3ca289b0 con 0x556a3c6caf60 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.093+0000 7f490ffff640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3c6caf60 0x556a3cb0ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.093+0000 7f48ef7fe640 10 client.5901 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.093+0000 7f490ffff640 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3c6caf60 0x556a3cb0ada0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ba9d43506af2668 server_cookie=ee6834c93d25e3c8 in_seq=0 out_seq=0 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f48ef7fe640 1 -- 172.21.15.22:0/1505555070 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a3ca289b0 con 0x556a3c6caf60 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f48ef7fe640 10 client.5901 handle_command_reply: tid=0 2024-09-21T05:42:12.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 client.5901 shutdown 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 msgr2=0x556a3caefdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 0x556a3caefdd0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4900009780 tx=0x7f4900005020 comp rx=0 tx=0).stop 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 shutdown_connections 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3c6caf60 0x556a3cb0ada0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c70c390 0x556a3cafa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c6eb6b0 0x556a3cacbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f48ef7fe640 0 client.5901 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 --2- 172.21.15.22:0/1505555070 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6e34d0 0x556a3caefdd0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 >> 172.21.15.22:0/1505555070 conn(0x556a3c7004e0 msgr2=0x556a3cafa150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 shutdown_connections 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 1 -- 172.21.15.22:0/1505555070 wait complete. 2024-09-21T05:42:12.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.094+0000 7f491da02740 20 client.5901 trim_cache size 0 max 16384 2024-09-21T05:42:12.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.112+0000 7f491da02740 1 Processor -- start 2024-09-21T05:42:12.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.112+0000 7f491da02740 1 -- start start 2024-09-21T05:42:12.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3caf41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f491da02740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3caf46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3cacb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f491da02740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49101ffe00 con 0x556a3c6eb6b0 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f491015e5e0 con 0x556a3c70c390 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49101f7680 con 0x556a3c6e34d0 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3caf46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3caf46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48930/0 (socket says 172.21.15.22:48930) 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 -- 172.21.15.22:0/367737880 learned_addr learned my addr 172.21.15.22:0/367737880 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f4915031640 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3cacb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f4914830640 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3caf41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 -- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 msgr2=0x556a3caf41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3caf41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 -- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 msgr2=0x556a3cacb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3cacb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 -- 172.21.15.22:0/367737880 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3caee9c0 con 0x556a3c6eb6b0 2024-09-21T05:42:12.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f490ffff640 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3caf46e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f48f8061d40 tx=0x7f48f8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a637432e535cacfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f4905ffb640 1 -- 172.21.15.22:0/367737880 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f80667e0 con 0x556a3c6eb6b0 2024-09-21T05:42:12.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f4905ffb640 1 -- 172.21.15.22:0/367737880 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f806b720 con 0x556a3c6eb6b0 2024-09-21T05:42:12.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.114+0000 7f4905ffb640 1 -- 172.21.15.22:0/367737880 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f806b290 con 0x556a3c6eb6b0 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 -- 172.21.15.22:0/367737880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 msgr2=0x556a3caf46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3caf46e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f48f8061d40 tx=0x7f48f8086040 comp rx=0 tx=0).stop 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 -- 172.21.15.22:0/367737880 shutdown_connections 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3cacb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3caf46e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 --2- 172.21.15.22:0/367737880 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3caf41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 -- 172.21.15.22:0/367737880 >> 172.21.15.22:0/367737880 conn(0x556a3c7004e0 msgr2=0x556a3caa95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 -- 172.21.15.22:0/367737880 shutdown_connections 2024-09-21T05:42:12.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.115+0000 7f491da02740 1 -- 172.21.15.22:0/367737880 wait complete. 2024-09-21T05:42:12.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.116+0000 7f491da02740 1 -- start start 2024-09-21T05:42:12.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.116+0000 7f491da02740 1 Processor -- start 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3cacbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3cb1b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3caf84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f491015df90 con 0x556a3c6eb6b0 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556a3caf2190 con 0x556a3c70c390 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556a3cb7c120 con 0x556a3c6e34d0 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f491da02740 10 client.0 _ll_register_callbacks cb 0x7f48f8001120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3cb1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3cb1b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48932/0 (socket says 172.21.15.22:48932) 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 -- 172.21.15.22:0/72805068 learned_addr learned my addr 172.21.15.22:0/72805068 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f4915031640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3caf84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f4914830640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3cacbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 -- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 msgr2=0x556a3cacbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3cacbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 -- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 msgr2=0x556a3caf84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3caf84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 -- 172.21.15.22:0/72805068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a3cacbb60 con 0x556a3c6eb6b0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f490ffff640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3cb1b600 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f48f800cdb0 tx=0x7f48f8000ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f6106f6b8c3833 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.118+0000 7f48ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f48ef7fe640 1 -- 172.21.15.22:0/72805068 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8071060 con 0x556a3c6eb6b0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f48ef7fe640 1 -- 172.21.15.22:0/72805068 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f48f806b490 con 0x556a3c6eb6b0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f806b720 con 0x556a3c6eb6b0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f48ef7fe640 1 -- 172.21.15.22:0/72805068 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f48f80667e0 con 0x556a3c6eb6b0 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 10 client.5907 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:12.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x556a3cb619b0 con 0x556a3c6eb6b0 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f48ef7fe640 1 -- 172.21.15.22:0/72805068 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48f806b720 con 0x556a3c6eb6b0 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f48ef7fe640 1 -- 172.21.15.22:0/72805068 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f48f807e020 con 0x556a3c6eb6b0 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 10 client.5907 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 10 client.5907 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 20 client.5907 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3cb7b9b0 0x556a3caf8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 4 client.5907 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556a3ca24460 con 0x556a3cb7b9b0 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.119+0000 7f4914830640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3cb7b9b0 0x556a3caf8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f48ef7fe640 10 client.5907 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f4914830640 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3cb7b9b0 0x556a3caf8a00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8bf4a06aa386d6 server_cookie=334e41a7e03c41a in_seq=0 out_seq=0 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f48ef7fe640 1 -- 172.21.15.22:0/72805068 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x556a3ca24460 con 0x556a3cb7b9b0 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f48ef7fe640 10 client.5907 handle_command_reply: tid=0 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 client.5907 shutdown 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 msgr2=0x556a3cb1b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3cb1b600 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f48f800cdb0 tx=0x7f48f8000ef0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 shutdown_connections 2024-09-21T05:42:12.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x556a3cb7b9b0 0x556a3caf8a00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556a3c70c390 0x556a3caf84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556a3c6eb6b0 0x556a3cb1b600 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f48ef7fe640 0 client.5907 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 --2- 172.21.15.22:0/72805068 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556a3c6e34d0 0x556a3cacbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 >> 172.21.15.22:0/72805068 conn(0x556a3c7004e0 msgr2=0x556a3caae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 shutdown_connections 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 1 -- 172.21.15.22:0/72805068 wait complete. 2024-09-21T05:42:12.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.120+0000 7f491da02740 20 client.5907 trim_cache size 0 max 16384 2024-09-21T05:42:12.129 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:12.129 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:42:12.129 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:42:12.129 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:42:12.129 INFO:teuthology.orchestra.run.smithi022.stdout: "eb94107a-f919-4784-b8a0-9ecc5d24cea9": { 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:41:41.755993+0000", 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:12.130 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:12.131 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:12.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f48f0078b90 msgr2=0x7f48f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f48f0078b90 0x7f48f007b0a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4900034000 tx=0x7f4900004730 comp rx=0 tx=0).stop 2024-09-21T05:42:12.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 msgr2=0x7f49101573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49101573c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f48f800c3f0 tx=0x7f48f800c8b0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 shutdown_connections 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f48f0078b90 0x7f48f007b0a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f491015c6f0 0x7f4910157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49100d1d40 0x7f49101573c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 --2- 172.21.15.22:0/1792182803 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49100ccba0 0x7f4910156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 >> 172.21.15.22:0/1792182803 conn(0x7f49100b3f10 msgr2=0x7f49100b5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 shutdown_connections 2024-09-21T05:42:12.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.121+0000 7f4916a94640 1 -- 172.21.15.22:0/1792182803 wait complete. 2024-09-21T05:42:12.134 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:42:12.135 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.249+0000 7f9917e27640 1 -- 172.21.15.22:0/3523811022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100b3a60 msgr2=0x7f99100cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.249+0000 7f9917e27640 1 --2- 172.21.15.22:0/3523811022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100b3a60 0x7f99100cfb70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f990c005fd0 tx=0x7f990c02a5d0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.250+0000 7f9917e27640 1 -- 172.21.15.22:0/3523811022 shutdown_connections 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.250+0000 7f9917e27640 1 --2- 172.21.15.22:0/3523811022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f99100d0e10 0x7f99100d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.250+0000 7f9917e27640 1 --2- 172.21.15.22:0/3523811022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100b3a60 0x7f99100cfb70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.250+0000 7f9917e27640 1 -- 172.21.15.22:0/3523811022 >> 172.21.15.22:0/3523811022 conn(0x7f991008bfc0 msgr2=0x7f991009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.250+0000 7f9917e27640 1 -- 172.21.15.22:0/3523811022 shutdown_connections 2024-09-21T05:42:12.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.250+0000 7f9917e27640 1 -- 172.21.15.22:0/3523811022 wait complete. 2024-09-21T05:42:12.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.251+0000 7f9917e27640 1 Processor -- start 2024-09-21T05:42:12.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.251+0000 7f9917e27640 1 -- start start 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9917e27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f991006e3c0 0x7f99100b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9917e27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 0x7f99101f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9917e27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100d0e10 0x7f9910155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9917e27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f99100d2cc0 con 0x7f99100b3a60 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9917e27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f99100d2b40 con 0x7f99100d0e10 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9917e27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f99100d2e40 con 0x7f991006e3c0 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 0x7f99101f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 0x7f99101f5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48940/0 (socket says 172.21.15.22:48940) 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 -- 172.21.15.22:0/641323416 learned_addr learned my addr 172.21.15.22:0/641323416 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99163c4640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100d0e10 0x7f9910155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f9915bc3640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f991006e3c0 0x7f99100b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 -- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f991006e3c0 msgr2=0x7f99100b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f991006e3c0 0x7f99100b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 -- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100d0e10 msgr2=0x7f9910155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100d0e10 0x7f9910155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.253+0000 7f99153c2640 1 -- 172.21.15.22:0/641323416 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99100b3f60 con 0x7f99100b3a60 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.254+0000 7f99153c2640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 0x7f99101f5760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f990000c3a0 tx=0x7f990000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.254+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900017070 con 0x7f99100b3a60 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.254+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99101f6020 con 0x7f99100b3a60 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.254+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99000052d0 con 0x7f99100b3a60 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.254+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9900003850 con 0x7f99100b3a60 2024-09-21T05:42:12.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.254+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99101f65e0 con 0x7f99100b3a60 2024-09-21T05:42:12.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.255+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99100d3360 con 0x7f99100b3a60 2024-09-21T05:42:12.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.256+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f990001a070 con 0x7f99100b3a60 2024-09-21T05:42:12.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.256+0000 7f99067fc640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f98f4078bc0 0x7f98f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.256+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f9900099ef0 con 0x7f99100b3a60 2024-09-21T05:42:12.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.256+0000 7f9915bc3640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f98f4078bc0 0x7f98f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.257+0000 7f9915bc3640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f98f4078bc0 0x7f98f407b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f990c033000 tx=0x7f990c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.261+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9900061c60 con 0x7f99100b3a60 2024-09-21T05:42:12.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.489+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99100b3580 con 0x7f99100b3a60 2024-09-21T05:42:12.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.489+0000 7f99067fc640 1 -- 172.21.15.22:0/641323416 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5226 (secure 0 0 0) 0x7f9900066690 con 0x7f99100b3a60 2024-09-21T05:42:12.492 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:12.492 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"btime":"2024-09-21T05:41:45:826771+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:41:45.657650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:42:12.492 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f98f4078bc0 msgr2=0x7f98f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f98f4078bc0 0x7f98f407b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f990c033000 tx=0x7f990c004730 comp rx=0 tx=0).stop 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 msgr2=0x7f99101f5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 0x7f99101f5760 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f990000c3a0 tx=0x7f990000c7a0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 shutdown_connections 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f99100d0e10 0x7f9910155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f99100b3a60 0x7f99101f5760 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f98f4078bc0 0x7f98f407b0d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 --2- 172.21.15.22:0/641323416 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f991006e3c0 0x7f99100b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.491+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 >> 172.21.15.22:0/641323416 conn(0x7f991008bfc0 msgr2=0x7f991006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.492+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 shutdown_connections 2024-09-21T05:42:12.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.492+0000 7f9917e27640 1 -- 172.21.15.22:0/641323416 wait complete. 2024-09-21T05:42:12.505 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T05:42:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.621+0000 7fb794d72640 1 -- 172.21.15.22:0/3268303779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900b66b0 msgr2=0x7fb7900d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.622+0000 7fb794d72640 1 --2- 172.21.15.22:0/3268303779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900b66b0 0x7fb7900d4d80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb7900d00e0 tx=0x7fb78402a410 comp rx=0 tx=0).stop 2024-09-21T05:42:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.622+0000 7fb794d72640 1 -- 172.21.15.22:0/3268303779 shutdown_connections 2024-09-21T05:42:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.622+0000 7fb794d72640 1 --2- 172.21.15.22:0/3268303779 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900b66b0 0x7fb7900d4d80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.622+0000 7fb794d72640 1 -- 172.21.15.22:0/3268303779 >> 172.21.15.22:0/3268303779 conn(0x7fb79009d7f0 msgr2=0x7fb7900d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.622+0000 7fb794d72640 1 -- 172.21.15.22:0/3268303779 shutdown_connections 2024-09-21T05:42:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.622+0000 7fb794d72640 1 -- 172.21.15.22:0/3268303779 wait complete. 2024-09-21T05:42:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.624+0000 7fb794d72640 1 Processor -- start 2024-09-21T05:42:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.624+0000 7fb794d72640 1 -- start start 2024-09-21T05:42:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb794d72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7900b9b60 0x7fb7900a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb794d72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 0x7fb79015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb794d72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb79015d0e0 0x7fb7900b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb794d72640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb79006f7f0 con 0x7fb7900cf540 2024-09-21T05:42:12.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb794d72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb79006f670 con 0x7fb7900b9b60 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb794d72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb79006f970 con 0x7fb79015d0e0 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 0x7fb79015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 0x7fb79015c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48960/0 (socket says 172.21.15.22:48960) 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 -- 172.21.15.22:0/2018888867 learned_addr learned my addr 172.21.15.22:0/2018888867 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78e59c640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7900b9b60 0x7fb7900a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb79015d0e0 0x7fb7900b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 -- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb79015d0e0 msgr2=0x7fb7900b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb79015d0e0 0x7fb7900b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 -- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7900b9b60 msgr2=0x7fb7900a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7900b9b60 0x7fb7900a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.625+0000 7fb78dd9b640 1 -- 172.21.15.22:0/2018888867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79015ce40 con 0x7fb7900cf540 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.626+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 0x7fb79015c900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb78402aa00 tx=0x7fb784033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.626+0000 7fb77effd640 1 -- 172.21.15.22:0/2018888867 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784031d60 con 0x7fb7900cf540 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.626+0000 7fb77effd640 1 -- 172.21.15.22:0/2018888867 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb784005020 con 0x7fb7900cf540 2024-09-21T05:42:12.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.626+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7901f5dc0 con 0x7fb7900cf540 2024-09-21T05:42:12.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.626+0000 7fb77effd640 1 -- 172.21.15.22:0/2018888867 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784005470 con 0x7fb7900cf540 2024-09-21T05:42:12.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.626+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7901f6380 con 0x7fb7900cf540 2024-09-21T05:42:12.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.628+0000 7fb77effd640 1 -- 172.21.15.22:0/2018888867 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7840047a0 con 0x7fb7900cf540 2024-09-21T05:42:12.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.628+0000 7fb77effd640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb770078bc0 0x7fb77007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.628+0000 7fb77effd640 1 -- 172.21.15.22:0/2018888867 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb7840bbf10 con 0x7fb7900cf540 2024-09-21T05:42:12.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.628+0000 7fb78e59c640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb770078bc0 0x7fb77007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.629+0000 7fb78e59c640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb770078bc0 0x7fb77007b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb780001710 tx=0x7fb780009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.629+0000 7fb79bdc1740 1 Processor -- start 2024-09-21T05:42:12.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.629+0000 7fb79bdc1740 1 -- start start 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8f2350 0x55bc1fcc3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 0x55bc1fce11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc1fc0c9a0 con 0x55bc1f8d1670 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc1fbfbdd0 con 0x55bc1f8c9490 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bc1fbffbe0 con 0x55bc1f8f2350 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 0x55bc1fce11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 0x55bc1fce11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48976/0 (socket says 172.21.15.22:48976) 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 -- 172.21.15.22:0/2518282262 learned_addr learned my addr 172.21.15.22:0/2518282262 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78e59c640 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8f2350 0x55bc1fcc3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 -- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8f2350 msgr2=0x55bc1fcc3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8f2350 0x55bc1fcc3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 -- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 msgr2=0x55bc1fcc42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 -- 172.21.15.22:0/2518282262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1fb94700 con 0x55bc1f8d1670 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.631+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 0x55bc1fce11c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b450 tx=0x7fb788009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d1e8a4d8ba1724 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb76ffff640 1 -- 172.21.15.22:0/2518282262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788003970 con 0x55bc1f8d1670 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb76ffff640 1 -- 172.21.15.22:0/2518282262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb788003b30 con 0x55bc1f8d1670 2024-09-21T05:42:12.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb76ffff640 1 -- 172.21.15.22:0/2518282262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78800a710 con 0x55bc1f8d1670 2024-09-21T05:42:12.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 -- 172.21.15.22:0/2518282262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 msgr2=0x55bc1fce11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 0x55bc1fce11c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b450 tx=0x7fb788009630 comp rx=0 tx=0).stop 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 -- 172.21.15.22:0/2518282262 shutdown_connections 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8d1670 0x55bc1fce11c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/2518282262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8f2350 0x55bc1fcc3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.632+0000 7fb79bdc1740 1 -- 172.21.15.22:0/2518282262 >> 172.21.15.22:0/2518282262 conn(0x55bc1f8e64a0 msgr2=0x55bc1fc9a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.633+0000 7fb79bdc1740 1 -- 172.21.15.22:0/2518282262 shutdown_connections 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.633+0000 7fb79bdc1740 1 -- 172.21.15.22:0/2518282262 wait complete. 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.634+0000 7fb79bdc1740 1 -- start start 2024-09-21T05:42:12.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.634+0000 7fb79bdc1740 1 Processor -- start 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcf1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8d1670 0x55bc1fc936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8f2350 0x55bc1fcb2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc1f96c700 con 0x55bc1f8c9490 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc1fba1340 con 0x55bc1f8f2350 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bc1fcc4800 con 0x55bc1f8d1670 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb79bdc1740 10 client.0 _ll_register_callbacks cb 0x55bc1fb9fee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcf1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcf1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48980/0 (socket says 172.21.15.22:48980) 2024-09-21T05:42:12.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 -- 172.21.15.22:0/442866300 learned_addr learned my addr 172.21.15.22:0/442866300 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8d1670 0x55bc1fc936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8f2350 0x55bc1fcb2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 -- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8d1670 msgr2=0x55bc1fc936e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8d1670 0x55bc1fc936e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 -- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8f2350 msgr2=0x55bc1fcb2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8f2350 0x55bc1fcb2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 -- 172.21.15.22:0/442866300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1f9d67c0 con 0x55bc1f8c9490 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb78e59c640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcf1e00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb780014700 tx=0x7fb780000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4528ac21cdd833b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb76d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.636+0000 7fb76d7fa640 1 -- 172.21.15.22:0/442866300 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780025020 con 0x55bc1f8c9490 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb788003b30 con 0x55bc1f8c9490 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb76d7fa640 1 -- 172.21.15.22:0/442866300 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb780013740 con 0x55bc1f8c9490 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb76d7fa640 1 -- 172.21.15.22:0/442866300 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb78002f050 con 0x55bc1f8c9490 2024-09-21T05:42:12.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 10 client.5925 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55bc1fa36f50 con 0x55bc1f8c9490 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb76d7fa640 1 -- 172.21.15.22:0/442866300 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb788003b30 con 0x55bc1f8c9490 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb76d7fa640 1 -- 172.21.15.22:0/442866300 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fb780032020 con 0x55bc1f8c9490 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 10 client.5925 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 10 client.5925 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 20 client.5925 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8b1130 0x55bc1fcda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 4 client.5925 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc1fc0e8f0 con 0x55bc1f8b1130 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.637+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8b1130 0x55bc1fcda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.638+0000 7fb76d7fa640 10 client.5925 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.638+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8b1130 0x55bc1fcda9e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d16b7cf88a1b690 server_cookie=b2d0c3e484995229 in_seq=0 out_seq=0 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb76d7fa640 1 -- 172.21.15.22:0/442866300 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc1fc0e8f0 con 0x55bc1f8b1130 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb76d7fa640 10 client.5925 handle_command_reply: tid=0 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 client.5925 shutdown 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 msgr2=0x55bc1fcf1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcf1e00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb780014700 tx=0x7fb780000f20 comp rx=0 tx=0).stop 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 shutdown_connections 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8b1130 0x55bc1fcda9e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8f2350 0x55bc1fcb2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb76d7fa640 0 client.5925 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8d1670 0x55bc1fc936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/442866300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcf1e00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 >> 172.21.15.22:0/442866300 conn(0x55bc1f8e64a0 msgr2=0x55bc1fc946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 shutdown_connections 2024-09-21T05:42:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 1 -- 172.21.15.22:0/442866300 wait complete. 2024-09-21T05:42:12.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.639+0000 7fb79bdc1740 20 client.5925 trim_cache size 0 max 16384 2024-09-21T05:42:12.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.653+0000 7fb79bdc1740 1 Processor -- start 2024-09-21T05:42:12.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.653+0000 7fb79bdc1740 1 -- start start 2024-09-21T05:42:12.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 0x55bc1fc9a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8b1130 0x55bc1fc944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fc949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb790201550 con 0x55bc1f8e64a0 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb79015fbb0 con 0x55bc1f8c9490 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb790160200 con 0x55bc1f8b1130 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.654+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 0x55bc1fc9a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 0x55bc1fc9a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48992/0 (socket says 172.21.15.22:48992) 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 -- 172.21.15.22:0/1551229950 learned_addr learned my addr 172.21.15.22:0/1551229950 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8b1130 0x55bc1fc944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fc949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 -- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8b1130 msgr2=0x55bc1fc944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8b1130 0x55bc1fc944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 -- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 msgr2=0x55bc1fc949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fc949f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 -- 172.21.15.22:0/1551229950 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc1f93b660 con 0x55bc1f8e64a0 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8b1130 0x55bc1fc944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb78e59c640 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 0x55bc1fc9a790 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb780013cc0 tx=0x7fb780007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41c9e8615671a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb76f7fe640 1 -- 172.21.15.22:0/1551229950 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780025030 con 0x55bc1f8e64a0 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb76f7fe640 1 -- 172.21.15.22:0/1551229950 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb78001b070 con 0x55bc1f8e64a0 2024-09-21T05:42:12.659 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.655+0000 7fb76f7fe640 1 -- 172.21.15.22:0/1551229950 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780007030 con 0x55bc1f8e64a0 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 -- 172.21.15.22:0/1551229950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 msgr2=0x55bc1fc9a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 0x55bc1fc9a790 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb780013cc0 tx=0x7fb780007d50 comp rx=0 tx=0).stop 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 -- 172.21.15.22:0/1551229950 shutdown_connections 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8c9490 0x55bc1fc949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8b1130 0x55bc1fc944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/1551229950 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8e64a0 0x55bc1fc9a790 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 -- 172.21.15.22:0/1551229950 >> 172.21.15.22:0/1551229950 conn(0x55bc1fc8ff80 msgr2=0x55bc1fcd4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 -- 172.21.15.22:0/1551229950 shutdown_connections 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.656+0000 7fb79bdc1740 1 -- 172.21.15.22:0/1551229950 wait complete. 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.657+0000 7fb79bdc1740 1 -- start start 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.657+0000 7fb79bdc1740 1 Processor -- start 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.658+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8b1130 0x55bc1fcb1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8e64a0 0x55bc1fd546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bc1fc93c20 con 0x55bc1f8c9490 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bc1fcdd370 con 0x55bc1f8b1130 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bc1fcdcf90 con 0x55bc1f8e64a0 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 10 client.0 _ll_register_callbacks cb 0x7fb7800013f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48996/0 (socket says 172.21.15.22:48996) 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 -- 172.21.15.22:0/3510498727 learned_addr learned my addr 172.21.15.22:0/3510498727 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78ed9d640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8e64a0 0x55bc1fd546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78e59c640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8b1130 0x55bc1fcb1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 -- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8e64a0 msgr2=0x55bc1fd546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8e64a0 0x55bc1fd546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 -- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8b1130 msgr2=0x55bc1fcb1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8b1130 0x55bc1fcb1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 -- 172.21.15.22:0/3510498727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc1fcd5250 con 0x55bc1f8c9490 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb78dd9b640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc3e60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb784009230 tx=0x7fb7840a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f92f70dd4e7f616 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb76d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb76d7fa640 1 -- 172.21.15.22:0/3510498727 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784048040 con 0x55bc1f8c9490 2024-09-21T05:42:12.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb76d7fa640 1 -- 172.21.15.22:0/3510498727 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb78407fef0 con 0x55bc1f8c9490 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.659+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78001b070 con 0x55bc1f8c9490 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb76d7fa640 1 -- 172.21.15.22:0/3510498727 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb7840886b0 con 0x55bc1f8c9490 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 10 client.5931 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55bc1fd61d90 con 0x55bc1f8c9490 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb76d7fa640 1 -- 172.21.15.22:0/3510498727 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb78001b070 con 0x55bc1f8c9490 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb76d7fa640 1 -- 172.21.15.22:0/3510498727 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fb784083ee0 con 0x55bc1f8c9490 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 10 client.5931 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 10 client.5931 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 20 client.5931 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8f2350 0x55bc1fcdbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 4 client.5931 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:42:12.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55bc1fc0a3a0 con 0x55bc1f8f2350 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb78e59c640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8f2350 0x55bc1fcdbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.660+0000 7fb76d7fa640 10 client.5931 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb78e59c640 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8f2350 0x55bc1fcdbb00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cc87ad788d07a07 server_cookie=18e6f0f2e00e5a70 in_seq=0 out_seq=0 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb76d7fa640 1 -- 172.21.15.22:0/3510498727 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x55bc1fc0a3a0 con 0x55bc1f8f2350 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb76d7fa640 10 client.5931 handle_command_reply: tid=0 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 client.5931 shutdown 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 msgr2=0x55bc1fcc3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc3e60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb784009230 tx=0x7fb7840a6040 comp rx=0 tx=0).stop 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 shutdown_connections 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55bc1f8f2350 0x55bc1fcdbb00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bc1f8e64a0 0x55bc1fd546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bc1f8c9490 0x55bc1fcc3e60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 --2- 172.21.15.22:0/3510498727 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bc1f8b1130 0x55bc1fcb1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb76d7fa640 0 client.5931 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 >> 172.21.15.22:0/3510498727 conn(0x55bc1fc8ff80 msgr2=0x55bc1fc9a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 shutdown_connections 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.661+0000 7fb79bdc1740 1 -- 172.21.15.22:0/3510498727 wait complete. 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb79bdc1740 20 client.5931 trim_cache size 0 max 16384 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stdout: "damage_type": "backtrace", 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 1036974415, 2024-09-21T05:42:12.669 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627777, 2024-09-21T05:42:12.670 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir_test/file" 2024-09-21T05:42:12.670 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:12.670 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb770078bc0 msgr2=0x7fb77007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb770078bc0 0x7fb77007b0d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb780001710 tx=0x7fb780009040 comp rx=0 tx=0).stop 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 msgr2=0x7fb79015c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 0x7fb79015c900 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb78402aa00 tx=0x7fb784033ce0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 shutdown_connections 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb770078bc0 0x7fb77007b0d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb79015d0e0 0x7fb7900b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7900cf540 0x7fb79015c900 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 --2- 172.21.15.22:0/2018888867 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7900b9b60 0x7fb7900a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 >> 172.21.15.22:0/2018888867 conn(0x7fb79009d7f0 msgr2=0x7fb7901619b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 shutdown_connections 2024-09-21T05:42:12.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.662+0000 7fb794d72640 1 -- 172.21.15.22:0/2018888867 wait complete. 2024-09-21T05:42:12.675 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:42:12.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/1113323164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee006e0e0 msgr2=0x7f3ee006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/1113323164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee006e0e0 0x7f3ee006e4f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8005fd0 tx=0x7f3ed802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/1113323164 shutdown_connections 2024-09-21T05:42:12.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/1113323164 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee006e0e0 0x7f3ee006e4f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/1113323164 >> v2:172.21.15.134:3300/0 conn(0x7f3ee00b9970 0x7f3ee00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/1113323164 >> 172.21.15.22:0/1113323164 conn(0x7f3ee009d7f0 msgr2=0x7f3ee00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:12.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/1113323164 shutdown_connections 2024-09-21T05:42:12.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.789+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/1113323164 wait complete. 2024-09-21T05:42:12.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.791+0000 7f3ee6a8c640 1 Processor -- start 2024-09-21T05:42:12.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.791+0000 7f3ee6a8c640 1 -- start start 2024-09-21T05:42:12.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee6a8c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 0x7f3ee01588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee6a8c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee00b5fa0 0x7f3ee00ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee6a8c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee00b9970 0x7f3ee00b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee6a8c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee006ea30 con 0x7f3ee006e0e0 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee6a8c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3ee00d5010 con 0x7f3ee00b9970 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee6a8c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee006ebb0 con 0x7f3ee00b5fa0 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee4828640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 0x7f3ee01588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee4828640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 0x7f3ee01588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49004/0 (socket says 172.21.15.22:49004) 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee4828640 1 -- 172.21.15.22:0/2713103373 learned_addr learned my addr 172.21.15.22:0/2713103373 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3edffff640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee00b5fa0 0x7f3ee00ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.792+0000 7f3ee5029640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee00b9970 0x7f3ee00b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee4828640 1 -- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee00b5fa0 msgr2=0x7f3ee00ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee4828640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee00b5fa0 0x7f3ee00ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee4828640 1 -- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee00b9970 msgr2=0x7f3ee00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee4828640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee00b9970 0x7f3ee00b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee4828640 1 -- 172.21.15.22:0/2713103373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee01f8f90 con 0x7f3ee006e0e0 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee4828640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 0x7f3ee01588e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400e7f0 tx=0x7f3ed400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed400ccc0 con 0x7f3ee006e0e0 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed4010430 con 0x7f3ee006e0e0 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ee01f9270 con 0x7f3ee006e0e0 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4010840 con 0x7f3ee006e0e0 2024-09-21T05:42:12.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.793+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ee01f9830 con 0x7f3ee006e0e0 2024-09-21T05:42:12.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.794+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ee00b3710 con 0x7f3ee006e0e0 2024-09-21T05:42:12.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.795+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ed40109e0 con 0x7f3ee006e0e0 2024-09-21T05:42:12.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.795+0000 7f3edd7fa640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ec4078bc0 0x7f3ec407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:12.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.795+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f3ed409b6d0 con 0x7f3ee006e0e0 2024-09-21T05:42:12.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.796+0000 7f3edffff640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ec4078bc0 0x7f3ec407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:12.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.796+0000 7f3edffff640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ec4078bc0 0x7f3ec407b0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc007fd0 tx=0x7f3ecc001370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:12.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:12.800+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ed40634c0 con 0x7f3ee006e0e0 2024-09-21T05:42:13.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.026+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3ee01e4000 con 0x7f3ee006e0e0 2024-09-21T05:42:13.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.026+0000 7f3edd7fa640 1 -- 172.21.15.22:0/2713103373 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f3ed4067ef0 con 0x7f3ee006e0e0 2024-09-21T05:42:13.029 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:13.029 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.028+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ec4078bc0 msgr2=0x7f3ec407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.028+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ec4078bc0 0x7f3ec407b0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc007fd0 tx=0x7f3ecc001370 comp rx=0 tx=0).stop 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.028+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 msgr2=0x7f3ee01588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.028+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 0x7f3ee01588e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400e7f0 tx=0x7f3ed400ecb0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 shutdown_connections 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3ee00b9970 0x7f3ee00b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ec4078bc0 0x7f3ec407b0d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ee00b5fa0 0x7f3ee00ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 --2- 172.21.15.22:0/2713103373 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3ee006e0e0 0x7f3ee01588e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 >> 172.21.15.22:0/2713103373 conn(0x7f3ee009d7f0 msgr2=0x7f3ee00cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 shutdown_connections 2024-09-21T05:42:13.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.029+0000 7f3ee6a8c640 1 -- 172.21.15.22:0/2713103373 wait complete. 2024-09-21T05:42:13.042 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T05:42:13.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:13.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-09-21T05:42:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 -- 172.21.15.22:0/2313883537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f580d1640 msgr2=0x7f7f5806e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 --2- 172.21.15.22:0/2313883537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f580d1640 0x7f7f5806e980 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009860 tx=0x7f7f4c02a4b0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 -- 172.21.15.22:0/2313883537 shutdown_connections 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 --2- 172.21.15.22:0/2313883537 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f580b38a0 0x7f7f5806eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 --2- 172.21.15.22:0/2313883537 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f580d1640 0x7f7f5806e980 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 -- 172.21.15.22:0/2313883537 >> 172.21.15.22:0/2313883537 conn(0x7f7f5809d820 msgr2=0x7f7f5808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 -- 172.21.15.22:0/2313883537 shutdown_connections 2024-09-21T05:42:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.161+0000 7f7f5eece640 1 -- 172.21.15.22:0/2313883537 wait complete. 2024-09-21T05:42:13.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.163+0000 7f7f5eece640 1 Processor -- start 2024-09-21T05:42:13.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.163+0000 7f7f5eece640 1 -- start start 2024-09-21T05:42:13.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5eece640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 0x7f7f581ea080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5eece640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f580b38a0 0x7f7f580b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5eece640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f581f8910 0x7f7f581f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5eece640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5806f790 con 0x7f7f5806ded0 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5eece640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5806f610 con 0x7f7f580b38a0 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5eece640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5806f910 con 0x7f7f581f8910 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.164+0000 7f7f5cc6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 0x7f7f581ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 0x7f7f581ea080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49034/0 (socket says 172.21.15.22:49034) 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/922138549 learned_addr learned my addr 172.21.15.22:0/922138549 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f581f8910 0x7f7f581f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f57fff640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f580b38a0 0x7f7f580b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f581f8910 msgr2=0x7f7f581f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f581f8910 0x7f7f581f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f580b38a0 msgr2=0x7f7f580b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f580b38a0 0x7f7f580b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/922138549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f581ea8b0 con 0x7f7f5806ded0 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 0x7f7f581ea080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c3f0 tx=0x7f7f4800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f557fa640 1 -- 172.21.15.22:0/922138549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48017070 con 0x7f7f5806ded0 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f557fa640 1 -- 172.21.15.22:0/922138549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f480035f0 con 0x7f7f5806ded0 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f581eab90 con 0x7f7f5806ded0 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f557fa640 1 -- 172.21.15.22:0/922138549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48003a40 con 0x7f7f5806ded0 2024-09-21T05:42:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.165+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f580b8f70 con 0x7f7f5806ded0 2024-09-21T05:42:13.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.167+0000 7f7f557fa640 1 -- 172.21.15.22:0/922138549 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7f4801a070 con 0x7f7f5806ded0 2024-09-21T05:42:13.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.168+0000 7f7f557fa640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7f44078bc0 0x7f7f4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.168+0000 7f7f557fa640 1 -- 172.21.15.22:0/922138549 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7f4809a060 con 0x7f7f5806ded0 2024-09-21T05:42:13.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.168+0000 7f7f57fff640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7f44078bc0 0x7f7f4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.168+0000 7f7f57fff640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7f44078bc0 0x7f7f4407b0d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c034000 tx=0x7f7f4c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.169+0000 7f7f65e37740 1 Processor -- start 2024-09-21T05:42:13.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.169+0000 7f7f65e37740 1 -- start start 2024-09-21T05:42:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b3704d0 0x56041b75d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b7327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 0x56041b732d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f65e37740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56041b68aa30 con 0x56041b34f7f0 2024-09-21T05:42:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56041b679e60 con 0x56041b347610 2024-09-21T05:42:13.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56041b67dc70 con 0x56041b3704d0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 0x56041b732d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 0x56041b732d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49036/0 (socket says 172.21.15.22:49036) 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 -- 172.21.15.22:0/962936319 learned_addr learned my addr 172.21.15.22:0/962936319 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b3704d0 0x56041b75d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f57fff640 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b7327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 -- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b3704d0 msgr2=0x56041b75d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b3704d0 0x56041b75d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 -- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 msgr2=0x56041b7327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b7327f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.170+0000 7f7f5d46b640 1 -- 172.21.15.22:0/962936319 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56041b620190 con 0x56041b34f7f0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.171+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 0x56041b732d30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000ae70 tx=0x7f7f5000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f677db8eaeeb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.171+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/962936319 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50004d20 con 0x56041b34f7f0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.171+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/962936319 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f50011650 con 0x56041b34f7f0 2024-09-21T05:42:13.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.171+0000 7f7f3e7fc640 1 -- 172.21.15.22:0/962936319 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50011a80 con 0x56041b34f7f0 2024-09-21T05:42:13.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 -- 172.21.15.22:0/962936319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 msgr2=0x56041b732d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 0x56041b732d30 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000ae70 tx=0x7f7f5000f730 comp rx=0 tx=0).stop 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 -- 172.21.15.22:0/962936319 shutdown_connections 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b34f7f0 0x56041b732d30 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b7327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 --2- 172.21.15.22:0/962936319 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b3704d0 0x56041b75d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 -- 172.21.15.22:0/962936319 >> 172.21.15.22:0/962936319 conn(0x56041b364620 msgr2=0x56041b718c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 -- 172.21.15.22:0/962936319 shutdown_connections 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.172+0000 7f7f65e37740 1 -- 172.21.15.22:0/962936319 wait complete. 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.173+0000 7f7f65e37740 1 -- start start 2024-09-21T05:42:13.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.173+0000 7f7f65e37740 1 Processor -- start 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b73fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b757490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b76fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56041b3ea880 con 0x56041b3704d0 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56041b6005a0 con 0x56041b347610 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56041b742cf0 con 0x56041b34f7f0 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f65e37740 10 client.0 _ll_register_callbacks cb 0x56041b62ea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f5d46b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b76fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f5d46b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b76fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49050/0 (socket says 172.21.15.22:49050) 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.174+0000 7f7f5d46b640 1 -- 172.21.15.22:0/2180213607 learned_addr learned my addr 172.21.15.22:0/2180213607 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f57fff640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b757490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b73fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5d46b640 1 -- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 msgr2=0x56041b757490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b757490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5d46b640 1 -- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 msgr2=0x56041b73fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b73fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5d46b640 1 -- 172.21.15.22:0/2180213607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56041b454940 con 0x56041b3704d0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f57fff640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b757490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b76fc00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000fc30 tx=0x7f7f5000ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0607224564c6e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f377fe640 1 -- 172.21.15.22:0/2180213607 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50004ce0 con 0x56041b3704d0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f377fe640 1 -- 172.21.15.22:0/2180213607 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7f50003b80 con 0x56041b3704d0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56041b650420 con 0x56041b3704d0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f377fe640 1 -- 172.21.15.22:0/2180213607 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7f50003dc0 con 0x56041b3704d0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f65e37740 10 client.5952 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x56041b4b50d0 con 0x56041b3704d0 2024-09-21T05:42:13.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.175+0000 7f7f377fe640 1 -- 172.21.15.22:0/2180213607 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56041b650420 con 0x56041b3704d0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f377fe640 1 -- 172.21.15.22:0/2180213607 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f7f50011650 con 0x56041b3704d0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f65e37740 10 client.5952 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f65e37740 10 client.5952 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f65e37740 20 client.5952 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f65e37740 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b32f2b0 0x56041b7419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f65e37740 4 client.5952 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56041b68c980 con 0x56041b32f2b0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b32f2b0 0x56041b7419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f377fe640 10 client.5952 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.176+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b32f2b0 0x56041b7419b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99afd7aae833419e server_cookie=e68748e45d6b7c81 in_seq=0 out_seq=0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f377fe640 1 -- 172.21.15.22:0/2180213607 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56041b68c980 con 0x56041b32f2b0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f377fe640 10 client.5952 handle_command_reply: tid=0 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 client.5952 shutdown 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 msgr2=0x56041b76fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b76fc00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000fc30 tx=0x7f7f5000ab80 comp rx=0 tx=0).stop 2024-09-21T05:42:13.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 shutdown_connections 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b32f2b0 0x56041b7419b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b76fc00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b757490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 --2- 172.21.15.22:0/2180213607 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b347610 0x56041b73fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f377fe640 0 client.5952 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 >> 172.21.15.22:0/2180213607 conn(0x56041b364620 msgr2=0x56041b732760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 shutdown_connections 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 1 -- 172.21.15.22:0/2180213607 wait complete. 2024-09-21T05:42:13.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.177+0000 7f7f65e37740 20 client.5952 trim_cache size 0 max 16384 2024-09-21T05:42:13.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.195+0000 7f7f65e37740 1 Processor -- start 2024-09-21T05:42:13.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.195+0000 7f7f65e37740 1 -- start start 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 0x56041b7d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b73f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b3704d0 0x56041b73fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f65e37740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58201e50 con 0x56041b347610 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58200f30 con 0x56041b3704d0 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7f580b8cc0 con 0x56041b34f7f0 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 0x56041b7d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 0x56041b7d5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49066/0 (socket says 172.21.15.22:49066) 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/3660202455 learned_addr learned my addr 172.21.15.22:0/3660202455 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f57fff640 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b73f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b3704d0 0x56041b73fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 msgr2=0x56041b73f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b73f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b3704d0 msgr2=0x56041b73fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b3704d0 0x56041b73fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 -- 172.21.15.22:0/3660202455 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56041b71a300 con 0x56041b347610 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 0x56041b7d5ee0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c3c0 tx=0x7f7f480983c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c8cb69422533c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f3dffb640 1 -- 172.21.15.22:0/3660202455 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4807b070 con 0x56041b347610 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f3dffb640 1 -- 172.21.15.22:0/3660202455 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f48066870 con 0x56041b347610 2024-09-21T05:42:13.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.197+0000 7f7f3dffb640 1 -- 172.21.15.22:0/3660202455 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4806b720 con 0x56041b347610 2024-09-21T05:42:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 -- 172.21.15.22:0/3660202455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 msgr2=0x56041b7d5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 0x56041b7d5ee0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c3c0 tx=0x7f7f480983c0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 -- 172.21.15.22:0/3660202455 shutdown_connections 2024-09-21T05:42:13.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b3704d0 0x56041b73fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b34f7f0 0x56041b73f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 --2- 172.21.15.22:0/3660202455 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b347610 0x56041b7d5ee0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 -- 172.21.15.22:0/3660202455 >> 172.21.15.22:0/3660202455 conn(0x56041b364620 msgr2=0x56041b70e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 -- 172.21.15.22:0/3660202455 shutdown_connections 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.198+0000 7f7f65e37740 1 -- 172.21.15.22:0/3660202455 wait complete. 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.199+0000 7f7f65e37740 1 -- start start 2024-09-21T05:42:13.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.199+0000 7f7f65e37740 1 Processor -- start 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b347610 0x56041b758a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b34f7f0 0x56041b7cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b7d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581f5fe0 con 0x56041b3704d0 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56041b733390 con 0x56041b34f7f0 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56041b7d40b0 con 0x56041b347610 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f65e37740 10 client.0 _ll_register_callbacks cb 0x7f7f48001120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b7d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b7d3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49078/0 (socket says 172.21.15.22:49078) 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f57fff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b34f7f0 0x56041b7cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 -- 172.21.15.22:0/1617642287 learned_addr learned my addr 172.21.15.22:0/1617642287 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b347610 0x56041b758a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 -- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b347610 msgr2=0x56041b758a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b347610 0x56041b758a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 -- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b34f7f0 msgr2=0x56041b7cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b34f7f0 0x56041b7cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 -- 172.21.15.22:0/1617642287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56041b3b97e0 con 0x56041b3704d0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f5d46b640 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b7d3430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x56041b6802a0 tx=0x7f7f5000ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ab97b3483b0b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.201+0000 7f7f377fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f377fe640 1 -- 172.21.15.22:0/1617642287 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50004ce0 con 0x56041b3704d0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f377fe640 1 -- 172.21.15.22:0/1617642287 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7f5001f070 con 0x56041b3704d0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f48066870 con 0x56041b3704d0 2024-09-21T05:42:13.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f377fe640 1 -- 172.21.15.22:0/1617642287 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f7f5003b050 con 0x56041b3704d0 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 10 client.5958 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x56041b646fb0 con 0x56041b3704d0 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f377fe640 1 -- 172.21.15.22:0/1617642287 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f48066870 con 0x56041b3704d0 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f377fe640 1 -- 172.21.15.22:0/1617642287 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f7f5003e020 con 0x56041b3704d0 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 10 client.5958 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 10 client.5958 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 20 client.5958 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b7cd510 0x56041b7d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 4 client.5958 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x56041b688430 con 0x56041b7cd510 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.202+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b7cd510 0x56041b7d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.203+0000 7f7f377fe640 10 client.5958 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.203+0000 7f7f5cc6a640 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b7cd510 0x56041b7d25c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=434958edfbb6ca6a server_cookie=6ac4735163cdbfd2 in_seq=0 out_seq=0 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f377fe640 1 -- 172.21.15.22:0/1617642287 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56041b688430 con 0x56041b7cd510 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f377fe640 10 client.5958 handle_command_reply: tid=0 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 client.5958 shutdown 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 msgr2=0x56041b7d3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b7d3430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x56041b6802a0 tx=0x7f7f5000ff80 comp rx=0 tx=0).stop 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 shutdown_connections 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x56041b7cd510 0x56041b7d25c0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f377fe640 0 client.5958 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56041b3704d0 0x56041b7d3430 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56041b34f7f0 0x56041b7cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 --2- 172.21.15.22:0/1617642287 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56041b347610 0x56041b758a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 >> 172.21.15.22:0/1617642287 conn(0x56041b364620 msgr2=0x56041b740560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 shutdown_connections 2024-09-21T05:42:13.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 1 -- 172.21.15.22:0/1617642287 wait complete. 2024-09-21T05:42:13.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.204+0000 7f7f65e37740 20 client.5958 trim_cache size 0 max 16384 2024-09-21T05:42:13.211 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f13ac341640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 pv6 ap=4 DIRTYPARENT s=0 n(v0 rc2024-09-21T05:41:40.688007+0000 1=1+0)->n(v0 rc2024-09-21T05:41:40.688007+0000 1=1+0) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={5264=0-4194304@1} caps={5264=pAsLsXs/-@6} | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55c929782b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:42:13.211 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:13.211 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:42:13.212 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "06f62961-4d6b-4cff-a0d0-49d9e1df39f2", 2024-09-21T05:42:13.212 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:42:13.212 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7f44078bc0 msgr2=0x7f7f4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7f44078bc0 0x7f7f4407b0d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c034000 tx=0x7f7f4c004730 comp rx=0 tx=0).stop 2024-09-21T05:42:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 msgr2=0x7f7f581ea080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 0x7f7f581ea080 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800c3f0 tx=0x7f7f4800c8b0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 shutdown_connections 2024-09-21T05:42:13.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7f44078bc0 0x7f7f4407b0d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7f581f8910 0x7f7f581f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7f580b38a0 0x7f7f580b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 --2- 172.21.15.22:0/922138549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7f5806ded0 0x7f7f581ea080 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 >> 172.21.15.22:0/922138549 conn(0x7f7f5809d820 msgr2=0x7f7f580ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 shutdown_connections 2024-09-21T05:42:13.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.205+0000 7f7f5eece640 1 -- 172.21.15.22:0/922138549 wait complete. 2024-09-21T05:42:13.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:13.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-21T05:42:13.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 -- 172.21.15.22:0/3273346478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b4530 msgr2=0x7fdd1c0d0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 --2- 172.21.15.22:0/3273346478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b4530 0x7fdd1c0d0bf0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c074c90 tx=0x7fdd1402a820 comp rx=0 tx=0).stop 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 -- 172.21.15.22:0/3273346478 shutdown_connections 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 --2- 172.21.15.22:0/3273346478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b4530 0x7fdd1c0d0bf0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 --2- 172.21.15.22:0/3273346478 >> v2:172.21.15.134:3300/0 conn(0x7fdd1c0cff70 0x7fdd1c15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 -- 172.21.15.22:0/3273346478 >> 172.21.15.22:0/3273346478 conn(0x7fdd1c0cee20 msgr2=0x7fdd1c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 -- 172.21.15.22:0/3273346478 shutdown_connections 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.336+0000 7fdd2402e640 1 -- 172.21.15.22:0/3273346478 wait complete. 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.338+0000 7fdd2402e640 1 Processor -- start 2024-09-21T05:42:13.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.338+0000 7fdd2402e640 1 -- start start 2024-09-21T05:42:13.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd2402e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd1c0cff70 0x7fdd1c0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd2402e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd1c160c90 0x7fdd1c152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd2402e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 0x7fdd1c0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd2402e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c0d16e0 con 0x7fdd1c0b6220 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd2402e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c0d1560 con 0x7fdd1c160c90 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd2402e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1c0d1860 con 0x7fdd1c0cff70 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 0x7fdd1c0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 0x7fdd1c0b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49094/0 (socket says 172.21.15.22:49094) 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 -- 172.21.15.22:0/162096564 learned_addr learned my addr 172.21.15.22:0/162096564 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd21dca640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd1c0cff70 0x7fdd1c0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd215c9640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd1c160c90 0x7fdd1c152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 -- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd1c0cff70 msgr2=0x7fdd1c0c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd1c0cff70 0x7fdd1c0c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 -- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd1c160c90 msgr2=0x7fdd1c152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd1c160c90 0x7fdd1c152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd225cb640 1 -- 172.21.15.22:0/162096564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1c15c830 con 0x7fdd1c0b6220 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.339+0000 7fdd21dca640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd1c0cff70 0x7fdd1c0c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.340+0000 7fdd225cb640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 0x7fdd1c0b6630 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009ca0 tx=0x7fdd14032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.340+0000 7fdd127fc640 1 -- 172.21.15.22:0/162096564 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1403f020 con 0x7fdd1c0b6220 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.340+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1c0b6b70 con 0x7fdd1c0b6220 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.340+0000 7fdd127fc640 1 -- 172.21.15.22:0/162096564 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd14004040 con 0x7fdd1c0b6220 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.340+0000 7fdd127fc640 1 -- 172.21.15.22:0/162096564 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14037040 con 0x7fdd1c0b6220 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.340+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd1c0c7a00 con 0x7fdd1c0b6220 2024-09-21T05:42:13.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.342+0000 7fdd127fc640 1 -- 172.21.15.22:0/162096564 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd14042020 con 0x7fdd1c0b6220 2024-09-21T05:42:13.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.343+0000 7fdd127fc640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd00078bc0 0x7fdd0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.343+0000 7fdd127fc640 1 -- 172.21.15.22:0/162096564 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fdd140bb2b0 con 0x7fdd1c0b6220 2024-09-21T05:42:13.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.343+0000 7fdd21dca640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd00078bc0 0x7fdd0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.344+0000 7fdd2b077740 1 Processor -- start 2024-09-21T05:42:13.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.344+0000 7fdd21dca640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd00078bc0 0x7fdd0007b0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd18005fd0 tx=0x7fdd1800f360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.344+0000 7fdd2b077740 1 -- start start 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e4247a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e42475a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424785810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd2b077740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e4246d1a60 con 0x55e4243b7390 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e4246c0e90 con 0x55e4243966b0 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e4246c4ca0 con 0x55e42438e4d0 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd21dca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e4247a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd21dca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e4247a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49106/0 (socket says 172.21.15.22:49106) 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd21dca640 1 -- 172.21.15.22:0/2412608385 learned_addr learned my addr 172.21.15.22:0/2412608385 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd225cb640 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424785810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.345+0000 7fdd215c9640 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e42475a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd21dca640 1 -- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 msgr2=0x55e42475a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd21dca640 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e42475a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd21dca640 1 -- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 msgr2=0x55e424785810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd21dca640 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424785810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd21dca640 1 -- 172.21.15.22:0/2412608385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4244997d0 con 0x55e4243b7390 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd225cb640 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424785810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdd21dca640 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e4247a4a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800fee0 tx=0x7fdd18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b689b5473bb8ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdcff7fe640 1 -- 172.21.15.22:0/2412608385 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18022070 con 0x55e4243b7390 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdcff7fe640 1 -- 172.21.15.22:0/2412608385 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd18011e50 con 0x55e4243b7390 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.346+0000 7fdcff7fe640 1 -- 172.21.15.22:0/2412608385 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1801e9a0 con 0x55e4243b7390 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 -- 172.21.15.22:0/2412608385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 msgr2=0x55e4247a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e4247a4a50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800fee0 tx=0x7fdd18012000 comp rx=0 tx=0).stop 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 -- 172.21.15.22:0/2412608385 shutdown_connections 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424785810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e42475a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 --2- 172.21.15.22:0/2412608385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e4247a4a50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 -- 172.21.15.22:0/2412608385 >> 172.21.15.22:0/2412608385 conn(0x55e4243ab4e0 msgr2=0x55e42475ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 -- 172.21.15.22:0/2412608385 shutdown_connections 2024-09-21T05:42:13.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.347+0000 7fdd2b077740 1 -- 172.21.15.22:0/2412608385 wait complete. 2024-09-21T05:42:13.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.348+0000 7fdd2b077740 1 -- start start 2024-09-21T05:42:13.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.348+0000 7fdd2b077740 1 Processor -- start 2024-09-21T05:42:13.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e4247b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e424788300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e424431740 con 0x55e42438e4d0 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e424666970 con 0x55e4243b7390 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e424787ff0 con 0x55e4243966b0 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd2b077740 10 client.0 _ll_register_callbacks cb 0x55e424675780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd21dca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e4247b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd21dca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e4247b7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49120/0 (socket says 172.21.15.22:49120) 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.349+0000 7fdd21dca640 1 -- 172.21.15.22:0/1217952071 learned_addr learned my addr 172.21.15.22:0/1217952071 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd215c9640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd225cb640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e424788300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd21dca640 1 -- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 msgr2=0x55e4247b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd21dca640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd21dca640 1 -- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 msgr2=0x55e424788300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd21dca640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e424788300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd21dca640 1 -- 172.21.15.22:0/1217952071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e424422970 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd215c9640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd225cb640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e424788300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd21dca640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e4247b7c60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800bfd0 tx=0x7fdd18020990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a419813e7c850d15 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdcfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdcfcff9640 1 -- 172.21.15.22:0/1217952071 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18022040 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdcfcff9640 1 -- 172.21.15.22:0/1217952071 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fdd1801e450 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd18011e50 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdcfcff9640 1 -- 172.21.15.22:0/1217952071 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd18020d90 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd2b077740 10 client.5973 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55e4244006a0 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.350+0000 7fdcfcff9640 1 -- 172.21.15.22:0/1217952071 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd18011e50 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdcfcff9640 1 -- 172.21.15.22:0/1217952071 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fdd18004020 con 0x55e42438e4d0 2024-09-21T05:42:13.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd2b077740 10 client.5973 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd2b077740 10 client.5973 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd2b077740 20 client.5973 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd2b077740 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e424375f60 0x55e4247ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd2b077740 4 client.5973 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4246d39b0 con 0x55e424375f60 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd215c9640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e424375f60 0x55e4247ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdcfcff9640 10 client.5973 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.351+0000 7fdd215c9640 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e424375f60 0x55e4247ba770 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb8a93749f904cb server_cookie=cb63bf13d59fd8e6 in_seq=0 out_seq=0 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdcfcff9640 1 -- 172.21.15.22:0/1217952071 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4246d39b0 con 0x55e424375f60 2024-09-21T05:42:13.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdcfcff9640 10 client.5973 handle_command_reply: tid=0 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 client.5973 shutdown 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 msgr2=0x55e4247b7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e4247b7c60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800bfd0 tx=0x7fdd18020990 comp rx=0 tx=0).stop 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 shutdown_connections 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e424375f60 0x55e4247ba770 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e424788300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdcfcff9640 0 client.5973 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 --2- 172.21.15.22:0/1217952071 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e4247b7c60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 >> 172.21.15.22:0/1217952071 conn(0x55e4243ab4e0 msgr2=0x55e4247a5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 shutdown_connections 2024-09-21T05:42:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 1 -- 172.21.15.22:0/1217952071 wait complete. 2024-09-21T05:42:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.352+0000 7fdd2b077740 20 client.5973 trim_cache size 0 max 16384 2024-09-21T05:42:13.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.370+0000 7fdd2b077740 1 Processor -- start 2024-09-21T05:42:13.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.370+0000 7fdd2b077740 1 -- start start 2024-09-21T05:42:13.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e424758b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.375 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424776ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e424777000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd2b077740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e42475c4e0 con 0x55e4243b7390 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c1fdcc0 con 0x55e4243966b0 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdd1c06f4d0 con 0x55e42438e4d0 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e424777000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e424777000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49130/0 (socket says 172.21.15.22:49130) 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 -- 172.21.15.22:0/3156821934 learned_addr learned my addr 172.21.15.22:0/3156821934 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd215c9640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424776ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd21dca640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e424758b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 -- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 msgr2=0x55e424758b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e424758b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 -- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 msgr2=0x55e424776ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424776ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 -- 172.21.15.22:0/3156821934 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e424759090 con 0x55e4243b7390 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd215c9640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424776ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd21dca640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e424758b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdd225cb640 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e424777000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009b30 tx=0x7fdd14082770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94fc6194371cfae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdcfeffd640 1 -- 172.21.15.22:0/3156821934 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14083000 con 0x55e4243b7390 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdcfeffd640 1 -- 172.21.15.22:0/3156821934 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd14087ae0 con 0x55e4243b7390 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.372+0000 7fdcfeffd640 1 -- 172.21.15.22:0/3156821934 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1408c990 con 0x55e4243b7390 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 -- 172.21.15.22:0/3156821934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 msgr2=0x55e424777000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e424777000 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009b30 tx=0x7fdd14082770 comp rx=0 tx=0).stop 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 -- 172.21.15.22:0/3156821934 shutdown_connections 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e4243b7390 0x55e424777000 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243966b0 0x55e424776ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 --2- 172.21.15.22:0/3156821934 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e42438e4d0 0x55e424758b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 -- 172.21.15.22:0/3156821934 >> 172.21.15.22:0/3156821934 conn(0x55e4243ab4e0 msgr2=0x55e42479bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 -- 172.21.15.22:0/3156821934 shutdown_connections 2024-09-21T05:42:13.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.373+0000 7fdd2b077740 1 -- 172.21.15.22:0/3156821934 wait complete. 2024-09-21T05:42:13.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.374+0000 7fdd2b077740 1 -- start start 2024-09-21T05:42:13.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.374+0000 7fdd2b077740 1 Processor -- start 2024-09-21T05:42:13.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e42481d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247772d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e4247a6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c15b570 con 0x55e42438e4d0 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1c15af20 con 0x55e4243b7390 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e42475c4e0 con 0x55e4243966b0 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 10 client.0 _ll_register_callbacks cb 0x7fdd0c007e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e42481d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e42481d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49144/0 (socket says 172.21.15.22:49144) 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 -- 172.21.15.22:0/565176169 learned_addr learned my addr 172.21.15.22:0/565176169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 -- 172.21.15.22:0/565176169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 msgr2=0x55e4247772d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247772d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 -- 172.21.15.22:0/565176169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 msgr2=0x55e4247a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e4247a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 -- 172.21.15.22:0/565176169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e42475a4c0 con 0x55e42438e4d0 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd21dca640 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e42481d580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd18010010 tx=0x7fdd1803bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7cfae5fa859efa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdcfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdcfcff9640 1 -- 172.21.15.22:0/565176169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18015050 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdcfcff9640 1 -- 172.21.15.22:0/565176169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fdd1802d040 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.376+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd14087ae0 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdcfcff9640 1 -- 172.21.15.22:0/565176169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fdd18049050 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 10 client.5979 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55e42479d610 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdcfcff9640 1 -- 172.21.15.22:0/565176169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd14087ae0 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdcfcff9640 1 -- 172.21.15.22:0/565176169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fdd1804c020 con 0x55e42438e4d0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 10 client.5979 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 10 client.5979 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 20 client.5979 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e4247a6880 0x55e424826ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 4 client.5979 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4246cf460 con 0x55e4247a6880 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd215c9640 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e4247a6880 0x55e424826ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdcfcff9640 10 client.5979 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.377+0000 7fdd215c9640 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e4247a6880 0x55e424826ea0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10abe0414570aa30 server_cookie=fa66e6f709f2fb62 in_seq=0 out_seq=0 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdcfcff9640 1 -- 172.21.15.22:0/565176169 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55e4246cf460 con 0x55e4247a6880 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdcfcff9640 10 client.5979 handle_command_reply: tid=0 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 client.5979 shutdown 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 msgr2=0x55e42481d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e42481d580 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdd18010010 tx=0x7fdd1803bca0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 shutdown_connections 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55e4247a6880 0x55e424826ea0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e4243b7390 0x55e4247a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdcfcff9640 0 client.5979 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e4243966b0 0x55e4247772d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 --2- 172.21.15.22:0/565176169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e42438e4d0 0x55e42481d580 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 >> 172.21.15.22:0/565176169 conn(0x55e4243ab4e0 msgr2=0x55e42479bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 shutdown_connections 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 1 -- 172.21.15.22:0/565176169 wait complete. 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.378+0000 7fdd2b077740 20 client.5979 trim_cache size 0 max 16384 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "06f62961-4d6b-4cff-a0d0-49d9e1df39f2": { 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:42:13.203585+0000", 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:42:13.385 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "eb94107a-f919-4784-b8a0-9ecc5d24cea9": { 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:41:41.755993+0000", 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:13.386 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd00078bc0 msgr2=0x7fdd0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd00078bc0 0x7fdd0007b0d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdd18005fd0 tx=0x7fdd1800f360 comp rx=0 tx=0).stop 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 msgr2=0x7fdd1c0b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 0x7fdd1c0b6630 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009ca0 tx=0x7fdd14032c90 comp rx=0 tx=0).stop 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 shutdown_connections 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdd1c0b6220 0x7fdd1c0b6630 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdd1c160c90 0x7fdd1c152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdd00078bc0 0x7fdd0007b0d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 --2- 172.21.15.22:0/162096564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdd1c0cff70 0x7fdd1c0c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 >> 172.21.15.22:0/162096564 conn(0x7fdd1c0cee20 msgr2=0x7fdd1c0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 shutdown_connections 2024-09-21T05:42:13.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.379+0000 7fdd2402e640 1 -- 172.21.15.22:0/162096564 wait complete. 2024-09-21T05:42:13.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:42:13.392 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:13.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 -- 172.21.15.22:0/4234157019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6000ce7d0 msgr2=0x7ff600071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 --2- 172.21.15.22:0/4234157019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6000ce7d0 0x7ff600071250 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009830 tx=0x7ff5f802a5a0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 -- 172.21.15.22:0/4234157019 shutdown_connections 2024-09-21T05:42:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 --2- 172.21.15.22:0/4234157019 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6000ce7d0 0x7ff600071250 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 -- 172.21.15.22:0/4234157019 >> 172.21.15.22:0/4234157019 conn(0x7ff6000cdcb0 msgr2=0x7ff6000ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 -- 172.21.15.22:0/4234157019 shutdown_connections 2024-09-21T05:42:13.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.506+0000 7ff6055c3640 1 -- 172.21.15.22:0/4234157019 wait complete. 2024-09-21T05:42:13.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.508+0000 7ff6055c3640 1 Processor -- start 2024-09-21T05:42:13.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.508+0000 7ff6055c3640 1 -- start start 2024-09-21T05:42:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff6055c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6000ce7d0 0x7ff600151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff6055c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 0x7ff6000ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff6055c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6000cd510 0x7ff6000b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff6055c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000d0540 con 0x7ff6001600a0 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff6055c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000d03c0 con 0x7ff6000cd510 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff6055c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff6000d06c0 con 0x7ff6000ce7d0 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff5fe7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 0x7ff6000ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff5fe7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 0x7ff6000ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49154/0 (socket says 172.21.15.22:49154) 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3545295255 learned_addr learned my addr 172.21.15.22:0/3545295255 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5ff7fe640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6000cd510 0x7ff6000b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.509+0000 7ff5feffd640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6000ce7d0 0x7ff600151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6000ce7d0 msgr2=0x7ff600151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5fe7fc640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6000ce7d0 0x7ff600151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6000cd510 msgr2=0x7ff6000b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5fe7fc640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6000cd510 0x7ff6000b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5fe7fc640 1 -- 172.21.15.22:0/3545295255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6000cbb60 con 0x7ff6001600a0 2024-09-21T05:42:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5fe7fc640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 0x7ff6000ccfd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c3f0 tx=0x7ff5e800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8017070 con 0x7ff6001600a0 2024-09-21T05:42:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5e80035f0 con 0x7ff6001600a0 2024-09-21T05:42:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6000cbe40 con 0x7ff6001600a0 2024-09-21T05:42:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8003a40 con 0x7ff6001600a0 2024-09-21T05:42:13.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.510+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6000cc400 con 0x7ff6001600a0 2024-09-21T05:42:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.511+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff600071250 con 0x7ff6001600a0 2024-09-21T05:42:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.512+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5e801a070 con 0x7ff6001600a0 2024-09-21T05:42:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.512+0000 7ff5f77fe640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5e0078b90 0x7ff5e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.513+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7ff5e8099f50 con 0x7ff6001600a0 2024-09-21T05:42:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.513+0000 7ff5feffd640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5e0078b90 0x7ff5e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.513+0000 7ff5feffd640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5e0078b90 0x7ff5e007b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000b450 tx=0x7ff5f000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.517+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5e8061dd0 con 0x7ff6001600a0 2024-09-21T05:42:13.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.745+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6000aeb50 con 0x7ff6001600a0 2024-09-21T05:42:13.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.746+0000 7ff5f77fe640 1 -- 172.21.15.22:0/3545295255 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5226 (secure 0 0 0) 0x7ff5e8066800 con 0x7ff6001600a0 2024-09-21T05:42:13.748 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:13.748 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":35,"btime":"2024-09-21T05:41:45:826771+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:41:45.657650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":6,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:42:13.748 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 35 2024-09-21T05:42:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5e0078b90 msgr2=0x7ff5e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5e0078b90 0x7ff5e007b0a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000b450 tx=0x7ff5f000d040 comp rx=0 tx=0).stop 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 msgr2=0x7ff6000ccfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 0x7ff6000ccfd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c3f0 tx=0x7ff5e800c8b0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 shutdown_connections 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff6000cd510 0x7ff6000b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff6001600a0 0x7ff6000ccfd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5e0078b90 0x7ff5e007b0a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 --2- 172.21.15.22:0/3545295255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff6000ce7d0 0x7ff600151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 >> 172.21.15.22:0/3545295255 conn(0x7ff6000cdcb0 msgr2=0x7ff60015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 shutdown_connections 2024-09-21T05:42:13.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.748+0000 7ff6055c3640 1 -- 172.21.15.22:0/3545295255 wait complete. 2024-09-21T05:42:13.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T05:42:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e348b640 1 -- 172.21.15.22:0/3826383318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8c4004e60 msgr2=0x7fb8c4005290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3826383318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8c4004e60 0x7fb8c4005290 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc001d90 tx=0x7fb8c4005830 comp rx=0 tx=0).stop 2024-09-21T05:42:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3826383318 reap_dead start 2024-09-21T05:42:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e348b640 1 -- 172.21.15.22:0/3826383318 shutdown_connections 2024-09-21T05:42:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e348b640 1 -- 172.21.15.22:0/3826383318 >> 172.21.15.22:0/3826383318 conn(0x7fb8dc09d840 msgr2=0x7fb8dc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e348b640 1 -- 172.21.15.22:0/3826383318 shutdown_connections 2024-09-21T05:42:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.879+0000 7fb8e348b640 1 -- 172.21.15.22:0/3826383318 wait complete. 2024-09-21T05:42:13.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.881+0000 7fb8e348b640 1 Processor -- start 2024-09-21T05:42:13.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.881+0000 7fb8e348b640 1 -- start start 2024-09-21T05:42:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e348b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8c4004e60 0x7fb8dc157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e348b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8dc15c280 0x7fb8dc0b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e348b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 0x7fb8dc0b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e348b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0b9b70 con 0x7fb8dc15c690 2024-09-21T05:42:13.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e348b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc0b99f0 con 0x7fb8dc15c280 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e348b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc0cfb10 con 0x7fb8c4004e60 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 0x7fb8dc0b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 0x7fb8dc0b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49164/0 (socket says 172.21.15.22:49164) 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3901266969 learned_addr learned my addr 172.21.15.22:0/3901266969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1227640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8c4004e60 0x7fb8dc157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8dc15c280 0x7fb8dc0b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8c4004e60 msgr2=0x7fb8dc157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8c4004e60 0x7fb8dc157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8dc15c280 msgr2=0x7fb8dc0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8dc15c280 0x7fb8dc0b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.882+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3901266969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8dc1f8830 con 0x7fb8dc15c690 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.883+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 0x7fb8dc0b3100 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc02ac80 tx=0x7fb8cc001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.883+0000 7fb8d9ffb640 1 -- 172.21.15.22:0/3901266969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0060b0 con 0x7fb8dc15c690 2024-09-21T05:42:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.883+0000 7fb8d9ffb640 1 -- 172.21.15.22:0/3901266969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8cc006250 con 0x7fb8dc15c690 2024-09-21T05:42:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.883+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8dc1f51f0 con 0x7fb8dc15c690 2024-09-21T05:42:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.883+0000 7fb8d9ffb640 1 -- 172.21.15.22:0/3901266969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc0066a0 con 0x7fb8dc15c690 2024-09-21T05:42:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.883+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8dc1f9010 con 0x7fb8dc15c690 2024-09-21T05:42:13.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.885+0000 7fb8d9ffb640 1 -- 172.21.15.22:0/3901266969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb8cc033480 con 0x7fb8dc15c690 2024-09-21T05:42:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.886+0000 7fb8d9ffb640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb8c8078b90 0x7fb8c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.886+0000 7fb8d9ffb640 1 -- 172.21.15.22:0/3901266969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb8cc0ba8f0 con 0x7fb8dc15c690 2024-09-21T05:42:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.886+0000 7fb8e1227640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb8c8078b90 0x7fb8c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.887+0000 7fb8e1227640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb8c8078b90 0x7fb8c807b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009c40 tx=0x7fb8d4005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.887+0000 7fb8ea3fc740 1 Processor -- start 2024-09-21T05:42:13.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.887+0000 7fb8ea3fc740 1 -- start start 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a12c350 0x55ba6a4fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 0x55ba6a4fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a51b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a4469a0 con 0x55ba6a103490 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a435dd0 con 0x55ba6a10b670 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ba6a439be0 con 0x55ba6a12c350 2024-09-21T05:42:13.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8e0a26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 0x55ba6a4fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8e0a26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 0x55ba6a4fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49172/0 (socket says 172.21.15.22:49172) 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8e0a26640 1 -- 172.21.15.22:0/3816320221 learned_addr learned my addr 172.21.15.22:0/3816320221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.888+0000 7fb8e1227640 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a12c350 0x55ba6a4fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a51b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e0a26640 1 -- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a12c350 msgr2=0x55ba6a4fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a12c350 0x55ba6a4fdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e0a26640 1 -- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 msgr2=0x55ba6a51b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a51b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e0a26640 1 -- 172.21.15.22:0/3816320221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6a3ce700 con 0x55ba6a103490 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a51b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 0x55ba6a4fe2c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0006b80 tx=0x7fb8d0006490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bbfb0f63d2fe25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8c2ffd640 1 -- 172.21.15.22:0/3816320221 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003bd0 con 0x55ba6a103490 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8c2ffd640 1 -- 172.21.15.22:0/3816320221 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d0003d70 con 0x55ba6a103490 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.889+0000 7fb8c2ffd640 1 -- 172.21.15.22:0/3816320221 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0011690 con 0x55ba6a103490 2024-09-21T05:42:13.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3816320221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 msgr2=0x55ba6a4fe2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 0x55ba6a4fe2c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0006b80 tx=0x7fb8d0006490 comp rx=0 tx=0).stop 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3816320221 shutdown_connections 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a51b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a103490 0x55ba6a4fe2c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3816320221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a12c350 0x55ba6a4fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3816320221 >> 172.21.15.22:0/3816320221 conn(0x55ba6a1204a0 msgr2=0x55ba6a4d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3816320221 shutdown_connections 2024-09-21T05:42:13.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.890+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3816320221 wait complete. 2024-09-21T05:42:13.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.891+0000 7fb8ea3fc740 1 -- start start 2024-09-21T05:42:13.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.891+0000 7fb8ea3fc740 1 Processor -- start 2024-09-21T05:42:13.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a4facb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a4d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a12c350 0x55ba6a4ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a1a6700 con 0x55ba6a10b670 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a3db340 con 0x55ba6a12c350 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ba6a4fe800 con 0x55ba6a103490 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8ea3fc740 10 client.0 _ll_register_callbacks cb 0x55ba6a3d9ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8e0a26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a4d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8e0a26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a4d3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49188/0 (socket says 172.21.15.22:49188) 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.892+0000 7fb8e0a26640 1 -- 172.21.15.22:0/2358882492 learned_addr learned my addr 172.21.15.22:0/2358882492 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e1227640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a4facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a12c350 0x55ba6a4ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e0a26640 1 -- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 msgr2=0x55ba6a4facb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a4facb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e0a26640 1 -- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a12c350 msgr2=0x55ba6a4ebda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a12c350 0x55ba6a4ebda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e0a26640 1 -- 172.21.15.22:0/2358882492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6a2107c0 con 0x55ba6a10b670 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e1227640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a4facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a12c350 0x55ba6a4ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a4d3150 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0000c00 tx=0x7fb8d001d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac17201b94bd4b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8bbfff640 1 -- 172.21.15.22:0/2358882492 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003bd0 con 0x55ba6a10b670 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8bbfff640 1 -- 172.21.15.22:0/2358882492 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb8d001dc80 con 0x55ba6a10b670 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8d0003d70 con 0x55ba6a10b670 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8bbfff640 1 -- 172.21.15.22:0/2358882492 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8d002e050 con 0x55ba6a10b670 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8ea3fc740 10 client.6000 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.893+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ba6a270f50 con 0x55ba6a10b670 2024-09-21T05:42:13.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8bbfff640 1 -- 172.21.15.22:0/2358882492 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8d0003d70 con 0x55ba6a10b670 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8bbfff640 1 -- 172.21.15.22:0/2358882492 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fb8d0031020 con 0x55ba6a10b670 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8ea3fc740 10 client.6000 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8ea3fc740 10 client.6000 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8ea3fc740 20 client.6000 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0eb130 0x55ba6a50f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8ea3fc740 4 client.6000 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba6a4488f0 con 0x55ba6a0eb130 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8e1227640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0eb130 0x55ba6a50f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8bbfff640 10 client.6000 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.894+0000 7fb8e1227640 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0eb130 0x55ba6a50f970 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19dd95ba5dac1398 server_cookie=301f7086ffac50af in_seq=0 out_seq=0 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8bbfff640 1 -- 172.21.15.22:0/2358882492 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba6a4488f0 con 0x55ba6a0eb130 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8bbfff640 10 client.6000 handle_command_reply: tid=0 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 client.6000 shutdown 2024-09-21T05:42:13.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 msgr2=0x55ba6a4d3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a4d3150 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb8d0000c00 tx=0x7fb8d001d680 comp rx=0 tx=0).stop 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 shutdown_connections 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0eb130 0x55ba6a50f970 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a12c350 0x55ba6a4ebda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a4d3150 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8bbfff640 0 client.6000 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/2358882492 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a4facb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 >> 172.21.15.22:0/2358882492 conn(0x55ba6a1204a0 msgr2=0x55ba6a4d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 shutdown_connections 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.895+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/2358882492 wait complete. 2024-09-21T05:42:13.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.896+0000 7fb8ea3fc740 20 client.6000 trim_cache size 0 max 16384 2024-09-21T05:42:13.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.909+0000 7fb8ea3fc740 1 Processor -- start 2024-09-21T05:42:13.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.909+0000 7fb8ea3fc740 1 -- start start 2024-09-21T05:42:13.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.910+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a1204a0 0x55ba6a4ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.910+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a103490 0x55ba6a52af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.910+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a50ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.910+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc201700 con 0x55ba6a10b670 2024-09-21T05:42:13.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.910+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb8dc200bc0 con 0x55ba6a103490 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.910+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb8dc2010b0 con 0x55ba6a1204a0 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a50ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a50ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49192/0 (socket says 172.21.15.22:49192) 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3105626295 learned_addr learned my addr 172.21.15.22:0/3105626295 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1227640 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a1204a0 0x55ba6a4ecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a103490 0x55ba6a52af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a1204a0 msgr2=0x55ba6a4ecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a1204a0 0x55ba6a4ecdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a103490 msgr2=0x55ba6a52af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a103490 0x55ba6a52af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 -- 172.21.15.22:0/3105626295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6a175660 con 0x55ba6a10b670 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a103490 0x55ba6a52af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a50ef90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0b8da0 tx=0x7fb8cc0825e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961d9dbf8c4afd61 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8c27fc640 1 -- 172.21.15.22:0/3105626295 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc045070 con 0x55ba6a10b670 2024-09-21T05:42:13.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8c27fc640 1 -- 172.21.15.22:0/3105626295 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8cc06d070 con 0x55ba6a10b670 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.911+0000 7fb8c27fc640 1 -- 172.21.15.22:0/3105626295 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc08be00 con 0x55ba6a10b670 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3105626295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 msgr2=0x55ba6a50ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a50ef90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0b8da0 tx=0x7fb8cc0825e0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3105626295 shutdown_connections 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a10b670 0x55ba6a50ef90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a103490 0x55ba6a52af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/3105626295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a1204a0 0x55ba6a4ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3105626295 >> 172.21.15.22:0/3105626295 conn(0x55ba6a4c9f80 msgr2=0x55ba6a4f7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3105626295 shutdown_connections 2024-09-21T05:42:13.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.912+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/3105626295 wait complete. 2024-09-21T05:42:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.913+0000 7fb8ea3fc740 1 -- start start 2024-09-21T05:42:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.913+0000 7fb8ea3fc740 1 Processor -- start 2024-09-21T05:42:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.914+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a57e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.914+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a4d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.914+0000 7fb8ea3fc740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 0x55ba6a53b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.914+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a4d36e0 con 0x55ba6a1204a0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.914+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a1a6700 con 0x55ba6a10b670 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.914+0000 7fb8ea3fc740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ba6a4ec9c0 con 0x55ba6a103490 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8ea3fc740 10 client.0 _ll_register_callbacks cb 0x7fb8d0006ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 0x55ba6a53b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 0x55ba6a53b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49200/0 (socket says 172.21.15.22:49200) 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 -- 172.21.15.22:0/50334385 learned_addr learned my addr 172.21.15.22:0/50334385 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1227640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a57e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e0a26640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a4d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 -- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 msgr2=0x55ba6a57e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a57e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 -- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 msgr2=0x55ba6a4d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a4d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 -- 172.21.15.22:0/50334385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6a581be0 con 0x55ba6a1204a0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8e1a28640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 0x55ba6a53b890 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc085040 tx=0x7fb8cc08b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345e52db67acca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:13.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8bbfff640 1 -- 172.21.15.22:0/50334385 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc045050 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8bbfff640 1 -- 172.21.15.22:0/50334385 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb8cc089380 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8cc06d070 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8bbfff640 1 -- 172.21.15.22:0/50334385 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb8cc082750 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8ea3fc740 10 client.6006 fetch_fsmap learned FSMap version 35 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.915+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55ba6a581be0 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8bbfff640 1 -- 172.21.15.22:0/50334385 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8cc06d070 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8bbfff640 1 -- 172.21.15.22:0/50334385 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fb8cc09cb60 con 0x55ba6a1204a0 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8ea3fc740 10 client.6006 fetch_fsmap finished waiting for FSMap version 35 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8ea3fc740 10 client.6006 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8ea3fc740 20 client.6006 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0c0570 0x55ba6a50fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8ea3fc740 4 client.6006 mds_command: new command op to 5679 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 --> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55ba6a4f8bf0 con 0x55ba6a0c0570 2024-09-21T05:42:13.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8e1227640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0c0570 0x55ba6a50fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8bbfff640 10 client.6006 ms_handle_connect on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.916+0000 7fb8e1227640 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0c0570 0x55ba6a50fc70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2aa136a8d1cd6ec server_cookie=3fd6659b29f8b63b in_seq=0 out_seq=0 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8bbfff640 1 -- 172.21.15.22:0/50334385 <== mds.0 v2:172.21.15.22:6833/913296056 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba6a4f8bf0 con 0x55ba6a0c0570 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8bbfff640 10 client.6006 handle_command_reply: tid=0 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 client.6006 shutdown 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 msgr2=0x55ba6a53b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 0x55ba6a53b890 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc085040 tx=0x7fb8cc08b280 comp rx=0 tx=0).stop 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 shutdown_connections 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:6833/913296056,v1:172.21.15.22:6835/913296056] conn(0x55ba6a0c0570 0x55ba6a50fc70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ba6a1204a0 0x55ba6a53b890 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ba6a10b670 0x55ba6a4d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 --2- 172.21.15.22:0/50334385 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ba6a103490 0x55ba6a57e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8bbfff640 0 client.6006 ms_handle_reset on v2:172.21.15.22:6833/913296056 2024-09-21T05:42:13.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 >> 172.21.15.22:0/50334385 conn(0x55ba6a4c9f80 msgr2=0x55ba6a5276f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 shutdown_connections 2024-09-21T05:42:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 1 -- 172.21.15.22:0/50334385 wait complete. 2024-09-21T05:42:13.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.917+0000 7fb8ea3fc740 20 client.6006 trim_cache size 0 max 16384 2024-09-21T05:42:13.924 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:13.924 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb8c8078b90 msgr2=0x7fb8c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb8c8078b90 0x7fb8c807b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb8d4009c40 tx=0x7fb8d4005f30 comp rx=0 tx=0).stop 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 msgr2=0x7fb8dc0b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 0x7fb8dc0b3100 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc02ac80 tx=0x7fb8cc001b10 comp rx=0 tx=0).stop 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 shutdown_connections 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb8dc15c690 0x7fb8dc0b3100 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb8dc15c280 0x7fb8dc0b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb8c8078b90 0x7fb8c807b0a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 --2- 172.21.15.22:0/3901266969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb8c4004e60 0x7fb8dc157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 >> 172.21.15.22:0/3901266969 conn(0x7fb8dc09d840 msgr2=0x7fb8dc0b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:13.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 shutdown_connections 2024-09-21T05:42:13.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:13.918+0000 7fb8e348b640 1 -- 172.21.15.22:0/3901266969 wait complete. 2024-09-21T05:42:13.931 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.044+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/1516448742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0cc004d40 msgr2=0x7fb0cc005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.044+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/1516448742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0cc004d40 0x7fb0cc005150 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005fd0 tx=0x7fb0cc005830 comp rx=0 tx=0).stop 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.044+0000 7fb0e6241640 1 -- 172.21.15.22:0/1516448742 reap_dead start 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.044+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/1516448742 shutdown_connections 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.044+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/1516448742 >> 172.21.15.22:0/1516448742 conn(0x7fb0e00b3f10 msgr2=0x7fb0e00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.045+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/1516448742 shutdown_connections 2024-09-21T05:42:14.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.045+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/1516448742 wait complete. 2024-09-21T05:42:14.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.046+0000 7fb0e7ca4640 1 Processor -- start 2024-09-21T05:42:14.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.046+0000 7fb0e7ca4640 1 -- start start 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e7ca4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 0x7fb0e0152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e7ca4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e015a1d0 0x7fb0e01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e7ca4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0e0156630 0x7fb0e01f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e7ca4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e00cd340 con 0x7fb0e01545b0 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e7ca4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e00cd1c0 con 0x7fb0e015a1d0 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e7ca4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e00d0e60 con 0x7fb0e0156630 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 0x7fb0e0152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 0x7fb0e0152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49224/0 (socket says 172.21.15.22:49224) 2024-09-21T05:42:14.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 -- 172.21.15.22:0/3153136568 learned_addr learned my addr 172.21.15.22:0/3153136568 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e6241640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0e0156630 0x7fb0e01f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e523f640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e015a1d0 0x7fb0e01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 -- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0e0156630 msgr2=0x7fb0e01f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0e0156630 0x7fb0e01f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 -- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e015a1d0 msgr2=0x7fb0e01560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e015a1d0 0x7fb0e01560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 -- 172.21.15.22:0/3153136568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e0152eb0 con 0x7fb0e01545b0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e6241640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0e0156630 0x7fb0e01f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e523f640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e015a1d0 0x7fb0e01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0e5a40640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 0x7fb0e0152970 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc011ae0 tx=0x7fb0dc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.048+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00fcb0 con 0x7fb0e01545b0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.049+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0dc00fe50 con 0x7fb0e01545b0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.049+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e01f63b0 con 0x7fb0e01545b0 2024-09-21T05:42:14.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.049+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc004280 con 0x7fb0e01545b0 2024-09-21T05:42:14.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.049+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e00b0f20 con 0x7fb0e01545b0 2024-09-21T05:42:14.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.049+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e00b5210 con 0x7fb0e01545b0 2024-09-21T05:42:14.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.050+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0dc0035f0 con 0x7fb0e01545b0 2024-09-21T05:42:14.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.051+0000 7fb0d67fc640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0c4078bc0 0x7fb0c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.051+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb0dc09b870 con 0x7fb0e01545b0 2024-09-21T05:42:14.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.051+0000 7fb0e523f640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0c4078bc0 0x7fb0c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.052+0000 7fb0e523f640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0c4078bc0 0x7fb0c407b0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0001330 tx=0x7fb0d0001200 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:14.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.056+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb0dc063ca0 con 0x7fb0e01545b0 2024-09-21T05:42:14.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.282+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb0e00c9130 con 0x7fb0e01545b0 2024-09-21T05:42:14.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.283+0000 7fb0d67fc640 1 -- 172.21.15.22:0/3153136568 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb0dc0635a0 con 0x7fb0e01545b0 2024-09-21T05:42:14.285 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:14.285 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T05:42:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0c4078bc0 msgr2=0x7fb0c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0c4078bc0 0x7fb0c407b0d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0001330 tx=0x7fb0d0001200 comp rx=0 tx=0).stop 2024-09-21T05:42:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 msgr2=0x7fb0e0152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 0x7fb0e0152970 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc011ae0 tx=0x7fb0dc009f10 comp rx=0 tx=0).stop 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 shutdown_connections 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb0c4078bc0 0x7fb0c407b0d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb0e0156630 0x7fb0e01f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb0e015a1d0 0x7fb0e01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 --2- 172.21.15.22:0/3153136568 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb0e01545b0 0x7fb0e0152970 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 >> 172.21.15.22:0/3153136568 conn(0x7fb0e00b3f10 msgr2=0x7fb0e015ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 shutdown_connections 2024-09-21T05:42:14.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.285+0000 7fb0e7ca4640 1 -- 172.21.15.22:0/3153136568 wait complete. 2024-09-21T05:42:14.298 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T05:42:14.298 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:42:14.298 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:42:14.337 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:42:14.380 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:42:14.380 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:42:14.380 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:14.380 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:14.397 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:14.397 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:14.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:42:14.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 -- 172.21.15.22:0/2697255939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100b5210 msgr2=0x7fbf100ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 --2- 172.21.15.22:0/2697255939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100b5210 0x7fbf100ceb50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009830 tx=0x7fbef802a5c0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 -- 172.21.15.22:0/2697255939 shutdown_connections 2024-09-21T05:42:14.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 --2- 172.21.15.22:0/2697255939 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100b5210 0x7fbf100ceb50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 -- 172.21.15.22:0/2697255939 >> 172.21.15.22:0/2697255939 conn(0x7fbf100b3f10 msgr2=0x7fbf100b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:14.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 -- 172.21.15.22:0/2697255939 shutdown_connections 2024-09-21T05:42:14.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.608+0000 7fbf15a38640 1 -- 172.21.15.22:0/2697255939 wait complete. 2024-09-21T05:42:14.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.610+0000 7fbf15a38640 1 Processor -- start 2024-09-21T05:42:14.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.610+0000 7fbf15a38640 1 -- start start 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.611+0000 7fbf15a38640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf100b5210 0x7fbf1015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.611+0000 7fbf15a38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf100ccba0 0x7fbf10145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.611+0000 7fbf15a38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 0x7fbf10146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.611+0000 7fbf15a38640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100d1440 con 0x7fbf100d1d40 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.611+0000 7fbf15a38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbf100d12c0 con 0x7fbf100ccba0 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.611+0000 7fbf15a38640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbf100d15c0 con 0x7fbf100b5210 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 0x7fbf10146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 0x7fbf10146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33502/0 (socket says 172.21.15.22:33502) 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 -- 172.21.15.22:0/1042014384 learned_addr learned my addr 172.21.15.22:0/1042014384 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0e7fc640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf100ccba0 0x7fbf10145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0effd640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf100b5210 0x7fbf1015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 -- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf100b5210 msgr2=0x7fbf1015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf100b5210 0x7fbf1015f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 -- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf100ccba0 msgr2=0x7fbf10145bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf100ccba0 0x7fbf10145bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 -- 172.21.15.22:0/1042014384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf10144b20 con 0x7fbf100d1d40 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf0f7fe640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 0x7fbf10146100 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800c3f0 tx=0x7fbf0800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08011790 con 0x7fbf100d1d40 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf08017070 con 0x7fbf100d1d40 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08011ca0 con 0x7fbf100d1d40 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf10144e00 con 0x7fbf100d1d40 2024-09-21T05:42:14.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.612+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101453c0 con 0x7fbf100d1d40 2024-09-21T05:42:14.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.613+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf100ceb50 con 0x7fbf100d1d40 2024-09-21T05:42:14.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.614+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf0801c460 con 0x7fbf100d1d40 2024-09-21T05:42:14.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.615+0000 7fbf07fff640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbef0078b90 0x7fbef007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:14.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.615+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fbf0809aae0 con 0x7fbf100d1d40 2024-09-21T05:42:14.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.615+0000 7fbf0effd640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbef0078b90 0x7fbef007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.616+0000 7fbf0effd640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbef0078b90 0x7fbef007b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007fd0 tx=0x7fbf00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:14.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.620+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf08062f20 con 0x7fbf100d1d40 2024-09-21T05:42:14.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.818+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbf10146640 con 0x7fbf100d1d40 2024-09-21T05:42:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.819+0000 7fbf07fff640 1 -- 172.21.15.22:0/1042014384 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) ==== 80+0+0 (secure 0 0 0) 0x7fbf08001080 con 0x7fbf100d1d40 2024-09-21T05:42:14.821 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbef0078b90 msgr2=0x7fbef007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbef0078b90 0x7fbef007b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf00007fd0 tx=0x7fbf00009040 comp rx=0 tx=0).stop 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 msgr2=0x7fbf10146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 0x7fbf10146100 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800c3f0 tx=0x7fbf0800c8b0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 shutdown_connections 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbf100d1d40 0x7fbf10146100 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbf100ccba0 0x7fbf10145bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbef0078b90 0x7fbef007b0a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 --2- 172.21.15.22:0/1042014384 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbf100b5210 0x7fbf1015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 >> 172.21.15.22:0/1042014384 conn(0x7fbf100b3f10 msgr2=0x7fbf100b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 shutdown_connections 2024-09-21T05:42:14.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:14.821+0000 7fbf15a38640 1 -- 172.21.15.22:0/1042014384 wait complete. 2024-09-21T05:42:14.834 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:42:14.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:14.834 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:14.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:42:14.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:14.878 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:42:14.934 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:42:14.935 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:42:14.935 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:14.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:42:14.951 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:42:14.951 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:14.952 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:14.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:42:14.991 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:42:14.991 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:42:14.991 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:15.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:42:15.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.156+0000 7fe0be269640 1 -- 172.21.15.22:0/1183474303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a4004800 msgr2=0x7fe0a4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.156+0000 7fe0be269640 1 --2- 172.21.15.22:0/1183474303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a4004800 0x7fe0a4004c10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fe0b80d4500 tx=0x7fe0a4005830 comp rx=0 tx=0).stop 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.157+0000 7fe0be269640 1 -- 172.21.15.22:0/1183474303 shutdown_connections 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.157+0000 7fe0be269640 1 --2- 172.21.15.22:0/1183474303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0a4004800 0x7fe0a4004c10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.157+0000 7fe0be269640 1 -- 172.21.15.22:0/1183474303 >> 172.21.15.22:0/1183474303 conn(0x7fe0b809d840 msgr2=0x7fe0b808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.157+0000 7fe0be269640 1 -- 172.21.15.22:0/1183474303 shutdown_connections 2024-09-21T05:42:15.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.157+0000 7fe0be269640 1 -- 172.21.15.22:0/1183474303 wait complete. 2024-09-21T05:42:15.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.158+0000 7fe0be269640 1 Processor -- start 2024-09-21T05:42:15.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.159+0000 7fe0be269640 1 -- start start 2024-09-21T05:42:15.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0be269640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0a4004800 0x7fe0b81f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0be269640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 0x7fe0b81613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0be269640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0b8159850 0x7fe0b8159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0be269640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b806fb20 con 0x7fe0b8160fe0 2024-09-21T05:42:15.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0be269640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b806f9a0 con 0x7fe0a4004800 2024-09-21T05:42:15.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0be269640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b806e610 con 0x7fe0b8159850 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 0x7fe0b81613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 0x7fe0b81613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33526/0 (socket says 172.21.15.22:33526) 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 -- 172.21.15.22:0/623311932 learned_addr learned my addr 172.21.15.22:0/623311932 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b7fff640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0a4004800 0x7fe0b81f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0bc806640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0b8159850 0x7fe0b8159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 -- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0b8159850 msgr2=0x7fe0b8159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0b8159850 0x7fe0b8159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 -- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0a4004800 msgr2=0x7fe0b81f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0a4004800 0x7fe0b81f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.160+0000 7fe0b77fe640 1 -- 172.21.15.22:0/623311932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b815fbc0 con 0x7fe0b8160fe0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.161+0000 7fe0b77fe640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 0x7fe0b81613f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0001780 tx=0x7fe0a002af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.161+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0042070 con 0x7fe0b8160fe0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.161+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0a0039070 con 0x7fe0b8160fe0 2024-09-21T05:42:15.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.161+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b81f6320 con 0x7fe0b8160fe0 2024-09-21T05:42:15.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.161+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0003830 con 0x7fe0b8160fe0 2024-09-21T05:42:15.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.161+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b815c8d0 con 0x7fe0b8160fe0 2024-09-21T05:42:15.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.162+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b806ea50 con 0x7fe0b8160fe0 2024-09-21T05:42:15.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.163+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0a003b070 con 0x7fe0b8160fe0 2024-09-21T05:42:15.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.163+0000 7fe0b4ff9640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe098078b90 0x7fe09807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.163+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a00ba170 con 0x7fe0b8160fe0 2024-09-21T05:42:15.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.163+0000 7fe0b7fff640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe098078b90 0x7fe09807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.164+0000 7fe0b7fff640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe098078b90 0x7fe09807b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80012d0 tx=0x7fe0a80066f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:15.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.168+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0b81f6320 con 0x7fe0b8160fe0 2024-09-21T05:42:15.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.391+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0b80b8a60 con 0x7fe0b8160fe0 2024-09-21T05:42:15.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.392+0000 7fe0b4ff9640 1 -- 172.21.15.22:0/623311932 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5227 (secure 0 0 0) 0x7fe0a0081ee0 con 0x7fe0b8160fe0 2024-09-21T05:42:15.394 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:15.394 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-21T05:42:13:919182+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:42:13.660372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:42:15.394 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe098078b90 msgr2=0x7fe09807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe098078b90 0x7fe09807b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80012d0 tx=0x7fe0a80066f0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 msgr2=0x7fe0b81613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 0x7fe0b81613f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0001780 tx=0x7fe0a002af40 comp rx=0 tx=0).stop 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 shutdown_connections 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe098078b90 0x7fe09807b0a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe0b8159850 0x7fe0b8159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe0b8160fe0 0x7fe0b81613f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 --2- 172.21.15.22:0/623311932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe0a4004800 0x7fe0b81f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 >> 172.21.15.22:0/623311932 conn(0x7fe0b809d840 msgr2=0x7fe0b815bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:15.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 shutdown_connections 2024-09-21T05:42:15.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.394+0000 7fe0be269640 1 -- 172.21.15.22:0/623311932 wait complete. 2024-09-21T05:42:15.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:15.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 -- 172.21.15.22:0/1164170010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0b4530 msgr2=0x7fb74c15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 --2- 172.21.15.22:0/1164170010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0b4530 0x7fb74c15c830 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb73c005fd0 tx=0x7fb73c02a540 comp rx=0 tx=0).stop 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 -- 172.21.15.22:0/1164170010 shutdown_connections 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 --2- 172.21.15.22:0/1164170010 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c160c90 0x7fb74c0d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 --2- 172.21.15.22:0/1164170010 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0b4530 0x7fb74c15c830 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 -- 172.21.15.22:0/1164170010 >> 172.21.15.22:0/1164170010 conn(0x7fb74c0cee20 msgr2=0x7fb74c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 -- 172.21.15.22:0/1164170010 shutdown_connections 2024-09-21T05:42:15.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.522+0000 7fb753b24640 1 -- 172.21.15.22:0/1164170010 wait complete. 2024-09-21T05:42:15.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.524+0000 7fb753b24640 1 Processor -- start 2024-09-21T05:42:15.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.524+0000 7fb753b24640 1 -- start start 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.525+0000 7fb753b24640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 0x7fb74c0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.525+0000 7fb753b24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c160c90 0x7fb74c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.525+0000 7fb753b24640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c0b36d0 0x7fb74c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.525+0000 7fb753b24640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c0d17c0 con 0x7fb74c0cff70 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb753b24640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c0d1640 con 0x7fb74c160c90 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb753b24640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb74c0d1940 con 0x7fb74c0b36d0 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 0x7fb74c0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 0x7fb74c0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33548/0 (socket says 172.21.15.22:33548) 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 -- 172.21.15.22:0/4165671245 learned_addr learned my addr 172.21.15.22:0/4165671245 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 -- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c0b36d0 msgr2=0x7fb74c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c0b36d0 0x7fb74c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 -- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c160c90 msgr2=0x7fb74c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7510bf640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c160c90 0x7fb74c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c160c90 0x7fb74c0c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 -- 172.21.15.22:0/4165671245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c073000 con 0x7fb74c0cff70 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7518c0640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 0x7fb74c0b0ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c3d0 tx=0x7fb74800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748017070 con 0x7fb74c0cff70 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7480053c0 con 0x7fb74c0cff70 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748003850 con 0x7fb74c0cff70 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74c06fbd0 con 0x7fb74c0cff70 2024-09-21T05:42:15.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.526+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74c0b45d0 con 0x7fb74c0cff70 2024-09-21T05:42:15.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.527+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74c0d1e60 con 0x7fb74c0cff70 2024-09-21T05:42:15.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.528+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb74801a070 con 0x7fb74c0cff70 2024-09-21T05:42:15.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.529+0000 7fb7427fc640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb730078b90 0x7fb73007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.529+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fb748099fe0 con 0x7fb74c0cff70 2024-09-21T05:42:15.531 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.529+0000 7fb7510bf640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb730078b90 0x7fb73007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.530+0000 7fb7510bf640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb730078b90 0x7fb73007b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb73c033000 tx=0x7fb73c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:15.535 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.533+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb748062540 con 0x7fb74c0cff70 2024-09-21T05:42:15.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.763+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb74c0b40a0 con 0x7fb74c0cff70 2024-09-21T05:42:15.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.764+0000 7fb7427fc640 1 -- 172.21.15.22:0/4165671245 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5227 (secure 0 0 0) 0x7fb748061e40 con 0x7fb74c0cff70 2024-09-21T05:42:15.766 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:15.766 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-21T05:42:13:919182+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:42:13.660372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:42:15.767 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb730078b90 msgr2=0x7fb73007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb730078b90 0x7fb73007b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb73c033000 tx=0x7fb73c004730 comp rx=0 tx=0).stop 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 msgr2=0x7fb74c0b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 0x7fb74c0b0ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb74800c3d0 tx=0x7fb74800c890 comp rx=0 tx=0).stop 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 shutdown_connections 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb730078b90 0x7fb73007b0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb74c0b36d0 0x7fb74c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb74c160c90 0x7fb74c0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 --2- 172.21.15.22:0/4165671245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb74c0cff70 0x7fb74c0b0ea0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 >> 172.21.15.22:0/4165671245 conn(0x7fb74c0cee20 msgr2=0x7fb74c0725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 shutdown_connections 2024-09-21T05:42:15.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.766+0000 7fb753b24640 1 -- 172.21.15.22:0/4165671245 wait complete. 2024-09-21T05:42:15.780 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:42:15.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f33988cc640 1 -- 172.21.15.22:0/837503300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f337c004800 msgr2=0x7f337c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f33988cc640 1 --2- 172.21.15.22:0/837503300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f337c004800 0x7f337c004c10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3388006fc0 tx=0x7f337c005830 comp rx=0 tx=0).stop 2024-09-21T05:42:15.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f3392d9d640 1 -- 172.21.15.22:0/837503300 reap_dead start 2024-09-21T05:42:15.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f33988cc640 1 -- 172.21.15.22:0/837503300 shutdown_connections 2024-09-21T05:42:15.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f33988cc640 1 -- 172.21.15.22:0/837503300 >> 172.21.15.22:0/837503300 conn(0x7f339409d7f0 msgr2=0x7f33940d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:15.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f33988cc640 1 -- 172.21.15.22:0/837503300 shutdown_connections 2024-09-21T05:42:15.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.894+0000 7f33988cc640 1 -- 172.21.15.22:0/837503300 wait complete. 2024-09-21T05:42:15.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.896+0000 7f33988cc640 1 Processor -- start 2024-09-21T05:42:15.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.896+0000 7f33988cc640 1 -- start start 2024-09-21T05:42:15.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f337c004800 0x7f33940cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 0x7f33940b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33941f9210 0x7f33940b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f339406e7f0 con 0x7f3394161a40 2024-09-21T05:42:15.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f33940d5010 con 0x7f33941f9210 2024-09-21T05:42:15.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f339406e970 con 0x7f337c004800 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 0x7f33940b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 0x7f33940b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33564/0 (socket says 172.21.15.22:33564) 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 -- 172.21.15.22:0/1438105486 learned_addr learned my addr 172.21.15.22:0/1438105486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3392d9d640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33941f9210 0x7f33940b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f339259c640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f337c004800 0x7f33940cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 -- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f337c004800 msgr2=0x7f33940cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f337c004800 0x7f33940cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 -- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33941f9210 msgr2=0x7f33940b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33941f9210 0x7f33940b6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 -- 172.21.15.22:0/1438105486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33940cd280 con 0x7f3394161a40 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3392d9d640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33941f9210 0x7f33940b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3391d9b640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 0x7f33940b5de0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f338802b0f0 tx=0x7f3388001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388031040 con 0x7f3394161a40 2024-09-21T05:42:15.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33880035f0 con 0x7f3394161a40 2024-09-21T05:42:15.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33940cd560 con 0x7f3394161a40 2024-09-21T05:42:15.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388003a20 con 0x7f3394161a40 2024-09-21T05:42:15.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.898+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33940cdb20 con 0x7f3394161a40 2024-09-21T05:42:15.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.899+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339406e0e0 con 0x7f3394161a40 2024-09-21T05:42:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.900+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3388034070 con 0x7f3394161a40 2024-09-21T05:42:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.901+0000 7f3382ffd640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3374078b90 0x7f337407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.901+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f33880bad20 con 0x7f3394161a40 2024-09-21T05:42:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.901+0000 7f339259c640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3374078b90 0x7f337407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:15.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.902+0000 7f339259c640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3374078b90 0x7f337407b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3384006fd0 tx=0x7f3384006f40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:15.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:15.906+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33880830e0 con 0x7f3394161a40 2024-09-21T05:42:16.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.096+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33940d2960 con 0x7f3394161a40 2024-09-21T05:42:16.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.098+0000 7f3382ffd640 1 -- 172.21.15.22:0/1438105486 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16713 (secure 0 0 0) 0x7f33880829e0 con 0x7f3394161a40 2024-09-21T05:42:16.100 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:16.100 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":57,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:34.632274+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:41:30.686369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T05:41:31.966340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:42:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3374078b90 msgr2=0x7f337407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3374078b90 0x7f337407b0a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3384006fd0 tx=0x7f3384006f40 comp rx=0 tx=0).stop 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 msgr2=0x7f33940b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 0x7f33940b5de0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f338802b0f0 tx=0x7f3388001cb0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 shutdown_connections 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f33941f9210 0x7f33940b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3394161a40 0x7f33940b5de0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3374078b90 0x7f337407b0a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 --2- 172.21.15.22:0/1438105486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f337c004800 0x7f33940cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 >> 172.21.15.22:0/1438105486 conn(0x7f339409d7f0 msgr2=0x7f33940d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 shutdown_connections 2024-09-21T05:42:16.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.100+0000 7f33988cc640 1 -- 172.21.15.22:0/1438105486 wait complete. 2024-09-21T05:42:16.114 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:42:16.114 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 -- 172.21.15.22:0/2477822072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 msgr2=0x7f78380cf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 --2- 172.21.15.22:0/2477822072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f78380cf9e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f783808d4e0 tx=0x7f783002a560 comp rx=0 tx=0).stop 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 -- 172.21.15.22:0/2477822072 shutdown_connections 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 --2- 172.21.15.22:0/2477822072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f78380cf9e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 --2- 172.21.15.22:0/2477822072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78380b9b70 0x7f78380cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 --2- 172.21.15.22:0/2477822072 >> v2:172.21.15.134:3300/0 conn(0x7f78380d2ed0 0x7f78380d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 -- 172.21.15.22:0/2477822072 >> 172.21.15.22:0/2477822072 conn(0x7f783808bfc0 msgr2=0x7f783809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:16.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 -- 172.21.15.22:0/2477822072 shutdown_connections 2024-09-21T05:42:16.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.228+0000 7f783cff9640 1 -- 172.21.15.22:0/2477822072 wait complete. 2024-09-21T05:42:16.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.230+0000 7f783cff9640 1 Processor -- start 2024-09-21T05:42:16.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.230+0000 7f783cff9640 1 -- start start 2024-09-21T05:42:16.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783cff9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78380b9b70 0x7f7838155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783cff9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f783809eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783cff9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78380d2ed0 0x7f78380ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783cff9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f78380d4640 con 0x7f78380cf5d0 2024-09-21T05:42:16.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783cff9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78380d44c0 con 0x7f78380b9b70 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783cff9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f78380d47c0 con 0x7f78380d2ed0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f783809eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f783809eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33584/0 (socket says 172.21.15.22:33584) 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 -- 172.21.15.22:0/1072607101 learned_addr learned my addr 172.21.15.22:0/1072607101 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7836d9d640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78380d2ed0 0x7f78380ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f783659c640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78380b9b70 0x7f7838155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 -- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78380d2ed0 msgr2=0x7f78380ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78380d2ed0 0x7f78380ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 -- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78380b9b70 msgr2=0x7f7838155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78380b9b70 0x7f7838155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 -- 172.21.15.22:0/1072607101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78381561a0 con 0x7f78380cf5d0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.232+0000 7f7835d9b640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f783809eb80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f782000c740 tx=0x7f782000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.233+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820007bb0 con 0x7f78380cf5d0 2024-09-21T05:42:16.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.233+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f782000ce50 con 0x7f78380cf5d0 2024-09-21T05:42:16.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.233+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78380a1450 con 0x7f78380cf5d0 2024-09-21T05:42:16.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.233+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820005280 con 0x7f78380cf5d0 2024-09-21T05:42:16.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.233+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78380a1a10 con 0x7f78380cf5d0 2024-09-21T05:42:16.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.234+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78380cf9e0 con 0x7f78380cf5d0 2024-09-21T05:42:16.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.235+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7820005420 con 0x7f78380cf5d0 2024-09-21T05:42:16.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.235+0000 7f782effd640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7818078b90 0x7f781807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.235+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f782009b190 con 0x7f78380cf5d0 2024-09-21T05:42:16.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.235+0000 7f783659c640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7818078b90 0x7f781807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.236+0000 7f783659c640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7818078b90 0x7f781807b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7828005be0 tx=0x7f782800a740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:16.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.240+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7820014030 con 0x7f78380cf5d0 2024-09-21T05:42:16.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.468+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78380b4020 con 0x7f78380cf5d0 2024-09-21T05:42:16.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.469+0000 7f782effd640 1 -- 172.21.15.22:0/1072607101 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5227 (secure 0 0 0) 0x7f78200635e0 con 0x7f78380cf5d0 2024-09-21T05:42:16.471 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:16.472 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-21T05:42:13:919182+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:42:13.660372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:42:16.472 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7818078b90 msgr2=0x7f781807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7818078b90 0x7f781807b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7828005be0 tx=0x7f782800a740 comp rx=0 tx=0).stop 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 msgr2=0x7f783809eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f783809eb80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f782000c740 tx=0x7f782000cc00 comp rx=0 tx=0).stop 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 shutdown_connections 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7818078b90 0x7f781807b0a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78380d2ed0 0x7f78380ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f78380cf5d0 0x7f783809eb80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 --2- 172.21.15.22:0/1072607101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f78380b9b70 0x7f7838155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 >> 172.21.15.22:0/1072607101 conn(0x7f783808bfc0 msgr2=0x7f78380b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:16.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.471+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 shutdown_connections 2024-09-21T05:42:16.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.472+0000 7f783cff9640 1 -- 172.21.15.22:0/1072607101 wait complete. 2024-09-21T05:42:16.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:16.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 -- 172.21.15.22:0/4024464622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000ced80 msgr2=0x7f1a000cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 --2- 172.21.15.22:0/4024464622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000ced80 0x7f1a000cf190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1a0008d160 tx=0x7f19f802aa30 comp rx=0 tx=0).stop 2024-09-21T05:42:16.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 -- 172.21.15.22:0/4024464622 shutdown_connections 2024-09-21T05:42:16.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 --2- 172.21.15.22:0/4024464622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000ced80 0x7f1a000cf190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 --2- 172.21.15.22:0/4024464622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a000b5fa0 0x7f1a000b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 --2- 172.21.15.22:0/4024464622 >> v2:172.21.15.134:3300/0 conn(0x7f1a000b9970 0x7f1a000b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 -- 172.21.15.22:0/4024464622 >> 172.21.15.22:0/4024464622 conn(0x7f1a0009d7f0 msgr2=0x7f1a000d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:16.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 -- 172.21.15.22:0/4024464622 shutdown_connections 2024-09-21T05:42:16.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.600+0000 7f1a04c94640 1 -- 172.21.15.22:0/4024464622 wait complete. 2024-09-21T05:42:16.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.602+0000 7f1a04c94640 1 Processor -- start 2024-09-21T05:42:16.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.602+0000 7f1a04c94640 1 -- start start 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f1a04c94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a000b5fa0 0x7f1a000a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f1a04c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 0x7f1a0015d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f1a04c94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a000ced80 0x7f1a0006e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f1a04c94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a000cf850 con 0x7f1a000b9970 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f1a04c94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a000cf6d0 con 0x7f1a000b5fa0 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f1a04c94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a000cf9d0 con 0x7f1a000ced80 2024-09-21T05:42:16.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.603+0000 7f19fdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 0x7f1a0015d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 0x7f1a0015d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33608/0 (socket says 172.21.15.22:33608) 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 -- 172.21.15.22:0/2270242552 learned_addr learned my addr 172.21.15.22:0/2270242552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fe59c640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a000b5fa0 0x7f1a000a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fed9d640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a000ced80 0x7f1a0006e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 -- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a000ced80 msgr2=0x7f1a0006e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a000ced80 0x7f1a0006e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 -- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a000b5fa0 msgr2=0x7f1a000a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a000b5fa0 0x7f1a000a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 -- 172.21.15.22:0/2270242552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a000cae80 con 0x7f1a000b9970 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19fdd9b640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 0x7f1a0015d820 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c940 tx=0x7f19e800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8007bb0 con 0x7f1a000b9970 2024-09-21T05:42:16.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a00161430 con 0x7f1a000b9970 2024-09-21T05:42:16.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19e8007e40 con 0x7f1a000b9970 2024-09-21T05:42:16.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800f690 con 0x7f1a000b9970 2024-09-21T05:42:16.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.604+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a000caba0 con 0x7f1a000b9970 2024-09-21T05:42:16.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.605+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a000ca910 con 0x7f1a000b9970 2024-09-21T05:42:16.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.606+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f19e8005020 con 0x7f1a000b9970 2024-09-21T05:42:16.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.607+0000 7f19f6ffd640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19e0078b90 0x7f19e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.607+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f19e809b710 con 0x7f1a000b9970 2024-09-21T05:42:16.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.607+0000 7f19fe59c640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19e0078b90 0x7f19e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.608+0000 7f19fe59c640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19e0078b90 0x7f19e007b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19f000ab20 tx=0x7f19f000a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:16.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.610+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19e8014030 con 0x7f1a000b9970 2024-09-21T05:42:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.837+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a000cfe80 con 0x7f1a000b9970 2024-09-21T05:42:16.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.837+0000 7f19f6ffd640 1 -- 172.21.15.22:0/2270242552 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5227 (secure 0 0 0) 0x7f19e8063580 con 0x7f1a000b9970 2024-09-21T05:42:16.840 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:16.840 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":36,"btime":"2024-09-21T05:42:13:919182+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:41:33.013872+0000","modified":"2024-09-21T05:42:13.660372+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":13,"addr":"172.21.15.22:6835/913296056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":913296056},{"type":"v1","addr":"172.21.15.22:6835","nonce":913296056}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5679,"qdb_cluster":[5679]},"id":4}]} 2024-09-21T05:42:16.840 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 36 2024-09-21T05:42:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19e0078b90 msgr2=0x7f19e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19e0078b90 0x7f19e007b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f19f000ab20 tx=0x7f19f000a250 comp rx=0 tx=0).stop 2024-09-21T05:42:16.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 msgr2=0x7f1a0015d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 0x7f1a0015d820 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f19e800c940 tx=0x7f19e800ce00 comp rx=0 tx=0).stop 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 shutdown_connections 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19e0078b90 0x7f19e007b0a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a000ced80 0x7f1a0006e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a000b9970 0x7f1a0015d820 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 --2- 172.21.15.22:0/2270242552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a000b5fa0 0x7f1a000a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 >> 172.21.15.22:0/2270242552 conn(0x7f1a0009d7f0 msgr2=0x7f1a000b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 shutdown_connections 2024-09-21T05:42:16.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.840+0000 7f1a04c94640 1 -- 172.21.15.22:0/2270242552 wait complete. 2024-09-21T05:42:16.853 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:42:16.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da7f83640 1 -- 172.21.15.22:0/4127535977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d880039e0 msgr2=0x7f5d88003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da7f83640 1 --2- 172.21.15.22:0/4127535977 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5d880039e0 0x7f5d88003e10 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c650 tx=0x7f5d88005830 comp rx=0 tx=0).stop 2024-09-21T05:42:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da6520640 1 -- 172.21.15.22:0/4127535977 reap_dead start 2024-09-21T05:42:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da7f83640 1 -- 172.21.15.22:0/4127535977 shutdown_connections 2024-09-21T05:42:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da7f83640 1 -- 172.21.15.22:0/4127535977 >> 172.21.15.22:0/4127535977 conn(0x7f5da008bfc0 msgr2=0x7f5da009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da7f83640 1 -- 172.21.15.22:0/4127535977 shutdown_connections 2024-09-21T05:42:16.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.969+0000 7f5da7f83640 1 -- 172.21.15.22:0/4127535977 wait complete. 2024-09-21T05:42:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.971+0000 7f5da7f83640 1 Processor -- start 2024-09-21T05:42:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.971+0000 7f5da7f83640 1 -- start start 2024-09-21T05:42:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da7f83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 0x7f5da00c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da7f83640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da01f4a90 0x7f5da01f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da7f83640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5da00cb5c0 0x7f5da00cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da7f83640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00d36c0 con 0x7f5da00ccb40 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da7f83640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5da00d3540 con 0x7f5da01f4a90 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da7f83640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5da00cf500 con 0x7f5da00cb5c0 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da5d1f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 0x7f5da00c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da5d1f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 0x7f5da00c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33632/0 (socket says 172.21.15.22:33632) 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.972+0000 7f5da5d1f640 1 -- 172.21.15.22:0/3758643923 learned_addr learned my addr 172.21.15.22:0/3758643923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da551e640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da01f4a90 0x7f5da01f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da6520640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5da00cb5c0 0x7f5da00cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da5d1f640 1 -- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5da00cb5c0 msgr2=0x7f5da00cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da5d1f640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5da00cb5c0 0x7f5da00cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da5d1f640 1 -- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da01f4a90 msgr2=0x7f5da01f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:16.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da5d1f640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da01f4a90 0x7f5da01f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da5d1f640 1 -- 172.21.15.22:0/3758643923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00c8b40 con 0x7f5da00ccb40 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da551e640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da01f4a90 0x7f5da01f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da5d1f640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 0x7f5da00c8020 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5d9002b4d0 tx=0x7f5d9000c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90042070 con 0x7f5da00ccb40 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d9002bad0 con 0x7f5da00ccb40 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da00c8de0 con 0x7f5da00ccb40 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90039070 con 0x7f5da00ccb40 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.973+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01f59d0 con 0x7f5da00ccb40 2024-09-21T05:42:16.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.974+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da006e170 con 0x7f5da00ccb40 2024-09-21T05:42:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.975+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5d90045070 con 0x7f5da00ccb40 2024-09-21T05:42:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.975+0000 7f5d9e7fc640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d8c078b90 0x7f5d8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.975+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f5d900bad30 con 0x7f5da00ccb40 2024-09-21T05:42:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.976+0000 7f5da551e640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d8c078b90 0x7f5d8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:16.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.976+0000 7f5da551e640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d8c078b90 0x7f5d8c07b0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5da008e710 tx=0x7f5d94005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:16.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:16.980+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5d90083200 con 0x7f5da00ccb40 2024-09-21T05:42:17.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.168+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5da0069180 con 0x7f5da00ccb40 2024-09-21T05:42:17.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.169+0000 7f5d9e7fc640 1 -- 172.21.15.22:0/3758643923 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16713 (secure 0 0 0) 0x7f5d90082b00 con 0x7f5da00ccb40 2024-09-21T05:42:17.172 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:17.172 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":57,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:41:34.632274+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:41:30.686369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T05:41:31.966340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:42:17.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.171+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d8c078b90 msgr2=0x7f5d8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.171+0000 7f5da7f83640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d8c078b90 0x7f5d8c07b0a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5da008e710 tx=0x7f5d94005f90 comp rx=0 tx=0).stop 2024-09-21T05:42:17.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.171+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 msgr2=0x7f5da00c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.171+0000 7f5da7f83640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 0x7f5da00c8020 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5d9002b4d0 tx=0x7f5d9000c320 comp rx=0 tx=0).stop 2024-09-21T05:42:17.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 shutdown_connections 2024-09-21T05:42:17.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5d8c078b90 0x7f5d8c07b0a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5da00cb5c0 0x7f5da00cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5da01f4a90 0x7f5da01f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 --2- 172.21.15.22:0/3758643923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5da00ccb40 0x7f5da00c8020 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 >> 172.21.15.22:0/3758643923 conn(0x7f5da008bfc0 msgr2=0x7f5da00d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:17.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 shutdown_connections 2024-09-21T05:42:17.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.172+0000 7f5da7f83640 1 -- 172.21.15.22:0/3758643923 wait complete. 2024-09-21T05:42:17.186 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:42:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 -- 172.21.15.22:0/1129740834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 msgr2=0x7fa854003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 --2- 172.21.15.22:0/1129740834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa854003ec0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c310 tx=0x7fa854005830 comp rx=0 tx=0).stop 2024-09-21T05:42:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 -- 172.21.15.22:0/1129740834 shutdown_connections 2024-09-21T05:42:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 --2- 172.21.15.22:0/1129740834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa854003ec0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 -- 172.21.15.22:0/1129740834 >> 172.21.15.22:0/1129740834 conn(0x7fa86c09d840 msgr2=0x7fa86c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 -- 172.21.15.22:0/1129740834 shutdown_connections 2024-09-21T05:42:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.302+0000 7fa872469640 1 -- 172.21.15.22:0/1129740834 wait complete. 2024-09-21T05:42:17.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.303+0000 7fa872469640 1 Processor -- start 2024-09-21T05:42:17.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.304+0000 7fa872469640 1 -- start start 2024-09-21T05:42:17.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa872469640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa86c0a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa872469640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa86c0cd060 0x7fa86c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa872469640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c1f87e0 0x7fa86c1f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa872469640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c06fb20 con 0x7fa854003a50 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa872469640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c06f9a0 con 0x7fa86c1f87e0 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa872469640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa86c06e610 con 0x7fa86c0cd060 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa86c0a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa86c0a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33650/0 (socket says 172.21.15.22:33650) 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 -- 172.21.15.22:0/725470703 learned_addr learned my addr 172.21.15.22:0/725470703 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa870a06640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c1f87e0 0x7fa86c1f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86b7fe640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa86c0cd060 0x7fa86c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 -- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa86c0cd060 msgr2=0x7fa86c0cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa86c0cd060 0x7fa86c0cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 -- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c1f87e0 msgr2=0x7fa86c1f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c1f87e0 0x7fa86c1f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa86bfff640 1 -- 172.21.15.22:0/725470703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86c1f9130 con 0x7fa854003a50 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.305+0000 7fa870a06640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c1f87e0 0x7fa86c1f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.306+0000 7fa86bfff640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa86c0a18d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c430 tx=0x7fa85c001100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.306+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c031050 con 0x7fa854003a50 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.306+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa85c0038e0 con 0x7fa854003a50 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.306+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c003d10 con 0x7fa854003a50 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.306+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c0cdd20 con 0x7fa854003a50 2024-09-21T05:42:17.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.306+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c14ca30 con 0x7fa854003a50 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.307+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa86c0c9310 con 0x7fa854003a50 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.308+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa85c02d440 con 0x7fa854003a50 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.308+0000 7fa868ff9640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa858078bc0 0x7fa85807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.308+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7fa85c0bb490 con 0x7fa854003a50 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.308+0000 7fa86b7fe640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa858078bc0 0x7fa85807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.309+0000 7fa86b7fe640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa858078bc0 0x7fa85807b0d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ce520 tx=0x7fa860006a80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:17.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.313+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa86c0c9310 con 0x7fa854003a50 2024-09-21T05:42:17.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.536+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa86c06ea50 con 0x7fa854003a50 2024-09-21T05:42:17.538 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:42:17.536+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:42:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.538+0000 7fa868ff9640 1 -- 172.21.15.22:0/725470703 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7fa85c046060 con 0x7fa854003a50 2024-09-21T05:42:17.540 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:42:17.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa858078bc0 msgr2=0x7fa85807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa858078bc0 0x7fa85807b0d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0ce520 tx=0x7fa860006a80 comp rx=0 tx=0).stop 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 msgr2=0x7fa86c0a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa86c0a18d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c430 tx=0x7fa85c001100 comp rx=0 tx=0).stop 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 shutdown_connections 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa86c1f87e0 0x7fa86c1f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa858078bc0 0x7fa85807b0d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa86c0cd060 0x7fa86c0cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 --2- 172.21.15.22:0/725470703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa854003a50 0x7fa86c0a18d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 >> 172.21.15.22:0/725470703 conn(0x7fa86c09d840 msgr2=0x7fa86c0c9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 shutdown_connections 2024-09-21T05:42:17.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.541+0000 7fa872469640 1 -- 172.21.15.22:0/725470703 wait complete. 2024-09-21T05:42:17.547 INFO:tasks.ceph.mds.c.smithi022.stderr: -7402> 2024-09-21T05:41:38.920+0000 7f13b0349640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:42:17.550 INFO:tasks.ceph.mds.c.smithi022.stderr: -4337> 2024-09-21T05:41:41.757+0000 7f13ac341640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 pv4 ap=4 s=0 n(v0 1=1+0)->n(v0 rc2024-09-21T05:41:40.688007+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={5264=0-4194304@1} caps={5264=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=5264 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=0 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x55c929782b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:42:17.552 INFO:tasks.ceph.mds.c.smithi022.stderr: -2124> 2024-09-21T05:42:13.205+0000 7f13ac341640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 pv6 ap=4 DIRTYPARENT s=0 n(v0 rc2024-09-21T05:41:40.688007+0000 1=1+0)->n(v0 rc2024-09-21T05:41:40.688007+0000 1=1+0) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={5264=0-4194304@1} caps={5264=pAsLsXs/-@6} | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55c929782b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:42:17.556 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:42:17.607 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:42:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 -- 172.21.15.22:0/3506109321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340b9b60 msgr2=0x7f34340d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 --2- 172.21.15.22:0/3506109321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340b9b60 0x7f34340d4d80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f342c0098e0 tx=0x7f342c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 -- 172.21.15.22:0/3506109321 shutdown_connections 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 --2- 172.21.15.22:0/3506109321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34340b66b0 0x7f343406eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 --2- 172.21.15.22:0/3506109321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340b9b60 0x7f34340d4d80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 -- 172.21.15.22:0/3506109321 >> 172.21.15.22:0/3506109321 conn(0x7f343409d7f0 msgr2=0x7f34340d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 -- 172.21.15.22:0/3506109321 shutdown_connections 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.676+0000 7f343b307640 1 -- 172.21.15.22:0/3506109321 wait complete. 2024-09-21T05:42:17.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.678+0000 7f343b307640 1 Processor -- start 2024-09-21T05:42:17.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.678+0000 7f343b307640 1 -- start start 2024-09-21T05:42:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f343b307640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34340b9b60 0x7f34341508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f343b307640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34340cf540 0x7f34340d41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f343b307640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 0x7f34340d4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f343b307640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f343406f630 con 0x7f34340c8050 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f343b307640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f343406f4b0 con 0x7f34340cf540 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f343b307640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f343406f7b0 con 0x7f34340b9b60 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 0x7f34340d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 0x7f34340d4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33686/0 (socket says 172.21.15.22:33686) 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 -- 172.21.15.22:0/1886345805 learned_addr learned my addr 172.21.15.22:0/1886345805 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34390a3640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34340b9b60 0x7f34341508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34388a2640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34340cf540 0x7f34340d41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 -- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34340b9b60 msgr2=0x7f34341508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34340b9b60 0x7f34341508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 -- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34340cf540 msgr2=0x7f34340d41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34340cf540 0x7f34340d41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.679+0000 7f34398a4640 1 -- 172.21.15.22:0/1886345805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34340c8830 con 0x7f34340c8050 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.680+0000 7f34398a4640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 0x7f34340d4710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f342400d9e0 tx=0x7f342400dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.680+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424017070 con 0x7f34340c8050 2024-09-21T05:42:17.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.680+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342400be30 con 0x7f34340c8050 2024-09-21T05:42:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.680+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34340c8b10 con 0x7f34340c8050 2024-09-21T05:42:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.680+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424003bb0 con 0x7f34340c8050 2024-09-21T05:42:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.680+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34340c90d0 con 0x7f34340c8050 2024-09-21T05:42:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.681+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343406fcd0 con 0x7f34340c8050 2024-09-21T05:42:17.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.682+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3424010460 con 0x7f34340c8050 2024-09-21T05:42:17.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.683+0000 7f3431ffb640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3420078b90 0x7f342007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:17.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.683+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5795+0+0 (secure 0 0 0) 0x7f342409b0a0 con 0x7f34340c8050 2024-09-21T05:42:17.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.683+0000 7f34390a3640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3420078b90 0x7f342007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:17.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.684+0000 7f34390a3640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3420078b90 0x7f342007b0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f342c034000 tx=0x7f342c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:17.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.688+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3424062ec0 con 0x7f34340c8050 2024-09-21T05:42:17.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:17.912+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f34340c85c0 con 0x7f34340c8050 2024-09-21T05:42:18.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.539+0000 7f3431ffb640 1 -- 172.21.15.22:0/1886345805 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f34240678f0 con 0x7f34340c8050 2024-09-21T05:42:18.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3420078b90 msgr2=0x7f342007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3420078b90 0x7f342007b0a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f342c034000 tx=0x7f342c004730 comp rx=0 tx=0).stop 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 msgr2=0x7f34340d4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 0x7f34340d4710 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f342400d9e0 tx=0x7f342400dea0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 shutdown_connections 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f34340c8050 0x7f34340d4710 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f34340cf540 0x7f34340d41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3420078b90 0x7f342007b0a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 --2- 172.21.15.22:0/1886345805 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f34340b9b60 0x7f34341508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 >> 172.21.15.22:0/1886345805 conn(0x7f343409d7f0 msgr2=0x7f34340b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 shutdown_connections 2024-09-21T05:42:18.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.542+0000 7f343b307640 1 -- 172.21.15.22:0/1886345805 wait complete. 2024-09-21T05:42:18.558 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:42:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d2c8b640 1 -- 172.21.15.22:0/253896633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21b4003a30 msgr2=0x7f21b4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/253896633 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21b4003a30 0x7f21b4003ea0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f21c800d420 tx=0x7f21b4005830 comp rx=0 tx=0).stop 2024-09-21T05:42:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d1228640 1 -- 172.21.15.22:0/253896633 reap_dead start 2024-09-21T05:42:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d2c8b640 1 -- 172.21.15.22:0/253896633 shutdown_connections 2024-09-21T05:42:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d2c8b640 1 -- 172.21.15.22:0/253896633 >> 172.21.15.22:0/253896633 conn(0x7f21cc0b1c40 msgr2=0x7f21cc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d2c8b640 1 -- 172.21.15.22:0/253896633 shutdown_connections 2024-09-21T05:42:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.683+0000 7f21d2c8b640 1 -- 172.21.15.22:0/253896633 wait complete. 2024-09-21T05:42:18.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.685+0000 7f21d2c8b640 1 Processor -- start 2024-09-21T05:42:18.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.685+0000 7f21d2c8b640 1 -- start start 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.686+0000 7f21d2c8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b4003a30 0x7f21cc1e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.686+0000 7f21d2c8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 0x7f21cc14f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d2c8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc1f3540 0x7f21cc1f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d2c8b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0ccfe0 con 0x7f21cc15dec0 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d2c8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0cce60 con 0x7f21cc1f3540 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d2c8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0cd160 con 0x7f21b4003a30 2024-09-21T05:42:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 0x7f21cc14f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 0x7f21cc14f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33718/0 (socket says 172.21.15.22:33718) 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 -- 172.21.15.22:0/3492121696 learned_addr learned my addr 172.21.15.22:0/3492121696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d0a27640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b4003a30 0x7f21cc1e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d1228640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc1f3540 0x7f21cc1f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 -- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b4003a30 msgr2=0x7f21cc1e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b4003a30 0x7f21cc1e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 -- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc1f3540 msgr2=0x7f21cc1f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc1f3540 0x7f21cc1f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 -- 172.21.15.22:0/3492121696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc14fad0 con 0x7f21cc15dec0 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d1228640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc1f3540 0x7f21cc1f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d0a27640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b4003a30 0x7f21cc1e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c3fff640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 0x7f21cc14f080 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f21bc010aa0 tx=0x7f21bc010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:18.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc00ebc0 con 0x7f21cc15dec0 2024-09-21T05:42:18.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21bc00ed60 con 0x7f21cc15dec0 2024-09-21T05:42:18.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc15ce20 con 0x7f21cc15dec0 2024-09-21T05:42:18.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0116a0 con 0x7f21cc15dec0 2024-09-21T05:42:18.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.687+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc15d3e0 con 0x7f21cc15dec0 2024-09-21T05:42:18.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.689+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc15cb90 con 0x7f21cc15dec0 2024-09-21T05:42:18.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.689+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21bc011840 con 0x7f21cc15dec0 2024-09-21T05:42:18.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.690+0000 7f21c17fa640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b8078b90 0x7f21b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:18.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.690+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5795+0+0 (secure 0 0 0) 0x7f21bc09b9b0 con 0x7f21cc15dec0 2024-09-21T05:42:18.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.690+0000 7f21d0a27640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b8078b90 0x7f21b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:18.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.691+0000 7f21d0a27640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b8078b90 0x7f21b807b0a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21c8034000 tx=0x7f21c80019f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:18.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.694+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21bc063e20 con 0x7f21cc15dec0 2024-09-21T05:42:18.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:18.883+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21cc0b4380 con 0x7f21cc15dec0 2024-09-21T05:42:19.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.546+0000 7f21c17fa640 1 -- 172.21.15.22:0/3492121696 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) ==== 181+0+0 (secure 0 0 0) 0x7f21cc0b4380 con 0x7f21cc15dec0 2024-09-21T05:42:19.549 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b8078b90 msgr2=0x7f21b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b8078b90 0x7f21b807b0a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21c8034000 tx=0x7f21c80019f0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 msgr2=0x7f21cc14f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 0x7f21cc14f080 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f21bc010aa0 tx=0x7f21bc010f60 comp rx=0 tx=0).stop 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 shutdown_connections 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc1f3540 0x7f21cc1f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc15dec0 0x7f21cc14f080 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b8078b90 0x7f21b807b0a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 --2- 172.21.15.22:0/3492121696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21b4003a30 0x7f21cc1e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.550+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 >> 172.21.15.22:0/3492121696 conn(0x7f21cc0b1c40 msgr2=0x7f21cc0b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.551+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 shutdown_connections 2024-09-21T05:42:19.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.551+0000 7f21d2c8b640 1 -- 172.21.15.22:0/3492121696 wait complete. 2024-09-21T05:42:19.567 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:42:19.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 -- 172.21.15.22:0/3432638458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88100b5f60 msgr2=0x7f88100b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:19.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3432638458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88100b5f60 0x7f88100b6370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f88100b70d0 tx=0x7f880802a980 comp rx=0 tx=0).stop 2024-09-21T05:42:19.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 -- 172.21.15.22:0/3432638458 shutdown_connections 2024-09-21T05:42:19.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3432638458 >> v2:172.21.15.134:3300/0 conn(0x7f88100d0e20 0x7f88100d1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3432638458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88100b5f60 0x7f88100b6370 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3432638458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f881006e0f0 0x7f881006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 -- 172.21.15.22:0/3432638458 >> 172.21.15.22:0/3432638458 conn(0x7f881009d800 msgr2=0x7f881008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:19.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 -- 172.21.15.22:0/3432638458 shutdown_connections 2024-09-21T05:42:19.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.693+0000 7f8816ab4640 1 -- 172.21.15.22:0/3432638458 wait complete. 2024-09-21T05:42:19.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.695+0000 7f8816ab4640 1 Processor -- start 2024-09-21T05:42:19.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.695+0000 7f8816ab4640 1 -- start start 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8816ab4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 0x7f88100b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8816ab4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88100b5f60 0x7f88101f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8816ab4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100d0e20 0x7f88101f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8816ab4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f88100b6c40 con 0x7f881006e0f0 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8816ab4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88100b6ac0 con 0x7f88100d0e20 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8816ab4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f88100b6dc0 con 0x7f88100b5f60 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8814850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 0x7f88100b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8814850640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 0x7f88100b7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33740/0 (socket says 172.21.15.22:33740) 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.696+0000 7f8814850640 1 -- 172.21.15.22:0/3655172217 learned_addr learned my addr 172.21.15.22:0/3655172217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8815051640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100d0e20 0x7f88101f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f880ffff640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88100b5f60 0x7f88101f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8814850640 1 -- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88100b5f60 msgr2=0x7f88101f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8814850640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88100b5f60 0x7f88101f8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8814850640 1 -- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100d0e20 msgr2=0x7f88101f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8814850640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100d0e20 0x7f88101f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8814850640 1 -- 172.21.15.22:0/3655172217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88101f6960 con 0x7f881006e0f0 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8815051640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100d0e20 0x7f88101f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8814850640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 0x7f88100b7d20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f880000c970 tx=0x7f880000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800004230 con 0x7f881006e0f0 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88000044c0 con 0x7f881006e0f0 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88100b85e0 con 0x7f881006e0f0 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800007bf0 con 0x7f881006e0f0 2024-09-21T05:42:19.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.697+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88101f6bd0 con 0x7f881006e0f0 2024-09-21T05:42:19.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.698+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88100b6370 con 0x7f881006e0f0 2024-09-21T05:42:19.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.699+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f88000035f0 con 0x7f881006e0f0 2024-09-21T05:42:19.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.700+0000 7f880d7fa640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87f0078b90 0x7f87f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:19.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.700+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5357+0+0 (secure 0 0 0) 0x7f880009b200 con 0x7f881006e0f0 2024-09-21T05:42:19.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.700+0000 7f880ffff640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87f0078b90 0x7f87f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:19.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.701+0000 7f880ffff640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87f0078b90 0x7f87f007b0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87f80058c0 tx=0x7f87f8005850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:19.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.704+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8800014070 con 0x7f881006e0f0 2024-09-21T05:42:19.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:19.896+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8810069110 con 0x7f881006e0f0 2024-09-21T05:42:20.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.551+0000 7f880d7fa640 1 -- 172.21.15.22:0/3655172217 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) ==== 169+0+0 (secure 0 0 0) 0x7f8800063740 con 0x7f881006e0f0 2024-09-21T05:42:20.554 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:42:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87f0078b90 msgr2=0x7f87f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87f0078b90 0x7f87f007b0a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87f80058c0 tx=0x7f87f8005850 comp rx=0 tx=0).stop 2024-09-21T05:42:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 msgr2=0x7f88100b7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 0x7f88100b7d20 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f880000c970 tx=0x7f880000ce30 comp rx=0 tx=0).stop 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 shutdown_connections 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88100d0e20 0x7f88101f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f87f0078b90 0x7f87f007b0a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f88100b5f60 0x7f88101f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 --2- 172.21.15.22:0/3655172217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f881006e0f0 0x7f88100b7d20 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 >> 172.21.15.22:0/3655172217 conn(0x7f881009d800 msgr2=0x7f88101e6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 shutdown_connections 2024-09-21T05:42:20.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.556+0000 7f8816ab4640 1 -- 172.21.15.22:0/3655172217 wait complete. 2024-09-21T05:42:20.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.696+0000 7f17ccf72640 1 -- 172.21.15.22:0/3850759357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b00053f0 msgr2=0x7f17b000ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.696+0000 7f17ccf72640 1 --2- 172.21.15.22:0/3850759357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b00053f0 0x7f17b000ba70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17c0001d90 tx=0x7f17b0005830 comp rx=0 tx=0).stop 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.696+0000 7f17ccf72640 1 -- 172.21.15.22:0/3850759357 shutdown_connections 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.696+0000 7f17ccf72640 1 --2- 172.21.15.22:0/3850759357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17b00053f0 0x7f17b000ba70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.696+0000 7f17ccf72640 1 -- 172.21.15.22:0/3850759357 >> 172.21.15.22:0/3850759357 conn(0x7f17c809d7f0 msgr2=0x7f17c80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.696+0000 7f17ccf72640 1 -- 172.21.15.22:0/3850759357 shutdown_connections 2024-09-21T05:42:20.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.697+0000 7f17ccf72640 1 -- 172.21.15.22:0/3850759357 wait complete. 2024-09-21T05:42:20.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.698+0000 7f17ccf72640 1 Processor -- start 2024-09-21T05:42:20.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.699+0000 7f17ccf72640 1 -- start start 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17ccf72640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c815e8e0 0x7f17c80cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17ccf72640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17c8156690 0x7f17c80c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17ccf72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 0x7f17c80c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17ccf72640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17c80b6120 con 0x7f17c80c8590 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17ccf72640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17c80b5fa0 con 0x7f17c815e8e0 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17ccf72640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17c80b62a0 con 0x7f17c8156690 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 0x7f17c80c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 0x7f17c80c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33754/0 (socket says 172.21.15.22:33754) 2024-09-21T05:42:20.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 -- 172.21.15.22:0/1233214209 learned_addr learned my addr 172.21.15.22:0/1233214209 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c659c640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c815e8e0 0x7f17c80cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c5d9b640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17c8156690 0x7f17c80c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 -- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17c8156690 msgr2=0x7f17c80c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17c8156690 0x7f17c80c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 -- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c815e8e0 msgr2=0x7f17c80cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c815e8e0 0x7f17c80cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.700+0000 7f17c6d9d640 1 -- 172.21.15.22:0/1233214209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c80c89a0 con 0x7f17c80c8590 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.701+0000 7f17c6d9d640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 0x7f17c80c9ee0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17c0001eb0 tx=0x7f17c0006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.701+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0003d80 con 0x7f17c80c8590 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.701+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17c0005b00 con 0x7f17c80c8590 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.701+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c815ee30 con 0x7f17c80c8590 2024-09-21T05:42:20.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.701+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0005ef0 con 0x7f17c80c8590 2024-09-21T05:42:20.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.701+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c815f3f0 con 0x7f17c80c8590 2024-09-21T05:42:20.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.702+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17c80b6800 con 0x7f17c80c8590 2024-09-21T05:42:20.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.703+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17c0006090 con 0x7f17c80c8590 2024-09-21T05:42:20.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.703+0000 7f17b6ffd640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f17a8078b90 0x7f17a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:20.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.703+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f17c00ba820 con 0x7f17c80c8590 2024-09-21T05:42:20.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.703+0000 7f17c659c640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f17a8078b90 0x7f17a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:20.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.704+0000 7f17c659c640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f17a8078b90 0x7f17a807b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17b80019d0 tx=0x7f17b80066f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:20.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.708+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17c80c89a0 con 0x7f17c80c8590 2024-09-21T05:42:20.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.895+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f17c80d3fb0 con 0x7f17c80c8590 2024-09-21T05:42:20.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.899+0000 7f17b6ffd640 1 -- 172.21.15.22:0/1233214209 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f17c0082ff0 con 0x7f17c80c8590 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f17a8078b90 msgr2=0x7f17a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f17a8078b90 0x7f17a807b0a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f17b80019d0 tx=0x7f17b80066f0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 msgr2=0x7f17c80c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 0x7f17c80c9ee0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17c0001eb0 tx=0x7f17c0006ca0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 shutdown_connections 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17c80c8590 0x7f17c80c9ee0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f17a8078b90 0x7f17a807b0a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17c8156690 0x7f17c80c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 --2- 172.21.15.22:0/1233214209 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17c815e8e0 0x7f17c80cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 >> 172.21.15.22:0/1233214209 conn(0x7f17c809d7f0 msgr2=0x7f17c80b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:20.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 shutdown_connections 2024-09-21T05:42:20.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:20.901+0000 7f17ccf72640 1 -- 172.21.15.22:0/1233214209 wait complete. 2024-09-21T05:42:20.915 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 -- 172.21.15.22:0/2844161428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 msgr2=0x7f081c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 --2- 172.21.15.22:0/2844161428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f081c003ec0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f082400c470 tx=0x7f081c005830 comp rx=0 tx=0).stop 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 -- 172.21.15.22:0/2844161428 shutdown_connections 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 --2- 172.21.15.22:0/2844161428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f081c003ec0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 -- 172.21.15.22:0/2844161428 >> 172.21.15.22:0/2844161428 conn(0x7f08340cdcb0 msgr2=0x7f08340ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 -- 172.21.15.22:0/2844161428 shutdown_connections 2024-09-21T05:42:21.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.029+0000 7f0839c12640 1 -- 172.21.15.22:0/2844161428 wait complete. 2024-09-21T05:42:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.031+0000 7f0839c12640 1 Processor -- start 2024-09-21T05:42:21.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.031+0000 7f0839c12640 1 -- start start 2024-09-21T05:42:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f08341f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f083406cde0 0x7f083406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08340c9670 0x7f08340c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08340b3450 con 0x7f081c003a50 2024-09-21T05:42:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08340b32d0 con 0x7f083406cde0 2024-09-21T05:42:21.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0834071650 con 0x7f08340c9670 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f08341f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f08341f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33766/0 (socket says 172.21.15.22:33766) 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 -- 172.21.15.22:0/484331516 learned_addr learned my addr 172.21.15.22:0/484331516 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0832ffd640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f083406cde0 0x7f083406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0833fff640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08340c9670 0x7f08340c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 -- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08340c9670 msgr2=0x7f08340c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08340c9670 0x7f08340c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 -- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f083406cde0 msgr2=0x7f083406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f083406cde0 0x7f083406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 -- 172.21.15.22:0/484331516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083406d730 con 0x7f081c003a50 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0832ffd640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f083406cde0 0x7f083406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0833fff640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08340c9670 0x7f08340c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:21.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f08337fe640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f08341f61a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f082402b2a0 tx=0x7f08240013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824003610 con 0x7f081c003a50 2024-09-21T05:42:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08240038a0 con 0x7f081c003a50 2024-09-21T05:42:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08341524f0 con 0x7f081c003a50 2024-09-21T05:42:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.033+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824003cd0 con 0x7f081c003a50 2024-09-21T05:42:21.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.034+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0834152ab0 con 0x7f081c003a50 2024-09-21T05:42:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.035+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08340b0ee0 con 0x7f081c003a50 2024-09-21T05:42:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.035+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0824046020 con 0x7f081c003a50 2024-09-21T05:42:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.036+0000 7f081bfff640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078b90 0x7f082007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.036+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f08240ba880 con 0x7f081c003a50 2024-09-21T05:42:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.036+0000 7f0832ffd640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078b90 0x7f082007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.037+0000 7f0832ffd640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078b90 0x7f082007b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08340ca380 tx=0x7f0828006b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:21.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.041+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f083406d730 con 0x7f081c003a50 2024-09-21T05:42:21.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.301+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f08340d1c00 con 0x7f081c003a50 2024-09-21T05:42:21.511 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:42:21.514 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:42:21.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.553+0000 7f081bfff640 1 -- 172.21.15.22:0/484331516 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v163) ==== 169+0+0 (secure 0 0 0) 0x7f08340b3600 con 0x7f081c003a50 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078b90 msgr2=0x7f082007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078b90 0x7f082007b0a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08340ca380 tx=0x7f0828006b00 comp rx=0 tx=0).stop 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 msgr2=0x7f08341f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f08341f61a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f082402b2a0 tx=0x7f08240013c0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 shutdown_connections 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0820078b90 0x7f082007b0a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08340c9670 0x7f08340c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f083406cde0 0x7f083406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 --2- 172.21.15.22:0/484331516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f081c003a50 0x7f08341f61a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 >> 172.21.15.22:0/484331516 conn(0x7f08340cdcb0 msgr2=0x7f08340cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:21.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 shutdown_connections 2024-09-21T05:42:21.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.556+0000 7f0839c12640 1 -- 172.21.15.22:0/484331516 wait complete. 2024-09-21T05:42:21.570 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-21T05:42:21.570 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-21T05:42:21.570 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:42:21.570 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:42:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 -- 172.21.15.22:0/3185143278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b4570 msgr2=0x7f248c162a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 --2- 172.21.15.22:0/3185143278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b4570 0x7f248c162a10 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f248c06de30 tx=0x7f248402a560 comp rx=0 tx=0).stop 2024-09-21T05:42:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 -- 172.21.15.22:0/3185143278 shutdown_connections 2024-09-21T05:42:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 --2- 172.21.15.22:0/3185143278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b4570 0x7f248c162a10 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 --2- 172.21.15.22:0/3185143278 >> v2:172.21.15.134:3300/0 conn(0x7f248c06d4f0 0x7f248c1624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 -- 172.21.15.22:0/3185143278 >> 172.21.15.22:0/3185143278 conn(0x7f248c0cdcc0 msgr2=0x7f248c0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 -- 172.21.15.22:0/3185143278 shutdown_connections 2024-09-21T05:42:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.725+0000 7f2493e94640 1 -- 172.21.15.22:0/3185143278 wait complete. 2024-09-21T05:42:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.727+0000 7f2493e94640 1 Processor -- start 2024-09-21T05:42:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.727+0000 7f2493e94640 1 -- start start 2024-09-21T05:42:21.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2493e94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f248c0b4570 0x7f248c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2493e94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 0x7f248c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2493e94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f248c1e9870 0x7f248c1e9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2493e94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0b4280 con 0x7f248c0b1e70 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2493e94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f248c0b4100 con 0x7f248c0b4570 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2493e94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f248c162f50 con 0x7f248c1e9870 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f249142f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 0x7f248c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f249142f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 0x7f248c0b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33784/0 (socket says 172.21.15.22:33784) 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f249142f640 1 -- 172.21.15.22:0/3170661810 learned_addr learned my addr 172.21.15.22:0/3170661810 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2492431640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f248c1e9870 0x7f248c1e9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.728+0000 7f2491c30640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f248c0b4570 0x7f248c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f249142f640 1 -- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f248c1e9870 msgr2=0x7f248c1e9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f249142f640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f248c1e9870 0x7f248c1e9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f249142f640 1 -- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f248c0b4570 msgr2=0x7f248c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f249142f640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f248c0b4570 0x7f248c159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f249142f640 1 -- 172.21.15.22:0/3170661810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c1ea1c0 con 0x7f248c0b1e70 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f249142f640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 0x7f248c0b2280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f247c00d700 tx=0x7f247c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c017070 con 0x7f248c0b1e70 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f247c005310 con 0x7f248c0b1e70 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c0b2ac0 con 0x7f248c0b1e70 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c0097c0 con 0x7f248c0b1e70 2024-09-21T05:42:21.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.729+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c0af6d0 con 0x7f248c0b1e70 2024-09-21T05:42:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.730+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c0b2830 con 0x7f248c0b1e70 2024-09-21T05:42:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.731+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f247c009960 con 0x7f248c0b1e70 2024-09-21T05:42:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.731+0000 7f24827fc640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2470078b90 0x7f247007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.731+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f247c09aaf0 con 0x7f248c0b1e70 2024-09-21T05:42:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.732+0000 7f2491c30640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2470078b90 0x7f247007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.732+0000 7f2491c30640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2470078b90 0x7f247007b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f248800b450 tx=0x7f248800d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:21.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.736+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f248c1ea1c0 con 0x7f248c0b1e70 2024-09-21T05:42:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.930+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f248c162040 con 0x7f248c0b1e70 2024-09-21T05:42:21.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.931+0000 7f24827fc640 1 -- 172.21.15.22:0/3170661810 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+13018 (secure 0 0 0) 0x7f247c062b40 con 0x7f248c0b1e70 2024-09-21T05:42:21.933 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:21.933 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":60,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:42:20.547986+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6835/913296056":"2024-09-22T05:42:17.537387+0000","172.21.15.22:6833/913296056":"2024-09-22T05:42:17.537387+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:42:21.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2470078b90 msgr2=0x7f247007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2470078b90 0x7f247007b0a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f248800b450 tx=0x7f248800d040 comp rx=0 tx=0).stop 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 msgr2=0x7f248c0b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 0x7f248c0b2280 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f247c00d700 tx=0x7f247c00dbc0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 shutdown_connections 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2470078b90 0x7f247007b0a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f248c1e9870 0x7f248c1e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f248c0b1e70 0x7f248c0b2280 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 --2- 172.21.15.22:0/3170661810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f248c0b4570 0x7f248c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 >> 172.21.15.22:0/3170661810 conn(0x7f248c0cdcc0 msgr2=0x7f248c0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 shutdown_connections 2024-09-21T05:42:21.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:21.933+0000 7f2493e94640 1 -- 172.21.15.22:0/3170661810 wait complete. 2024-09-21T05:42:21.946 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:42:21.947 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 -- 172.21.15.22:0/1714980519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0d14f0 msgr2=0x7f54ac162b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 --2- 172.21.15.22:0/1714980519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0d14f0 0x7f54ac162b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005fd0 tx=0x7f54a402aa10 comp rx=0 tx=0).stop 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 -- 172.21.15.22:0/1714980519 shutdown_connections 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 --2- 172.21.15.22:0/1714980519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0d14f0 0x7f54ac162b40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 --2- 172.21.15.22:0/1714980519 >> v2:172.21.15.134:3300/0 conn(0x7f54ac0b9b60 0x7f54ac0d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 -- 172.21.15.22:0/1714980519 >> 172.21.15.22:0/1714980519 conn(0x7f54ac09d8f0 msgr2=0x7f54ac0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 -- 172.21.15.22:0/1714980519 shutdown_connections 2024-09-21T05:42:22.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.064+0000 7f54b3e15640 1 -- 172.21.15.22:0/1714980519 wait complete. 2024-09-21T05:42:22.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.066+0000 7f54b3e15640 1 Processor -- start 2024-09-21T05:42:22.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.066+0000 7f54b3e15640 1 -- start start 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.067+0000 7f54b3e15640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0b3a10 0x7f54ac1f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.067+0000 7f54b3e15640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 0x7f54ac1f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.067+0000 7f54b3e15640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ac0d14f0 0x7f54ac1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.067+0000 7f54b3e15640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac163080 con 0x7f54ac0b9b60 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.067+0000 7f54b3e15640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac0d1fa0 con 0x7f54ac0d14f0 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.067+0000 7f54b3e15640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54ac163200 con 0x7f54ac0b3a10 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 0x7f54ac1f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 0x7f54ac1f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33796/0 (socket says 172.21.15.22:33796) 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 -- 172.21.15.22:0/3949595444 learned_addr learned my addr 172.21.15.22:0/3949595444 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b1bb1640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0b3a10 0x7f54ac1f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b23b2640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ac0d14f0 0x7f54ac1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 -- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0b3a10 msgr2=0x7f54ac1f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0b3a10 0x7f54ac1f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 -- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ac0d14f0 msgr2=0x7f54ac1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ac0d14f0 0x7f54ac1563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 -- 172.21.15.22:0/3949595444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac1f96e0 con 0x7f54ac0b9b60 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b23b2640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ac0d14f0 0x7f54ac1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b13b0640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 0x7f54ac1f65c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c3a0 tx=0x7f549c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c017070 con 0x7f54ac0b9b60 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f549c0053c0 con 0x7f54ac0b9b60 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003850 con 0x7f54ac0b9b60 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac1f6b00 con 0x7f54ac0b9b60 2024-09-21T05:42:22.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.068+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac1f9bb0 con 0x7f54ac0b9b60 2024-09-21T05:42:22.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.069+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ac1f9920 con 0x7f54ac0b9b60 2024-09-21T05:42:22.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.070+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f549c01a070 con 0x7f54ac0b9b60 2024-09-21T05:42:22.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.071+0000 7f54a27fc640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5490078b90 0x7f549007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.071+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f549c099d40 con 0x7f54ac0b9b60 2024-09-21T05:42:22.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.071+0000 7f54b1bb1640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5490078b90 0x7f549007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.072+0000 7f54b1bb1640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5490078b90 0x7f549007b0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54a80095f0 tx=0x7f54a8005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:22.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.075+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54ac0d1900 con 0x7f54ac0b9b60 2024-09-21T05:42:22.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.261+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54ac0ba0d0 con 0x7f54ac0b9b60 2024-09-21T05:42:22.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.262+0000 7f54a27fc640 1 -- 172.21.15.22:0/3949595444 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) ==== 93+0+10 (secure 0 0 0) 0x7f549c061ec0 con 0x7f54ac0b9b60 2024-09-21T05:42:22.264 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5490078b90 msgr2=0x7f549007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5490078b90 0x7f549007b0a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f54a80095f0 tx=0x7f54a8005f90 comp rx=0 tx=0).stop 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 msgr2=0x7f54ac1f65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 0x7f54ac1f65c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c3a0 tx=0x7f549c00c860 comp rx=0 tx=0).stop 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 shutdown_connections 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54ac0d14f0 0x7f54ac1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ac0b9b60 0x7f54ac1f65c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5490078b90 0x7f549007b0a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 --2- 172.21.15.22:0/3949595444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ac0b3a10 0x7f54ac1f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 >> 172.21.15.22:0/3949595444 conn(0x7f54ac09d8f0 msgr2=0x7f54ac06e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 shutdown_connections 2024-09-21T05:42:22.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.264+0000 7f54b3e15640 1 -- 172.21.15.22:0/3949595444 wait complete. 2024-09-21T05:42:22.277 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.392+0000 7f919b4f9640 1 -- 172.21.15.22:0/1313221754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91940b66b0 msgr2=0x7f919406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.392+0000 7f919b4f9640 1 --2- 172.21.15.22:0/1313221754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91940b66b0 0x7f919406eab0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f918c005fd0 tx=0x7f918c02a8e0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.392+0000 7f919b4f9640 1 -- 172.21.15.22:0/1313221754 shutdown_connections 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.393+0000 7f919b4f9640 1 --2- 172.21.15.22:0/1313221754 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91940b66b0 0x7f919406eab0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.393+0000 7f919b4f9640 1 --2- 172.21.15.22:0/1313221754 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940b9b60 0x7f91940d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.393+0000 7f919b4f9640 1 -- 172.21.15.22:0/1313221754 >> 172.21.15.22:0/1313221754 conn(0x7f919409d7f0 msgr2=0x7f91940d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.393+0000 7f919b4f9640 1 -- 172.21.15.22:0/1313221754 shutdown_connections 2024-09-21T05:42:22.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.393+0000 7f919b4f9640 1 -- 172.21.15.22:0/1313221754 wait complete. 2024-09-21T05:42:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.394+0000 7f919b4f9640 1 Processor -- start 2024-09-21T05:42:22.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.394+0000 7f919b4f9640 1 -- start start 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f919b4f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91940b9b60 0x7f91941f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f919b4f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940cf540 0x7f919414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f919b4f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 0x7f919415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f919b4f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f919406f5d0 con 0x7f91941f8b10 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f919b4f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f919406f450 con 0x7f91940b9b60 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f919b4f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f919406f750 con 0x7f91940cf540 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 0x7f919415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 0x7f919415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33802/0 (socket says 172.21.15.22:33802) 2024-09-21T05:42:22.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 -- 172.21.15.22:0/2074946282 learned_addr learned my addr 172.21.15.22:0/2074946282 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199295640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91940b9b60 0x7f91941f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9198a94640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940cf540 0x7f919414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 -- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940cf540 msgr2=0x7f919414e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940cf540 0x7f919414e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 -- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91940b9b60 msgr2=0x7f91941f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91940b9b60 0x7f91941f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 -- 172.21.15.22:0/2074946282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91940cab90 con 0x7f91941f8b10 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199295640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91940b9b60 0x7f91941f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9198a94640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940cf540 0x7f919414e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.396+0000 7f9199a96640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 0x7f919415f5d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f918c009d90 tx=0x7f918c030ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.397+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c004040 con 0x7f91941f8b10 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.397+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f918c004200 con 0x7f91941f8b10 2024-09-21T05:42:22.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.397+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c004650 con 0x7f91941f8b10 2024-09-21T05:42:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.397+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91941f9720 con 0x7f91941f8b10 2024-09-21T05:42:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.397+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91941f5630 con 0x7f91941f8b10 2024-09-21T05:42:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.398+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91941f9490 con 0x7f91941f8b10 2024-09-21T05:42:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.399+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f918c031470 con 0x7f91941f8b10 2024-09-21T05:42:22.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.399+0000 7f9189ffb640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9178078bc0 0x7f917807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.399+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f918c0bc440 con 0x7f91941f8b10 2024-09-21T05:42:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.399+0000 7f9199295640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9178078bc0 0x7f917807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:22.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.400+0000 7f9199295640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9178078bc0 0x7f917807b0d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9190005860 tx=0x7f9190005e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:22.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.404+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91940cab90 con 0x7f91941f8b10 2024-09-21T05:42:22.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:22.666+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f91940b9650 con 0x7f91941f8b10 2024-09-21T05:42:23.563 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.561+0000 7f9189ffb640 1 -- 172.21.15.22:0/2074946282 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v165) ==== 169+0+0 (secure 0 0 0) 0x7f91940b9650 con 0x7f91941f8b10 2024-09-21T05:42:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9178078bc0 msgr2=0x7f917807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9178078bc0 0x7f917807b0d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9190005860 tx=0x7f9190005e10 comp rx=0 tx=0).stop 2024-09-21T05:42:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 msgr2=0x7f919415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 0x7f919415f5d0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f918c009d90 tx=0x7f918c030ef0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 shutdown_connections 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91941f8b10 0x7f919415f5d0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9178078bc0 0x7f917807b0d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91940cf540 0x7f919414e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 --2- 172.21.15.22:0/2074946282 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91940b9b60 0x7f91941f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 >> 172.21.15.22:0/2074946282 conn(0x7f919409d7f0 msgr2=0x7f91940b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 shutdown_connections 2024-09-21T05:42:23.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.564+0000 7f919b4f9640 1 -- 172.21.15.22:0/2074946282 wait complete. 2024-09-21T05:42:23.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:42:23.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 -- 172.21.15.22:0/3411825311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 msgr2=0x7f64b40d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 --2- 172.21.15.22:0/3411825311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b40d25b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f64b408ccf0 tx=0x7f64ac02a8f0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 -- 172.21.15.22:0/3411825311 shutdown_connections 2024-09-21T05:42:23.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 --2- 172.21.15.22:0/3411825311 >> v2:172.21.15.134:3300/0 conn(0x7f64b40cf540 0x7f64b40b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 --2- 172.21.15.22:0/3411825311 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b40d25b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 --2- 172.21.15.22:0/3411825311 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64b40b9b60 0x7f64b40d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 -- 172.21.15.22:0/3411825311 >> 172.21.15.22:0/3411825311 conn(0x7f64b409d7f0 msgr2=0x7f64b40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:23.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 -- 172.21.15.22:0/3411825311 shutdown_connections 2024-09-21T05:42:23.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.702+0000 7f64b8d50640 1 -- 172.21.15.22:0/3411825311 wait complete. 2024-09-21T05:42:23.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.704+0000 7f64b8d50640 1 Processor -- start 2024-09-21T05:42:23.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.704+0000 7f64b8d50640 1 -- start start 2024-09-21T05:42:23.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b8d50640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b41f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:23.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b8d50640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64b40b9b60 0x7f64b40a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:23.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b8d50640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b40cf540 0x7f64b40a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b8d50640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40d2f90 con 0x7f64b40b64e0 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b8d50640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f64b40d2e10 con 0x7f64b40cf540 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b8d50640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f64b40d3110 con 0x7f64b40b9b60 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b41f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b259c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b41f8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33832/0 (socket says 172.21.15.22:33832) 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b259c640 1 -- 172.21.15.22:0/2270415521 learned_addr learned my addr 172.21.15.22:0/2270415521 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b1d9b640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64b40b9b60 0x7f64b40a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.705+0000 7f64b2d9d640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b40cf540 0x7f64b40a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b259c640 1 -- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64b40b9b60 msgr2=0x7f64b40a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b259c640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64b40b9b60 0x7f64b40a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b259c640 1 -- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b40cf540 msgr2=0x7f64b40a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b259c640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b40cf540 0x7f64b40a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b259c640 1 -- 172.21.15.22:0/2270415521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b41f5720 con 0x7f64b40b64e0 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b259c640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b41f8d40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d6a0 tx=0x7f64a400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:23.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4017070 con 0x7f64b40b64e0 2024-09-21T05:42:23.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64a4004310 con 0x7f64b40b64e0 2024-09-21T05:42:23.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b41f9500 con 0x7f64b40b64e0 2024-09-21T05:42:23.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4009a80 con 0x7f64b40b64e0 2024-09-21T05:42:23.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.706+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b41f9ac0 con 0x7f64b40b64e0 2024-09-21T05:42:23.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.707+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64b40d4d80 con 0x7f64b40b64e0 2024-09-21T05:42:23.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.708+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64a4009c20 con 0x7f64b40b64e0 2024-09-21T05:42:23.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.708+0000 7f64a2ffd640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6494078b90 0x7f649407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:23.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.709+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f64a409aa90 con 0x7f64b40b64e0 2024-09-21T05:42:23.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.709+0000 7f64b1d9b640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6494078b90 0x7f649407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:23.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.709+0000 7f64b1d9b640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6494078b90 0x7f649407b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64a800bb50 tx=0x7f64a800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:23.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.713+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f64a4062b70 con 0x7f64b40b64e0 2024-09-21T05:42:23.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.900+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f64b40d3fb0 con 0x7f64b40b64e0 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.901+0000 7f64a2ffd640 1 -- 172.21.15.22:0/2270415521 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7f64a40675a0 con 0x7f64b40b64e0 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:42:23.903 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:42:23.904 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:42:23.905 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6494078b90 msgr2=0x7f649407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6494078b90 0x7f649407b0a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f64a800bb50 tx=0x7f64a800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 msgr2=0x7f64b41f8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b41f8d40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f64a400d6a0 tx=0x7f64a400db60 comp rx=0 tx=0).stop 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 shutdown_connections 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f64b40cf540 0x7f64b40a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6494078b90 0x7f649407b0a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f64b40b9b60 0x7f64b40a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 --2- 172.21.15.22:0/2270415521 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f64b40b64e0 0x7f64b41f8d40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 >> 172.21.15.22:0/2270415521 conn(0x7f64b409d7f0 msgr2=0x7f64b40b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 shutdown_connections 2024-09-21T05:42:23.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:23.903+0000 7f64b8d50640 1 -- 172.21.15.22:0/2270415521 wait complete. 2024-09-21T05:42:23.916 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:42:24.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 -- 172.21.15.22:0/2808425563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 msgr2=0x7fa1200b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 --2- 172.21.15.22:0/2808425563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200b5fa0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fa110005fd0 tx=0x7fa11002a580 comp rx=0 tx=0).stop 2024-09-21T05:42:24.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 -- 172.21.15.22:0/2808425563 shutdown_connections 2024-09-21T05:42:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 --2- 172.21.15.22:0/2808425563 >> v2:172.21.15.134:3300/0 conn(0x7fa1200b64e0 0x7fa1200ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 --2- 172.21.15.22:0/2808425563 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200b5fa0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 --2- 172.21.15.22:0/2808425563 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1200d4c30 0x7fa1200b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 -- 172.21.15.22:0/2808425563 >> 172.21.15.22:0/2808425563 conn(0x7fa12009d830 msgr2=0x7fa1200d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 -- 172.21.15.22:0/2808425563 shutdown_connections 2024-09-21T05:42:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.034+0000 7fa12824d640 1 -- 172.21.15.22:0/2808425563 wait complete. 2024-09-21T05:42:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.036+0000 7fa12824d640 1 Processor -- start 2024-09-21T05:42:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.036+0000 7fa12824d640 1 -- start start 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa12824d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa12824d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1200b64e0 0x7fa12014e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa12824d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1200d4c30 0x7fa1201f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa12824d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200cf700 con 0x7fa1200b3910 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa12824d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1200cf580 con 0x7fa1200b64e0 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa12824d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1200cf880 con 0x7fa1200d4c30 2024-09-21T05:42:24.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa125fe9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa125fe9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33854/0 (socket says 172.21.15.22:33854) 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa125fe9640 1 -- 172.21.15.22:0/3893190462 learned_addr learned my addr 172.21.15.22:0/3893190462 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa1257e8640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1200b64e0 0x7fa12014e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa1267ea640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1200d4c30 0x7fa1201f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa125fe9640 1 -- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1200d4c30 msgr2=0x7fa1201f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.037+0000 7fa125fe9640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1200d4c30 0x7fa1201f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa125fe9640 1 -- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1200b64e0 msgr2=0x7fa12014e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa125fe9640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1200b64e0 0x7fa12014e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa125fe9640 1 -- 172.21.15.22:0/3893190462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12009e4f0 con 0x7fa1200b3910 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa125fe9640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200cbd00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00ea20 tx=0x7fa11c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c00ccc0 con 0x7fa1200b3910 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa11c00ce60 con 0x7fa1200b3910 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1201602e0 con 0x7fa1200b3910 2024-09-21T05:42:24.041 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c010650 con 0x7fa1200b3910 2024-09-21T05:42:24.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.038+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12014e5b0 con 0x7fa1200b3910 2024-09-21T05:42:24.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.039+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1200d2270 con 0x7fa1200b3910 2024-09-21T05:42:24.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.040+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa11c0107f0 con 0x7fa1200b3910 2024-09-21T05:42:24.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.040+0000 7fa1167fc640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa104078b90 0x7fa10407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.040+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7fa11c09b440 con 0x7fa1200b3910 2024-09-21T05:42:24.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.041+0000 7fa1257e8640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa104078b90 0x7fa10407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.041+0000 7fa1257e8640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa104078b90 0x7fa10407b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa110033000 tx=0x7fa110004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:24.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.045+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1200d2270 con 0x7fa1200b3910 2024-09-21T05:42:24.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.234+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa1200ced80 con 0x7fa1200b3910 2024-09-21T05:42:24.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.238+0000 7fa1167fc640 1 -- 172.21.15.22:0/3893190462 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7fa11c0635c0 con 0x7fa1200b3910 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.240+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa104078b90 msgr2=0x7fa10407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.240+0000 7fa12824d640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa104078b90 0x7fa10407b0a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa110033000 tx=0x7fa110004730 comp rx=0 tx=0).stop 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 msgr2=0x7fa1200cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200cbd00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00ea20 tx=0x7fa11c00ee20 comp rx=0 tx=0).stop 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 shutdown_connections 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa104078b90 0x7fa10407b0a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1200d4c30 0x7fa1201f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1200b64e0 0x7fa12014e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 --2- 172.21.15.22:0/3893190462 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1200b3910 0x7fa1200cbd00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 >> 172.21.15.22:0/3893190462 conn(0x7fa12009d830 msgr2=0x7fa1200b9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:24.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 shutdown_connections 2024-09-21T05:42:24.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.241+0000 7fa12824d640 1 -- 172.21.15.22:0/3893190462 wait complete. 2024-09-21T05:42:24.254 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:42:24.254 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:42:24.297 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:42:24.340 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:42:24.340 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:24.340 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:24.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:42:24.356 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:42:24.356 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:24.399 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:42:24.399 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:24.400 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:24.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:42:24.415 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:42:24.416 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:24.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:42:24.457 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:42:24.458 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 -- 172.21.15.22:0/2779137312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc0cd3b0 msgr2=0x7f0ccc157130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/2779137312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc0cd3b0 0x7f0ccc157130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc0b1c00 tx=0x7f0cbc02a5f0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 -- 172.21.15.22:0/2779137312 shutdown_connections 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/2779137312 >> v2:172.21.15.134:3300/0 conn(0x7f0ccc0ced00 0x7f0ccc0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/2779137312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc0cd3b0 0x7f0ccc157130 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 -- 172.21.15.22:0/2779137312 >> 172.21.15.22:0/2779137312 conn(0x7f0ccc06d550 msgr2=0x7f0ccc071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 -- 172.21.15.22:0/2779137312 shutdown_connections 2024-09-21T05:42:24.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.572+0000 7f0cd15ad640 1 -- 172.21.15.22:0/2779137312 wait complete. 2024-09-21T05:42:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.574+0000 7f0cd15ad640 1 Processor -- start 2024-09-21T05:42:24.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.574+0000 7f0cd15ad640 1 -- start start 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.575+0000 7f0cd15ad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ccc0cd3b0 0x7f0ccc0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ccc0ced00 0x7f0ccc14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 0x7f0ccc14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc0d0660 con 0x7f0ccc160f60 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ccc1614b0 con 0x7f0ccc0ced00 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0ccc0d07e0 con 0x7f0ccc0cd3b0 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 0x7f0ccc14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 0x7f0ccc14c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58778/0 (socket says 172.21.15.22:58778) 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 -- 172.21.15.22:0/1665694468 learned_addr learned my addr 172.21.15.22:0/1665694468 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccaffd640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ccc0cd3b0 0x7f0ccc0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cc3fff640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ccc0ced00 0x7f0ccc14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 -- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ccc0cd3b0 msgr2=0x7f0ccc0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ccc0cd3b0 0x7f0ccc0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 -- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ccc0ced00 msgr2=0x7f0ccc14d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ccc0ced00 0x7f0ccc14d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 -- 172.21.15.22:0/1665694468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ccc0ca1c0 con 0x7f0ccc160f60 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0ccb7fe640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 0x7f0ccc14c800 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400eab0 tx=0x7f0cc400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400cea0 con 0x7f0ccc160f60 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cc4004b20 con 0x7f0ccc160f60 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ccc150040 con 0x7f0ccc160f60 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cc4005020 con 0x7f0ccc160f60 2024-09-21T05:42:24.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.576+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ccc0cb210 con 0x7f0ccc160f60 2024-09-21T05:42:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.577+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ccc0cad50 con 0x7f0ccc160f60 2024-09-21T05:42:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.578+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0cc40051c0 con 0x7f0ccc160f60 2024-09-21T05:42:24.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.579+0000 7f0cc37fe640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cac078bc0 0x7f0cac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.579+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f0cc409b5b0 con 0x7f0ccc160f60 2024-09-21T05:42:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.579+0000 7f0ccaffd640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cac078bc0 0x7f0cac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:24.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.580+0000 7f0ccaffd640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cac078bc0 0x7f0cac07b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc033000 tx=0x7f0cbc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:24.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.584+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ccc0cad50 con 0x7f0ccc160f60 2024-09-21T05:42:24.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.813+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ccc0d0d70 con 0x7f0ccc160f60 2024-09-21T05:42:24.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.814+0000 7f0cc37fe640 1 -- 172.21.15.22:0/1665694468 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3917 (secure 0 0 0) 0x7f0cc4063690 con 0x7f0ccc160f60 2024-09-21T05:42:24.816 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:24.816 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":38,"btime":"2024-09-21T05:42:18:539041+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5458,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2271083942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2271083942},{"type":"v1","addr":"172.21.15.134:6836","nonce":2271083942}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5461,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/1037946776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1037946776},{"type":"v1","addr":"172.21.15.134:6837","nonce":1037946776}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/1722231251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1722231251},{"type":"v1","addr":"172.21.15.22:6834","nonce":1722231251}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4273972334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4273972334},{"type":"v1","addr":"172.21.15.22:6835","nonce":4273972334}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-09-21T05:42:24.816 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 38 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cac078bc0 msgr2=0x7f0cac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cac078bc0 0x7f0cac07b0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc033000 tx=0x7f0cbc004730 comp rx=0 tx=0).stop 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 msgr2=0x7f0ccc14c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 0x7f0ccc14c800 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400eab0 tx=0x7f0cc400ef70 comp rx=0 tx=0).stop 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 shutdown_connections 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ccc160f60 0x7f0ccc14c800 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ccc0ced00 0x7f0ccc14d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0cac078bc0 0x7f0cac07b0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 --2- 172.21.15.22:0/1665694468 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ccc0cd3b0 0x7f0ccc0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 >> 172.21.15.22:0/1665694468 conn(0x7f0ccc06d550 msgr2=0x7f0ccc153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:24.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 shutdown_connections 2024-09-21T05:42:24.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:24.816+0000 7f0cd15ad640 1 -- 172.21.15.22:0/1665694468 wait complete. 2024-09-21T05:42:24.830 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:42:24.830 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:42:24.831 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:42:24.831 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:42:24.831 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:42:24.831 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:42:24.832 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:42:24.832 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:42:24.832 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:42:24.832 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:42:24.833 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:42:24.833 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:42:24.833 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:42:24.833 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:42:24.833 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:42:24.834 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:42:24.840 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:42:24.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:42:24.843 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:42:24.844 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:42:24.845 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:42:24.845 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:42:24.846 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:42:24.846 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:42:24.886 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:42:24.886 DEBUG:teuthology.parallel:result is None 2024-09-21T05:42:24.887 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:42:24.887 DEBUG:teuthology.parallel:result is None 2024-09-21T05:42:24.888 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:42:24.889 DEBUG:teuthology.parallel:result is None 2024-09-21T05:42:24.889 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:42:24.889 DEBUG:teuthology.parallel:result is None 2024-09-21T05:42:24.890 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:42:24.977 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:42:24.977 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:42:24.985 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:42:24.985 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:42:25.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 -- 172.21.15.22:0/3467082440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 msgr2=0x7f9468004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 --2- 172.21.15.22:0/3467082440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f9468004c10 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f9474009b30 tx=0x7f9468005830 comp rx=0 tx=0).stop 2024-09-21T05:42:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 -- 172.21.15.22:0/3467082440 shutdown_connections 2024-09-21T05:42:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 --2- 172.21.15.22:0/3467082440 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f9468004c10 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 -- 172.21.15.22:0/3467082440 >> 172.21.15.22:0/3467082440 conn(0x7f948008bfc0 msgr2=0x7f948009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 -- 172.21.15.22:0/3467082440 shutdown_connections 2024-09-21T05:42:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.012+0000 7f9485eec640 1 -- 172.21.15.22:0/3467082440 wait complete. 2024-09-21T05:42:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.014+0000 7f9485eec640 1 Processor -- start 2024-09-21T05:42:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.014+0000 7f9485eec640 1 -- start start 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9485eec640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f94801599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9485eec640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9480159f50 0x7f94801582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9485eec640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9480160890 0x7f9480160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9485eec640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f948006f920 con 0x7f9468004800 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9485eec640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f948006f7a0 con 0x7f9480160890 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9485eec640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f948006faa0 con 0x7f9480159f50 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f94801599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f94801599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58822/0 (socket says 172.21.15.22:58822) 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 -- 172.21.15.22:0/3905976676 learned_addr learned my addr 172.21.15.22:0/3905976676 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947ffff640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9480160890 0x7f9480160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947effd640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9480159f50 0x7f94801582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 -- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9480159f50 msgr2=0x7f94801582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9480159f50 0x7f94801582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 -- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9480160890 msgr2=0x7f9480160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9480160890 0x7f9480160ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 -- 172.21.15.22:0/3905976676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801611e0 con 0x7f9468004800 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f947f7fe640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f94801599c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9470003ea0 tx=0x7f9470003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470015070 con 0x7f9468004800 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9470011840 con 0x7f9468004800 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.016+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470011c70 con 0x7f9468004800 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.017+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9480158c00 con 0x7f9468004800 2024-09-21T05:42:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.017+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801591c0 con 0x7f9468004800 2024-09-21T05:42:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.017+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94800d1390 con 0x7f9468004800 2024-09-21T05:42:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.018+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9470018070 con 0x7f9468004800 2024-09-21T05:42:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.019+0000 7f9467fff640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f946c078bc0 0x7f946c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.019+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 4943+0+0 (secure 0 0 0) 0x7f947009b4a0 con 0x7f9468004800 2024-09-21T05:42:25.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.019+0000 7f947effd640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f946c078bc0 0x7f946c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.020+0000 7f947effd640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f946c078bc0 0x7f946c07b0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9474035000 tx=0x7f94740046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.023+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f94700635a0 con 0x7f9468004800 2024-09-21T05:42:25.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.223+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f948006cde0 con 0x7f9468004800 2024-09-21T05:42:25.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.580+0000 7f9467fff640 1 -- 172.21.15.22:0/3905976676 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) ==== 97+0+0 (secure 0 0 0) 0x7f9470067fd0 con 0x7f9468004800 2024-09-21T05:42:25.582 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f946c078bc0 msgr2=0x7f946c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f946c078bc0 0x7f946c07b0d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9474035000 tx=0x7f94740046c0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 msgr2=0x7f94801599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f94801599c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9470003ea0 tx=0x7f9470003f90 comp rx=0 tx=0).stop 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 shutdown_connections 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9480160890 0x7f9480160ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f946c078bc0 0x7f946c07b0d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9480159f50 0x7f94801582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 --2- 172.21.15.22:0/3905976676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9468004800 0x7f94801599c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 >> 172.21.15.22:0/3905976676 conn(0x7f948008bfc0 msgr2=0x7f94800b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 shutdown_connections 2024-09-21T05:42:25.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.583+0000 7f9485eec640 1 -- 172.21.15.22:0/3905976676 wait complete. 2024-09-21T05:42:25.599 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:42:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3750077070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00ceb40 msgr2=0x7ff0a00cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3750077070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00ceb40 0x7ff0a00cef50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff098005fd0 tx=0x7ff09802aa10 comp rx=0 tx=0).stop 2024-09-21T05:42:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3750077070 shutdown_connections 2024-09-21T05:42:25.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3750077070 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00ceb40 0x7ff0a00cef50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3750077070 >> v2:172.21.15.134:3300/0 conn(0x7ff0a006e170 0x7ff0a00b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3750077070 >> 172.21.15.22:0/3750077070 conn(0x7ff0a008bfc0 msgr2=0x7ff0a009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:25.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3750077070 shutdown_connections 2024-09-21T05:42:25.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.716+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3750077070 wait complete. 2024-09-21T05:42:25.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.718+0000 7ff0a66b6640 1 Processor -- start 2024-09-21T05:42:25.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.718+0000 7ff0a66b6640 1 -- start start 2024-09-21T05:42:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a66b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 0x7ff0a01f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a66b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a00ceb40 0x7ff0a00cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a66b6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00b62e0 0x7ff0a00cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a66b6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00d0e60 con 0x7ff0a006f4b0 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a66b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00d0ce0 con 0x7ff0a00ceb40 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a66b6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a00d0fe0 con 0x7ff0a00b62e0 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff09ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 0x7ff0a01f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff09ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 0x7ff0a01f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58828/0 (socket says 172.21.15.22:58828) 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff09ffff640 1 -- 172.21.15.22:0/3674634186 learned_addr learned my addr 172.21.15.22:0/3674634186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff09f7fe640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a00ceb40 0x7ff0a00cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.719+0000 7ff0a4c53640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00b62e0 0x7ff0a00cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09ffff640 1 -- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00b62e0 msgr2=0x7ff0a00cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09ffff640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00b62e0 0x7ff0a00cbb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09ffff640 1 -- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a00ceb40 msgr2=0x7ff0a00cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:25.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09ffff640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a00ceb40 0x7ff0a00cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09ffff640 1 -- 172.21.15.22:0/3674634186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a01f8880 con 0x7ff0a006f4b0 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09f7fe640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a00ceb40 0x7ff0a00cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09ffff640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 0x7ff0a01f4ad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e9f0 tx=0x7ff09000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09000cd80 con 0x7ff0a006f4b0 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff090004020 con 0x7ff0a006f4b0 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a01f8b60 con 0x7ff0a006f4b0 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090004470 con 0x7ff0a006f4b0 2024-09-21T05:42:25.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.720+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a01f9120 con 0x7ff0a006f4b0 2024-09-21T05:42:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.721+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a00d15a0 con 0x7ff0a006f4b0 2024-09-21T05:42:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.722+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff0900035f0 con 0x7ff0a006f4b0 2024-09-21T05:42:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.722+0000 7ff09cff9640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff080078b90 0x7ff08007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.723+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7ff09009b310 con 0x7ff0a006f4b0 2024-09-21T05:42:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.723+0000 7ff09f7fe640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff080078b90 0x7ff08007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:25.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.723+0000 7ff09f7fe640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff080078b90 0x7ff08007b0a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff088007fd0 tx=0x7ff088009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:25.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.727+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff090063330 con 0x7ff0a006f4b0 2024-09-21T05:42:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.992+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff0a00d4000 con 0x7ff0a006f4b0 2024-09-21T05:42:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.992+0000 7ff09cff9640 1 -- 172.21.15.22:0/3674634186 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7ff090067d60 con 0x7ff0a006f4b0 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.995 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:42:25.996 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:42:25.997 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:42:25.998 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:25.999 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:42:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:42:26.000 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.001 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:42:26.002 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.003 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.004 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow r", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.005 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:42:26.006 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.007 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:42:26.008 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.009 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:42:26.009 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff080078b90 msgr2=0x7ff08007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff080078b90 0x7ff08007b0a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff088007fd0 tx=0x7ff088009040 comp rx=0 tx=0).stop 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 msgr2=0x7ff0a01f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 0x7ff0a01f4ad0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e9f0 tx=0x7ff09000eeb0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 shutdown_connections 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff080078b90 0x7ff08007b0a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff0a00b62e0 0x7ff0a00cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff0a00ceb40 0x7ff0a00cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 --2- 172.21.15.22:0/3674634186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff0a006f4b0 0x7ff0a01f4ad0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 >> 172.21.15.22:0/3674634186 conn(0x7ff0a008bfc0 msgr2=0x7ff0a006eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:26.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 shutdown_connections 2024-09-21T05:42:26.011 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:25.995+0000 7ff0a66b6640 1 -- 172.21.15.22:0/3674634186 wait complete. 2024-09-21T05:42:26.011 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:42:26.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/1801931846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 msgr2=0x7fc1f40b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/1801931846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f40b5fa0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8005fd0 tx=0x7fc1e802a4b0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/1801931846 shutdown_connections 2024-09-21T05:42:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/1801931846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f40b5fa0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/1801931846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1f40b9b60 0x7fc1f40d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/1801931846 >> 172.21.15.22:0/1801931846 conn(0x7fc1f409d7f0 msgr2=0x7fc1f40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/1801931846 shutdown_connections 2024-09-21T05:42:26.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.126+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/1801931846 wait complete. 2024-09-21T05:42:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.128+0000 7fc1f9ef9640 1 Processor -- start 2024-09-21T05:42:26.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.128+0000 7fc1f9ef9640 1 -- start start 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.129+0000 7fc1f9ef9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1f40b64e0 0x7fc1f41508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.129+0000 7fc1f9ef9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f40b9b60 0x7fc1f41f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.129+0000 7fc1f9ef9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f41f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.129+0000 7fc1f9ef9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f40d2e40 con 0x7fc1f40cf540 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.129+0000 7fc1f9ef9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f40d2cc0 con 0x7fc1f40b9b60 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.129+0000 7fc1f9ef9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f40d2fc0 con 0x7fc1f40b64e0 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f41f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f41f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58848/0 (socket says 172.21.15.22:58848) 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 -- 172.21.15.22:0/3776698861 learned_addr learned my addr 172.21.15.22:0/3776698861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:26.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f37fe640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1f40b64e0 0x7fc1f41508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f2ffd640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f40b9b60 0x7fc1f41f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 -- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1f40b64e0 msgr2=0x7fc1f41508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1f40b64e0 0x7fc1f41508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 -- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f40b9b60 msgr2=0x7fc1f41f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f40b9b60 0x7fc1f41f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 -- 172.21.15.22:0/3776698861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f41f6590 con 0x7fc1f40cf540 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f2ffd640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f40b9b60 0x7fc1f41f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f3fff640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f41f5d30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00c760 tx=0x7fc1ec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec017070 con 0x7fc1f40cf540 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1ec00ce10 con 0x7fc1f40cf540 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f41f8ff0 con 0x7fc1f40cf540 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0052b0 con 0x7fc1f40cf540 2024-09-21T05:42:26.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.130+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f41f95b0 con 0x7fc1f40cf540 2024-09-21T05:42:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.132+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f41f6300 con 0x7fc1f40cf540 2024-09-21T05:42:26.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.132+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1ec01a070 con 0x7fc1f40cf540 2024-09-21T05:42:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.133+0000 7fc1dbfff640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1e0078bc0 0x7fc1e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.133+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7fc1ec099ee0 con 0x7fc1f40cf540 2024-09-21T05:42:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.133+0000 7fc1f37fe640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1e0078bc0 0x7fc1e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.134+0000 7fc1f37fe640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1e0078bc0 0x7fc1e007b0d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4005fd0 tx=0x7fc1e4005cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:26.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.137+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1f40b68f0 con 0x7fc1f40cf540 2024-09-21T05:42:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.326+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc1f40d3490 con 0x7fc1f40cf540 2024-09-21T05:42:26.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.326+0000 7fc1dbfff640 1 -- 172.21.15.22:0/3776698861 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) ==== 84+0+65 (secure 0 0 0) 0x7fc1ec062790 con 0x7fc1f40cf540 2024-09-21T05:42:26.328 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:26.328 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:42:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:42:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:42:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:42:26.329 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:26.329 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1e0078bc0 msgr2=0x7fc1e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1e0078bc0 0x7fc1e007b0d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4005fd0 tx=0x7fc1e4005cf0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 msgr2=0x7fc1f41f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f41f5d30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00c760 tx=0x7fc1ec00cc20 comp rx=0 tx=0).stop 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 shutdown_connections 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1f40cf540 0x7fc1f41f5d30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1f40b9b60 0x7fc1f41f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc1e0078bc0 0x7fc1e007b0d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 --2- 172.21.15.22:0/3776698861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1f40b64e0 0x7fc1f41508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.328+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 >> 172.21.15.22:0/3776698861 conn(0x7fc1f409d7f0 msgr2=0x7fc1f40b83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.329+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 shutdown_connections 2024-09-21T05:42:26.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.329+0000 7fc1f9ef9640 1 -- 172.21.15.22:0/3776698861 wait complete. 2024-09-21T05:42:26.342 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:42:26.342 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:42:26.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 -- 172.21.15.22:0/2663940257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500d0340 msgr2=0x7fa7500ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 --2- 172.21.15.22:0/2663940257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500d0340 0x7fa7500ccbc0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa7500d33f0 tx=0x7fa74002aa10 comp rx=0 tx=0).stop 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 -- 172.21.15.22:0/2663940257 shutdown_connections 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 --2- 172.21.15.22:0/2663940257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500d0340 0x7fa7500ccbc0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 --2- 172.21.15.22:0/2663940257 >> v2:172.21.15.134:3300/0 conn(0x7fa7501601c0 0x7fa7500d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 -- 172.21.15.22:0/2663940257 >> 172.21.15.22:0/2663940257 conn(0x7fa750072a90 msgr2=0x7fa750072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 -- 172.21.15.22:0/2663940257 shutdown_connections 2024-09-21T05:42:26.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.461+0000 7fa754c63640 1 -- 172.21.15.22:0/2663940257 wait complete. 2024-09-21T05:42:26.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.463+0000 7fa754c63640 1 Processor -- start 2024-09-21T05:42:26.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.463+0000 7fa754c63640 1 -- start start 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.464+0000 7fa754c63640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 0x7fa7500cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.464+0000 7fa754c63640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7500d0340 0x7fa7500c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.464+0000 7fa754c63640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7501601c0 0x7fa750157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.464+0000 7fa754c63640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa75015d460 con 0x7fa7500cff30 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.464+0000 7fa754c63640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa75015d2e0 con 0x7fa7500d0340 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.464+0000 7fa754c63640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa75015d5e0 con 0x7fa7501601c0 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 0x7fa7500cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 0x7fa7500cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58880/0 (socket says 172.21.15.22:58880) 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 -- 172.21.15.22:0/4148914045 learned_addr learned my addr 172.21.15.22:0/4148914045 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74dd9b640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7500d0340 0x7fa7500c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74ed9d640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7501601c0 0x7fa750157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 -- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7501601c0 msgr2=0x7fa750157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7501601c0 0x7fa750157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 -- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7500d0340 msgr2=0x7fa7500c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7500d0340 0x7fa7500c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 -- 172.21.15.22:0/4148914045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7500ca160 con 0x7fa7500cff30 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa74e59c640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 0x7fa7500cc0c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa74800c9a0 tx=0x7fa74800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748007bb0 con 0x7fa7500cff30 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa748007e40 con 0x7fa7500cff30 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7500b6dd0 con 0x7fa7500cff30 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74800f690 con 0x7fa7500cff30 2024-09-21T05:42:26.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.465+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7500c9420 con 0x7fa7500cff30 2024-09-21T05:42:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.466+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75015e270 con 0x7fa7500cff30 2024-09-21T05:42:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.467+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa74800f830 con 0x7fa7500cff30 2024-09-21T05:42:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.468+0000 7fa746ffd640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa730078b90 0x7fa73007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.468+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 4857+0+0 (secure 0 0 0) 0x7fa74809b0e0 con 0x7fa7500cff30 2024-09-21T05:42:26.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.468+0000 7fa74dd9b640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa730078b90 0x7fa73007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:26.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.469+0000 7fa74dd9b640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa730078b90 0x7fa73007b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7500b6850 tx=0x7fa738009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:26.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.472+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa748014030 con 0x7fa7500cff30 2024-09-21T05:42:26.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:26.660+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa75015ec90 con 0x7fa7500cff30 2024-09-21T05:42:27.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.600+0000 7fa746ffd640 1 -- 172.21.15.22:0/4148914045 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) ==== 138+0+0 (secure 0 0 0) 0x7fa748063220 con 0x7fa7500cff30 2024-09-21T05:42:27.602 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa730078b90 msgr2=0x7fa73007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa730078b90 0x7fa73007b0a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa7500b6850 tx=0x7fa738009060 comp rx=0 tx=0).stop 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 msgr2=0x7fa7500cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 0x7fa7500cc0c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa74800c9a0 tx=0x7fa74800ce60 comp rx=0 tx=0).stop 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 shutdown_connections 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa730078b90 0x7fa73007b0a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa7501601c0 0x7fa750157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa7500d0340 0x7fa7500c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 --2- 172.21.15.22:0/4148914045 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa7500cff30 0x7fa7500cc0c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 >> 172.21.15.22:0/4148914045 conn(0x7fa750072a90 msgr2=0x7fa75015f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 shutdown_connections 2024-09-21T05:42:27.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.603+0000 7fa754c63640 1 -- 172.21.15.22:0/4148914045 wait complete. 2024-09-21T05:42:27.622 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 -- 172.21.15.22:0/171344297 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f434406e3c0 msgr2=0x7f43440cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 --2- 172.21.15.22:0/171344297 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f434406e3c0 0x7f43440cfb70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f432c009830 tx=0x7f432c02a510 comp rx=0 tx=0).stop 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 -- 172.21.15.22:0/171344297 shutdown_connections 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 --2- 172.21.15.22:0/171344297 >> v2:172.21.15.134:3300/0 conn(0x7f43440d0e10 0x7f43440d1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 --2- 172.21.15.22:0/171344297 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f434406e3c0 0x7f43440cfb70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 -- 172.21.15.22:0/171344297 >> 172.21.15.22:0/171344297 conn(0x7f434408bfc0 msgr2=0x7f434409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:27.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 -- 172.21.15.22:0/171344297 shutdown_connections 2024-09-21T05:42:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.752+0000 7f434945c640 1 -- 172.21.15.22:0/171344297 wait complete. 2024-09-21T05:42:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.753+0000 7f434945c640 1 Processor -- start 2024-09-21T05:42:27.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.754+0000 7f434945c640 1 -- start start 2024-09-21T05:42:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f434945c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f434406e3c0 0x7f4344161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f434945c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43440b3a60 0x7f43440b6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f434945c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 0x7f43441f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f434945c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43440d2d20 con 0x7f43440d0e10 2024-09-21T05:42:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f434945c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43440d2ba0 con 0x7f434406e3c0 2024-09-21T05:42:27.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f434945c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43440d2ea0 con 0x7f43440b3a60 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 0x7f43441f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 0x7f43441f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58882/0 (socket says 172.21.15.22:58882) 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 -- 172.21.15.22:0/4215381240 learned_addr learned my addr 172.21.15.22:0/4215381240 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f4342ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f434406e3c0 0x7f4344161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43427fc640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43440b3a60 0x7f43440b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 -- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43440b3a60 msgr2=0x7f43440b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43440b3a60 0x7f43440b6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 -- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f434406e3c0 msgr2=0x7f4344161800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f434406e3c0 0x7f4344161800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.755+0000 7f43437fe640 1 -- 172.21.15.22:0/4215381240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441f6330 con 0x7f43440d0e10 2024-09-21T05:42:27.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.756+0000 7f43437fe640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 0x7f43441f5df0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f433400e970 tx=0x7f433400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.756+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400cd80 con 0x7f43440d0e10 2024-09-21T05:42:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.756+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4344160760 con 0x7f43440d0e10 2024-09-21T05:42:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.756+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4334004020 con 0x7f43440d0e10 2024-09-21T05:42:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.756+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334004470 con 0x7f43440d0e10 2024-09-21T05:42:27.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.756+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4344160d20 con 0x7f43440d0e10 2024-09-21T05:42:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.757+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43441604d0 con 0x7f43440d0e10 2024-09-21T05:42:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.758+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43340035f0 con 0x7f43440d0e10 2024-09-21T05:42:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.758+0000 7f433b7fe640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4324078bc0 0x7f432407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.758+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5227+0+0 (secure 0 0 0) 0x7f433409b750 con 0x7f43440d0e10 2024-09-21T05:42:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.759+0000 7f4342ffd640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4324078bc0 0x7f432407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:27.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.759+0000 7f4342ffd640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4324078bc0 0x7f432407b0d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f433c004a30 tx=0x7f433c001a70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:27.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.762+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4334063680 con 0x7f43440d0e10 2024-09-21T05:42:27.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:27.953+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f43440d3350 con 0x7f43440d0e10 2024-09-21T05:42:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.606+0000 7f433b7fe640 1 -- 172.21.15.22:0/4215381240 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) ==== 170+0+0 (secure 0 0 0) 0x7f43340680b0 con 0x7f43440d0e10 2024-09-21T05:42:28.608 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.609+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4324078bc0 msgr2=0x7f432407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.609+0000 7f434945c640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4324078bc0 0x7f432407b0d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f433c004a30 tx=0x7f433c001a70 comp rx=0 tx=0).stop 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.609+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 msgr2=0x7f43441f5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.609+0000 7f434945c640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 0x7f43441f5df0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f433400e970 tx=0x7f433400ee30 comp rx=0 tx=0).stop 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 shutdown_connections 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43440d0e10 0x7f43441f5df0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4324078bc0 0x7f432407b0d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43440b3a60 0x7f43440b6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 --2- 172.21.15.22:0/4215381240 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f434406e3c0 0x7f4344161800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 >> 172.21.15.22:0/4215381240 conn(0x7f434408bfc0 msgr2=0x7f43440b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 shutdown_connections 2024-09-21T05:42:28.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.610+0000 7f434945c640 1 -- 172.21.15.22:0/4215381240 wait complete. 2024-09-21T05:42:28.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2538645487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a415c580 msgr2=0x7fe4a4070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2538645487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a415c580 0x7fe4a4070540 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009830 tx=0x7fe48c02a5c0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2538645487 shutdown_connections 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2538645487 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a415c580 0x7fe4a4070540 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2538645487 >> 172.21.15.22:0/2538645487 conn(0x7fe4a406def0 msgr2=0x7fe4a406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2538645487 shutdown_connections 2024-09-21T05:42:28.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.760+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2538645487 wait complete. 2024-09-21T05:42:28.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.762+0000 7fe4ab0bd640 1 Processor -- start 2024-09-21T05:42:28.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.762+0000 7fe4ab0bd640 1 -- start start 2024-09-21T05:42:28.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4ab0bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 0x7fe4a4157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:28.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4ab0bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a4155ce0 0x7fe4a415a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4ab0bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a415aca0 0x7fe4a4159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4ab0bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a40b5280 con 0x7fe4a4074420 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4ab0bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a40b5100 con 0x7fe4a4155ce0 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4ab0bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a40b5400 con 0x7fe4a415aca0 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 0x7fe4a4157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 0x7fe4a4157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58902/0 (socket says 172.21.15.22:58902) 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 -- 172.21.15.22:0/2007645931 learned_addr learned my addr 172.21.15.22:0/2007645931 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a3fff640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a4155ce0 0x7fe4a415a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a965a640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a415aca0 0x7fe4a4159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 -- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a415aca0 msgr2=0x7fe4a4159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a415aca0 0x7fe4a4159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 -- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a4155ce0 msgr2=0x7fe4a415a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a4155ce0 0x7fe4a415a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 -- 172.21.15.22:0/2007645931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a4157be0 con 0x7fe4a4074420 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a3fff640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a4155ce0 0x7fe4a415a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a965a640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a415aca0 0x7fe4a4159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.764+0000 7fe4a8e59640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 0x7fe4a4157390 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00cbf0 tx=0x7fe49c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.765+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c016020 con 0x7fe4a4074420 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.765+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe49c00add0 con 0x7fe4a4074420 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.765+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a4157ec0 con 0x7fe4a4074420 2024-09-21T05:42:28.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.765+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c012680 con 0x7fe4a4074420 2024-09-21T05:42:28.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.765+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a4158480 con 0x7fe4a4074420 2024-09-21T05:42:28.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.766+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a40cf520 con 0x7fe4a4074420 2024-09-21T05:42:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.766+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe49c019020 con 0x7fe4a4074420 2024-09-21T05:42:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.767+0000 7fe4a17fa640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe484078b90 0x7fe48407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.767+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5609+0+0 (secure 0 0 0) 0x7fe49c09b050 con 0x7fe4a4074420 2024-09-21T05:42:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.767+0000 7fe4a3fff640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe484078b90 0x7fe48407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:28.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.768+0000 7fe4a3fff640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe484078b90 0x7fe48407b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe48c034000 tx=0x7fe48c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:28.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:28.772+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe49c020030 con 0x7fe4a4074420 2024-09-21T05:42:29.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.000+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe4a415c590 con 0x7fe4a4074420 2024-09-21T05:42:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.607+0000 7fe4a17fa640 1 -- 172.21.15.22:0/2007645931 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v41) ==== 174+0+0 (secure 0 0 0) 0x7fe49c0634f0 con 0x7fe4a4074420 2024-09-21T05:42:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 10 and data pool 11 2024-09-21T05:42:29.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.612+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe484078b90 msgr2=0x7fe48407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.612+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe484078b90 0x7fe48407b0a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe48c034000 tx=0x7fe48c004730 comp rx=0 tx=0).stop 2024-09-21T05:42:29.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.612+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 msgr2=0x7fe4a4157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.612+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 0x7fe4a4157390 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00cbf0 tx=0x7fe49c00a430 comp rx=0 tx=0).stop 2024-09-21T05:42:29.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.613+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 shutdown_connections 2024-09-21T05:42:29.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.613+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe484078b90 0x7fe48407b0a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.613+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4a415aca0 0x7fe4a4159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.613+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4a4155ce0 0x7fe4a415a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.613+0000 7fe4ab0bd640 1 --2- 172.21.15.22:0/2007645931 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4a4074420 0x7fe4a4157390 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.613+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 >> 172.21.15.22:0/2007645931 conn(0x7fe4a406def0 msgr2=0x7fe4a415dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:29.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.614+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 shutdown_connections 2024-09-21T05:42:29.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.615+0000 7fe4ab0bd640 1 -- 172.21.15.22:0/2007645931 wait complete. 2024-09-21T05:42:29.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:42:29.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 -- 172.21.15.22:0/3522605332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f8005280 msgr2=0x7f70f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 --2- 172.21.15.22:0/3522605332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f8005280 0x7f70f800b9c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7108009ed0 tx=0x7f70f8005830 comp rx=0 tx=0).stop 2024-09-21T05:42:29.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 -- 172.21.15.22:0/3522605332 shutdown_connections 2024-09-21T05:42:29.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 --2- 172.21.15.22:0/3522605332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f70f8005280 0x7f70f800b9c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 -- 172.21.15.22:0/3522605332 >> 172.21.15.22:0/3522605332 conn(0x7f7110072ae0 msgr2=0x7f7110072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:29.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 -- 172.21.15.22:0/3522605332 shutdown_connections 2024-09-21T05:42:29.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.769+0000 7f711575f640 1 -- 172.21.15.22:0/3522605332 wait complete. 2024-09-21T05:42:29.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.771+0000 7f711575f640 1 Processor -- start 2024-09-21T05:42:29.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.771+0000 7f711575f640 1 -- start start 2024-09-21T05:42:29.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f711575f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f8005280 0x7f71100cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f711575f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f711006eb00 0x7f711006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f711575f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 0x7f71101f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f711575f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f711015ca70 con 0x7f711006f450 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f711575f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f711015c8f0 con 0x7f70f8005280 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f711575f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f711015cbf0 con 0x7f711006eb00 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f710f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 0x7f71101f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f710f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 0x7f71101f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58918/0 (socket says 172.21.15.22:58918) 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.772+0000 7f710f7fe640 1 -- 172.21.15.22:0/1670546085 learned_addr learned my addr 172.21.15.22:0/1670546085 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710effd640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f8005280 0x7f71100cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710e7fc640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f711006eb00 0x7f711006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710f7fe640 1 -- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f711006eb00 msgr2=0x7f711006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710f7fe640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f711006eb00 0x7f711006ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710f7fe640 1 -- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f8005280 msgr2=0x7f71100cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710f7fe640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f8005280 0x7f71100cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710f7fe640 1 -- 172.21.15.22:0/1670546085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711006f860 con 0x7f711006f450 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f710f7fe640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 0x7f71101f8db0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f710802af20 tx=0x7f7108034ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108041070 con 0x7f711006f450 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71080325f0 con 0x7f711006f450 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71101599c0 con 0x7f711006f450 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108009200 con 0x7f711006f450 2024-09-21T05:42:29.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.773+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711006e300 con 0x7f711006f450 2024-09-21T05:42:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.774+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71100d1000 con 0x7f711006f450 2024-09-21T05:42:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.775+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7108044070 con 0x7f711006f450 2024-09-21T05:42:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.775+0000 7f70f77fe640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70fc078b90 0x7f70fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.776+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f71080bb380 con 0x7f711006f450 2024-09-21T05:42:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.776+0000 7f710effd640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70fc078b90 0x7f70fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:29.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.776+0000 7f710effd640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70fc078b90 0x7f70fc07b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7100001220 tx=0x7f7100006720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:29.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.780+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7108083200 con 0x7f711006f450 2024-09-21T05:42:29.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.972+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7110159750 con 0x7f711006f450 2024-09-21T05:42:29.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.973+0000 7f70f77fe640 1 -- 172.21.15.22:0/1670546085 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16729 (secure 0 0 0) 0x7f7108087c30 con 0x7f711006f450 2024-09-21T05:42:29.976 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:29.976 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":67,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:42:29.001873+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:42:26.665513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T05:42:27.958639+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:42:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70fc078b90 msgr2=0x7f70fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70fc078b90 0x7f70fc07b0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7100001220 tx=0x7f7100006720 comp rx=0 tx=0).stop 2024-09-21T05:42:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 msgr2=0x7f71101f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:29.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 0x7f71101f8db0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f710802af20 tx=0x7f7108034ca0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 shutdown_connections 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f711006f450 0x7f71101f8db0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f70fc078b90 0x7f70fc07b0a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f711006eb00 0x7f711006ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 --2- 172.21.15.22:0/1670546085 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f70f8005280 0x7f71100cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 >> 172.21.15.22:0/1670546085 conn(0x7f7110072ae0 msgr2=0x7f71100d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 shutdown_connections 2024-09-21T05:42:29.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:29.976+0000 7f711575f640 1 -- 172.21.15.22:0/1670546085 wait complete. 2024-09-21T05:42:29.990 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:42:30.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.103+0000 7f283cfe6640 1 -- 172.21.15.22:0/3468435821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380d14f0 msgr2=0x7f2838162b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.103+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3468435821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380d14f0 0x7f2838162b40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2830005fd0 tx=0x7f283002aa10 comp rx=0 tx=0).stop 2024-09-21T05:42:30.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.103+0000 7f283cfe6640 1 -- 172.21.15.22:0/3468435821 shutdown_connections 2024-09-21T05:42:30.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.103+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3468435821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380d14f0 0x7f2838162b40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.103+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3468435821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28380b9b60 0x7f28380d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.103+0000 7f283cfe6640 1 -- 172.21.15.22:0/3468435821 >> 172.21.15.22:0/3468435821 conn(0x7f283809d8f0 msgr2=0x7f28380d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:30.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.104+0000 7f283cfe6640 1 -- 172.21.15.22:0/3468435821 shutdown_connections 2024-09-21T05:42:30.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.104+0000 7f283cfe6640 1 -- 172.21.15.22:0/3468435821 wait complete. 2024-09-21T05:42:30.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.106+0000 7f283cfe6640 1 Processor -- start 2024-09-21T05:42:30.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.106+0000 7f283cfe6640 1 -- start start 2024-09-21T05:42:30.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283cfe6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28380b3a10 0x7f28380caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283cfe6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 0x7f28381f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283cfe6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28380d14f0 0x7f28381f9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283cfe6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2838163080 con 0x7f28380b9b60 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283cfe6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28380d1fa0 con 0x7f28380d14f0 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283cfe6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2838163200 con 0x7f28380b3a10 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f282dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 0x7f28381f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f282dd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 0x7f28381f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58936/0 (socket says 172.21.15.22:58936) 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f282dd9b640 1 -- 172.21.15.22:0/3185676826 learned_addr learned my addr 172.21.15.22:0/3185676826 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:30.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f283659c640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28380b3a10 0x7f28380caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f2836d9d640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28380d14f0 0x7f28381f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f282dd9b640 1 -- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28380b3a10 msgr2=0x7f28380caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.107+0000 7f282dd9b640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28380b3a10 0x7f28380caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282dd9b640 1 -- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28380d14f0 msgr2=0x7f28381f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282dd9b640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28380d14f0 0x7f28381f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282dd9b640 1 -- 172.21.15.22:0/3185676826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2838155830 con 0x7f28380b9b60 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282dd9b640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 0x7f28381f9010 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f282000c370 tx=0x7f282000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820017070 con 0x7f28380b9b60 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28200053c0 con 0x7f28380b9b60 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28380b6ea0 con 0x7f28380b9b60 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820003850 con 0x7f28380b9b60 2024-09-21T05:42:30.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.108+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2838154f90 con 0x7f28380b9b60 2024-09-21T05:42:30.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.109+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28381542e0 con 0x7f28380b9b60 2024-09-21T05:42:30.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.110+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f282001a070 con 0x7f28380b9b60 2024-09-21T05:42:30.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.110+0000 7f282f7fe640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2818078b90 0x7f281807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.110+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f282009a090 con 0x7f28380b9b60 2024-09-21T05:42:30.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.111+0000 7f283659c640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2818078b90 0x7f281807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.111+0000 7f283659c640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2818078b90 0x7f281807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2828009b30 tx=0x7f2828005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:30.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.115+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28381542e0 con 0x7f28380b9b60 2024-09-21T05:42:30.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.301+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28380ba0d0 con 0x7f28380b9b60 2024-09-21T05:42:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.303+0000 7f282f7fe640 1 -- 172.21.15.22:0/3185676826 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16729 (secure 0 0 0) 0x7f28380d1900 con 0x7f28380b9b60 2024-09-21T05:42:30.305 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:30.305 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":67,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:42:29.001873+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:42:26.665513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T05:42:27.958639+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:42:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2818078b90 msgr2=0x7f281807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2818078b90 0x7f281807b0a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2828009b30 tx=0x7f2828005f90 comp rx=0 tx=0).stop 2024-09-21T05:42:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 msgr2=0x7f28381f9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 0x7f28381f9010 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f282000c370 tx=0x7f282000c830 comp rx=0 tx=0).stop 2024-09-21T05:42:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 shutdown_connections 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28380d14f0 0x7f28381f9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28380b9b60 0x7f28381f9010 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2818078b90 0x7f281807b0a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 --2- 172.21.15.22:0/3185676826 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28380b3a10 0x7f28380caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 >> 172.21.15.22:0/3185676826 conn(0x7f283809d8f0 msgr2=0x7f2838159310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 shutdown_connections 2024-09-21T05:42:30.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.305+0000 7f283cfe6640 1 -- 172.21.15.22:0/3185676826 wait complete. 2024-09-21T05:42:30.319 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:42:30.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 -- 172.21.15.22:0/2031554346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380b7fe0 msgr2=0x7f76380ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 --2- 172.21.15.22:0/2031554346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380b7fe0 0x7f76380ced30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7638072440 tx=0x7f762c02a4d0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 -- 172.21.15.22:0/2031554346 shutdown_connections 2024-09-21T05:42:30.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 --2- 172.21.15.22:0/2031554346 >> v2:172.21.15.134:3300/0 conn(0x7f76380cf270 0x7f7638072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 --2- 172.21.15.22:0/2031554346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380b7fe0 0x7f76380ced30 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 --2- 172.21.15.22:0/2031554346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76380cd360 0x7f763815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 -- 172.21.15.22:0/2031554346 >> 172.21.15.22:0/2031554346 conn(0x7f76380710b0 msgr2=0x7f76380714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:30.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 -- 172.21.15.22:0/2031554346 shutdown_connections 2024-09-21T05:42:30.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.436+0000 7f763e162640 1 -- 172.21.15.22:0/2031554346 wait complete. 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.438+0000 7f763e162640 1 Processor -- start 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.439+0000 7f763e162640 1 -- start start 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f763e162640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76380b7fe0 0x7f76380a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f763e162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 0x7f7638159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f763e162640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76380cf270 0x7f76381f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f763e162640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7638073120 con 0x7f76380cd360 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f763e162640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7638072fa0 con 0x7f76380cf270 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f763e162640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f76380732a0 con 0x7f76380b7fe0 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f7636ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 0x7f7638159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f7636ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 0x7f7638159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58948/0 (socket says 172.21.15.22:58948) 2024-09-21T05:42:30.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f7636ffd640 1 -- 172.21.15.22:0/3955096905 learned_addr learned my addr 172.21.15.22:0/3955096905 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f76377fe640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76380b7fe0 0x7f76380a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.440+0000 7f7637fff640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76380cf270 0x7f76381f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f7636ffd640 1 -- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76380b7fe0 msgr2=0x7f76380a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f7636ffd640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76380b7fe0 0x7f76380a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f7636ffd640 1 -- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76380cf270 msgr2=0x7f76381f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f7636ffd640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76380cf270 0x7f76381f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f7636ffd640 1 -- 172.21.15.22:0/3955096905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763815fca0 con 0x7f76380cd360 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f7636ffd640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 0x7f7638159930 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f762c0099c0 tx=0x7f762c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c03e030 con 0x7f76380cd360 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f762c003db0 con 0x7f76380cd360 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c035040 con 0x7f76380cd360 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763815ea70 con 0x7f76380cd360 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.441+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763815f010 con 0x7f76380cd360 2024-09-21T05:42:30.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.442+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f763815c6a0 con 0x7f76380cd360 2024-09-21T05:42:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.443+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f762c041030 con 0x7f76380cd360 2024-09-21T05:42:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.443+0000 7f761ffff640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7624078b90 0x7f762407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.444+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f762c0bb870 con 0x7f76380cd360 2024-09-21T05:42:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.444+0000 7f76377fe640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7624078b90 0x7f762407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.444+0000 7f76377fe640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7624078b90 0x7f762407b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7628009a40 tx=0x7f7628005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:30.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.448+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f763815fca0 con 0x7f76380cd360 2024-09-21T05:42:30.677 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:30.675+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f76381f7c70 con 0x7f76380cd360 2024-09-21T05:42:31.585 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-21T05:42:31.582+0000 7f6abf3f7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:42:31.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.629+0000 7f761ffff640 1 -- 172.21.15.22:0/3955096905 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f762c083c30 con 0x7f76380cd360 2024-09-21T05:42:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7624078b90 msgr2=0x7f762407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7624078b90 0x7f762407b0a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7628009a40 tx=0x7f7628005f90 comp rx=0 tx=0).stop 2024-09-21T05:42:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 msgr2=0x7f7638159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 0x7f7638159930 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f762c0099c0 tx=0x7f762c0038a0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 shutdown_connections 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f76380cf270 0x7f76381f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f76380cd360 0x7f7638159930 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7624078b90 0x7f762407b0a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 --2- 172.21.15.22:0/3955096905 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f76380b7fe0 0x7f76380a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 >> 172.21.15.22:0/3955096905 conn(0x7f76380710b0 msgr2=0x7f76380d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 shutdown_connections 2024-09-21T05:42:31.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.632+0000 7f763e162640 1 -- 172.21.15.22:0/3955096905 wait complete. 2024-09-21T05:42:31.648 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:42:31.648 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.774+0000 7f1c7f59e640 1 -- 172.21.15.22:0/3707002532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c68004d90 msgr2=0x7f1c68005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.774+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/3707002532 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c68004d90 0x7f1c68005220 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1c78005fd0 tx=0x7f1c68005830 comp rx=0 tx=0).stop 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.774+0000 7f1c7ed9d640 1 -- 172.21.15.22:0/3707002532 reap_dead start 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.774+0000 7f1c7f59e640 1 -- 172.21.15.22:0/3707002532 shutdown_connections 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.774+0000 7f1c7f59e640 1 -- 172.21.15.22:0/3707002532 >> 172.21.15.22:0/3707002532 conn(0x7f1c8009d680 msgr2=0x7f1c800d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.775+0000 7f1c7f59e640 1 -- 172.21.15.22:0/3707002532 shutdown_connections 2024-09-21T05:42:31.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.775+0000 7f1c7f59e640 1 -- 172.21.15.22:0/3707002532 wait complete. 2024-09-21T05:42:31.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.777+0000 7f1c7f59e640 1 Processor -- start 2024-09-21T05:42:31.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.777+0000 7f1c7f59e640 1 -- start start 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7f59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c68004d90 0x7f1c801d3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c800a1620 0x7f1c800a1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 0x7f1c801e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7f59e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1c800b5040 con 0x7f1c801e54a0 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1c800b4ec0 con 0x7f1c800a1620 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7f59e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1c800b4700 con 0x7f1c68004d90 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 0x7f1c801e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 0x7f1c801e58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58972/0 (socket says 172.21.15.22:58972) 2024-09-21T05:42:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 -- 172.21.15.22:0/1030746671 learned_addr learned my addr 172.21.15.22:0/1030746671 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7e59c640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c68004d90 0x7f1c801d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7dd9b640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c800a1620 0x7f1c800a1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 -- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c68004d90 msgr2=0x7f1c801d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c68004d90 0x7f1c801d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 -- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c800a1620 msgr2=0x7f1c800a1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c800a1620 0x7f1c800a1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.778+0000 7f1c7ed9d640 1 -- 172.21.15.22:0/1030746671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c801e5df0 con 0x7f1c801e54a0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.779+0000 7f1c7ed9d640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 0x7f1c801e58b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1c78036040 tx=0x7f1c78034680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.779+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78038040 con 0x7f1c801e54a0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.779+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c7802b650 con 0x7f1c801e54a0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.779+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c801d5bb0 con 0x7f1c801e54a0 2024-09-21T05:42:31.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.779+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78004d80 con 0x7f1c801e54a0 2024-09-21T05:42:31.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.779+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c801d6170 con 0x7f1c801e54a0 2024-09-21T05:42:31.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.780+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c800b4c10 con 0x7f1c801e54a0 2024-09-21T05:42:31.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.781+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c78044030 con 0x7f1c801e54a0 2024-09-21T05:42:31.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.781+0000 7f1c6effd640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1c60078b90 0x7f1c6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:31.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.781+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f1c780ba970 con 0x7f1c801e54a0 2024-09-21T05:42:31.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.782+0000 7f1c7e59c640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1c60078b90 0x7f1c6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:31.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.782+0000 7f1c7e59c640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1c60078b90 0x7f1c6007b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c70005820 tx=0x7f1c70005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:31.786+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c7804b060 con 0x7f1c801e54a0 2024-09-21T05:42:32.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.015+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1c801d3570 con 0x7f1c801e54a0 2024-09-21T05:42:32.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.634+0000 7f1c6effd640 1 -- 172.21.15.22:0/1030746671 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f1c801e5df0 con 0x7f1c801e54a0 2024-09-21T05:42:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1c60078b90 msgr2=0x7f1c6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1c60078b90 0x7f1c6007b0a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1c70005820 tx=0x7f1c70005f90 comp rx=0 tx=0).stop 2024-09-21T05:42:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 msgr2=0x7f1c801e58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 0x7f1c801e58b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1c78036040 tx=0x7f1c78034680 comp rx=0 tx=0).stop 2024-09-21T05:42:32.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 shutdown_connections 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1c801e54a0 0x7f1c801e58b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1c800a1620 0x7f1c800a1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1c60078b90 0x7f1c6007b0a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 --2- 172.21.15.22:0/1030746671 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1c68004d90 0x7f1c801d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 >> 172.21.15.22:0/1030746671 conn(0x7f1c8009d680 msgr2=0x7f1c800b6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 shutdown_connections 2024-09-21T05:42:32.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.637+0000 7f1c7f59e640 1 -- 172.21.15.22:0/1030746671 wait complete. 2024-09-21T05:42:32.653 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:42:32.653 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:32.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 -- 172.21.15.22:0/1009079273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 msgr2=0x7f75180cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 --2- 172.21.15.22:0/1009079273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 0x7f75180cf090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7508009830 tx=0x7f750802a510 comp rx=0 tx=0).stop 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 -- 172.21.15.22:0/1009079273 shutdown_connections 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 --2- 172.21.15.22:0/1009079273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 0x7f75180cf090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 --2- 172.21.15.22:0/1009079273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75180d2ed0 0x7f75180d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 -- 172.21.15.22:0/1009079273 >> 172.21.15.22:0/1009079273 conn(0x7f751808bfc0 msgr2=0x7f751809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 -- 172.21.15.22:0/1009079273 shutdown_connections 2024-09-21T05:42:32.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.773+0000 7f751f700640 1 -- 172.21.15.22:0/1009079273 wait complete. 2024-09-21T05:42:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.775+0000 7f751f700640 1 Processor -- start 2024-09-21T05:42:32.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.775+0000 7f751f700640 1 -- start start 2024-09-21T05:42:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751f700640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 0x7f751809e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751f700640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75180cf5d0 0x7f7518151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751f700640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 0x7f75180ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751f700640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75180d4940 con 0x7f75180d2ed0 2024-09-21T05:42:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751f700640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75180cfde0 con 0x7f75180b9b70 2024-09-21T05:42:32.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751f700640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f75180d4ac0 con 0x7f75180cf5d0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751dc9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 0x7f75180ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751dc9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 0x7f75180ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58978/0 (socket says 172.21.15.22:58978) 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751d49c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 0x7f751809e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751dc9d640 1 -- 172.21.15.22:0/2995840616 learned_addr learned my addr 172.21.15.22:0/2995840616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.776+0000 7f751cc9b640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75180cf5d0 0x7f7518151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751dc9d640 1 -- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75180cf5d0 msgr2=0x7f7518151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751dc9d640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75180cf5d0 0x7f7518151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751dc9d640 1 -- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 msgr2=0x7f751809e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751dc9d640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 0x7f751809e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751dc9d640 1 -- 172.21.15.22:0/2995840616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75180cbe00 con 0x7f75180d2ed0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751dc9d640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 0x7f75180ca2c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f751000c720 tx=0x7f751000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510003b70 con 0x7f75180d2ed0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7510003d30 con 0x7f75180d2ed0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751809e7e0 con 0x7f75180d2ed0 2024-09-21T05:42:32.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751000aab0 con 0x7f75180d2ed0 2024-09-21T05:42:32.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.777+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7518151d70 con 0x7f75180d2ed0 2024-09-21T05:42:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.778+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75180d5050 con 0x7f75180d2ed0 2024-09-21T05:42:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.779+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f751000ac50 con 0x7f75180d2ed0 2024-09-21T05:42:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.780+0000 7f750dffb640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f74fc078b90 0x7f74fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.780+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f751009ab90 con 0x7f75180d2ed0 2024-09-21T05:42:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.780+0000 7f751d49c640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f74fc078b90 0x7f74fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:32.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.780+0000 7f751d49c640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f74fc078b90 0x7f74fc07b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7514009660 tx=0x7f75140091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:32.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:32.784+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7510062860 con 0x7f75180d2ed0 2024-09-21T05:42:33.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.014+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75180b4020 con 0x7f75180d2ed0 2024-09-21T05:42:33.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.015+0000 7f750dffb640 1 -- 172.21.15.22:0/2995840616 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5228 (secure 0 0 0) 0x7f7510067290 con 0x7f75180d2ed0 2024-09-21T05:42:33.017 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:33.017 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":45,"btime":"2024-09-21T05:42:32:630314+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:32.016336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/1820431436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1820431436},{"type":"v1","addr":"172.21.15.134:6837","nonce":1820431436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-09-21T05:42:33.017 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 45 2024-09-21T05:42:33.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f74fc078b90 msgr2=0x7f74fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f74fc078b90 0x7f74fc07b0a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7514009660 tx=0x7f75140091e0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 msgr2=0x7f75180ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 0x7f75180ca2c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f751000c720 tx=0x7f751000cbe0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 shutdown_connections 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75180d2ed0 0x7f75180ca2c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f74fc078b90 0x7f74fc07b0a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75180cf5d0 0x7f7518151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 --2- 172.21.15.22:0/2995840616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75180b9b70 0x7f751809e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 >> 172.21.15.22:0/2995840616 conn(0x7f751808bfc0 msgr2=0x7f7518161cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 shutdown_connections 2024-09-21T05:42:33.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.017+0000 7f751f700640 1 -- 172.21.15.22:0/2995840616 wait complete. 2024-09-21T05:42:33.031 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1217978835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1d4005330 msgr2=0x7fa1d400b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1217978835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1d4005330 0x7fa1d400b9e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40043b0 tx=0x7fa1d4005830 comp rx=0 tx=0).stop 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1217978835 shutdown_connections 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1217978835 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1d4005330 0x7fa1d400b9e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1217978835 >> 172.21.15.22:0/1217978835 conn(0x7fa1ec0cee20 msgr2=0x7fa1ec0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1217978835 shutdown_connections 2024-09-21T05:42:33.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.145+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1217978835 wait complete. 2024-09-21T05:42:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.147+0000 7fa1f1d4a640 1 Processor -- start 2024-09-21T05:42:33.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.147+0000 7fa1f1d4a640 1 -- start start 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1d4005330 0x7fa1ec06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 0x7fa1ec06ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1ec1523e0 0x7fa1ec1527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b3e20 con 0x7fa1ec154f30 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec15cba0 con 0x7fa1ec1523e0 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec0b3fa0 con 0x7fa1d4005330 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 0x7fa1ec06ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 0x7fa1ec06ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58994/0 (socket says 172.21.15.22:58994) 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 -- 172.21.15.22:0/1627081710 learned_addr learned my addr 172.21.15.22:0/1627081710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1ebfff640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1ec1523e0 0x7fa1ec1527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eb7fe640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1d4005330 0x7fa1ec06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 -- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1d4005330 msgr2=0x7fa1ec06e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1d4005330 0x7fa1ec06e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 -- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1ec1523e0 msgr2=0x7fa1ec1527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1ec1523e0 0x7fa1ec1527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 -- 172.21.15.22:0/1627081710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec1f8b30 con 0x7fa1ec154f30 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1eaffd640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 0x7fa1ec06ed00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0006660 tx=0x7fa1e00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0015070 con 0x7fa1ec154f30 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1e000f980 con 0x7fa1ec154f30 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec1f8e10 con 0x7fa1ec154f30 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.149+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000fdd0 con 0x7fa1ec154f30 2024-09-21T05:42:33.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.150+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec1f93d0 con 0x7fa1ec154f30 2024-09-21T05:42:33.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.151+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec1f88a0 con 0x7fa1ec154f30 2024-09-21T05:42:33.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.151+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa1e0011430 con 0x7fa1ec154f30 2024-09-21T05:42:33.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.152+0000 7fa1d3fff640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa1d8078b90 0x7fa1d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.152+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e009b930 con 0x7fa1ec154f30 2024-09-21T05:42:33.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.152+0000 7fa1eb7fe640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa1d8078b90 0x7fa1d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.153+0000 7fa1eb7fe640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa1d8078b90 0x7fa1d807b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc003700 tx=0x7fa1dc00b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:33.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.157+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1e00637c0 con 0x7fa1ec154f30 2024-09-21T05:42:33.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.344+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1ec0b44a0 con 0x7fa1ec154f30 2024-09-21T05:42:33.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.345+0000 7fa1d3fff640 1 -- 172.21.15.22:0/1627081710 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16716 (secure 0 0 0) 0x7fa1e00681f0 con 0x7fa1ec154f30 2024-09-21T05:42:33.347 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:33.347 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":68,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:42:30.617274+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:42:26.665513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T05:42:27.958639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:42:33.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa1d8078b90 msgr2=0x7fa1d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa1d8078b90 0x7fa1d807b0a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc003700 tx=0x7fa1dc00b260 comp rx=0 tx=0).stop 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 msgr2=0x7fa1ec06ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 0x7fa1ec06ed00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0006660 tx=0x7fa1e00039c0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 shutdown_connections 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa1ec1523e0 0x7fa1ec1527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa1ec154f30 0x7fa1ec06ed00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa1d8078b90 0x7fa1d807b0a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 --2- 172.21.15.22:0/1627081710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa1d4005330 0x7fa1ec06e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 >> 172.21.15.22:0/1627081710 conn(0x7fa1ec0cee20 msgr2=0x7fa1ec0b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 shutdown_connections 2024-09-21T05:42:33.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.347+0000 7fa1f1d4a640 1 -- 172.21.15.22:0/1627081710 wait complete. 2024-09-21T05:42:33.361 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:42:33.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:42:33.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 -- 172.21.15.22:0/2200868362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3570005250 msgr2=0x7f357000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 --2- 172.21.15.22:0/2200868362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3570005250 0x7f357000b9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3580005fd0 tx=0x7f3570005830 comp rx=0 tx=0).stop 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 -- 172.21.15.22:0/2200868362 shutdown_connections 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 --2- 172.21.15.22:0/2200868362 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3570005250 0x7f357000b9c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 -- 172.21.15.22:0/2200868362 >> 172.21.15.22:0/2200868362 conn(0x7f35880b2160 msgr2=0x7f35880b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 -- 172.21.15.22:0/2200868362 shutdown_connections 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.476+0000 7f358e4bd640 1 -- 172.21.15.22:0/2200868362 wait complete. 2024-09-21T05:42:33.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.478+0000 7f358e4bd640 1 Processor -- start 2024-09-21T05:42:33.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.478+0000 7f358e4bd640 1 -- start start 2024-09-21T05:42:33.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358e4bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3570005250 0x7f35880cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358e4bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 0x7f358815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358e4bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f358815a8a0 0x7f35881f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358e4bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f35880cf250 con 0x7f358814ac90 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358e4bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f35880cf0d0 con 0x7f358815a8a0 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358e4bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f35880cf3d0 con 0x7f3570005250 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f35877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 0x7f358815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f35877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 0x7f358815a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59008/0 (socket says 172.21.15.22:59008) 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f35877fe640 1 -- 172.21.15.22:0/572635920 learned_addr learned my addr 172.21.15.22:0/572635920 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f358ca5a640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f358815a8a0 0x7f35881f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.479+0000 7f3587fff640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3570005250 0x7f35880cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f35877fe640 1 -- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3570005250 msgr2=0x7f35880cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f35877fe640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3570005250 0x7f35880cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f35877fe640 1 -- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f358815a8a0 msgr2=0x7f35881f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f35877fe640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f358815a8a0 0x7f35881f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f35877fe640 1 -- 172.21.15.22:0/572635920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358815acb0 con 0x7f358814ac90 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f35877fe640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 0x7f358815a360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f357c001050 tx=0x7f357c00eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c01c070 con 0x7f358814ac90 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f357c00a790 con 0x7f358814ac90 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c00abe0 con 0x7f358814ac90 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3588147c40 con 0x7f358814ac90 2024-09-21T05:42:33.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.480+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358814a1b0 con 0x7f358814ac90 2024-09-21T05:42:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.481+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3588149fd0 con 0x7f358814ac90 2024-09-21T05:42:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.482+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f357c01f070 con 0x7f358814ac90 2024-09-21T05:42:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.483+0000 7f3584ff9640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3574078b90 0x7f357407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.483+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f357c09c460 con 0x7f358814ac90 2024-09-21T05:42:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.483+0000 7f3587fff640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3574078b90 0x7f357407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.483+0000 7f3587fff640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3574078b90 0x7f357407b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f3578006310 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:33.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.487+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3588149fd0 con 0x7f358814ac90 2024-09-21T05:42:33.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.673+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f358815c920 con 0x7f3574078b90 2024-09-21T05:42:33.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.678+0000 7f3584ff9640 1 -- 172.21.15.22:0/572635920 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252056 (secure 0 0 0) 0x7f358815c920 con 0x7f3574078b90 2024-09-21T05:42:33.683 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:33.684 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:42:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3574078b90 msgr2=0x7f357407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3574078b90 0x7f357407b0a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3578000c00 tx=0x7f3578006310 comp rx=0 tx=0).stop 2024-09-21T05:42:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 msgr2=0x7f358815a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 0x7f358815a360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f357c001050 tx=0x7f357c00eb10 comp rx=0 tx=0).stop 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 shutdown_connections 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f358815a8a0 0x7f35881f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f358814ac90 0x7f358815a360 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3574078b90 0x7f357407b0a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 --2- 172.21.15.22:0/572635920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3570005250 0x7f35880cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 >> 172.21.15.22:0/572635920 conn(0x7f35880b2160 msgr2=0x7f35880734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 shutdown_connections 2024-09-21T05:42:33.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.681+0000 7f358e4bd640 1 -- 172.21.15.22:0/572635920 wait complete. 2024-09-21T05:42:33.694 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-21T05:42:31.785200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257036,"kb_used_data":7484,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717684,"statfs":{"total":773094113280,"available":772830908416,"internally_reserved":0,"allocated":7663616,"data_stored":3869602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.178563"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.649381+0000","last_change":"2024-09-21T05:42:29.648038+0000","last_active":"2024-09-21T05:42:29.649381+0000","last_peered":"2024-09-21T05:42:29.649381+0000","last_clean":"2024-09-21T05:42:29.649381+0000","last_became_active":"2024-09-21T05:42:29.647589+0000","last_became_peered":"2024-09-21T05:42:29.647589+0000","last_unstale":"2024-09-21T05:42:29.649381+0000","last_undegraded":"2024-09-21T05:42:29.649381+0000","last_fullsized":"2024-09-21T05:42:29.649381+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:34.190668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634627+0000","last_change":"2024-09-21T05:42:29.632172+0000","last_active":"2024-09-21T05:42:29.634627+0000","last_peered":"2024-09-21T05:42:29.634627+0000","last_clean":"2024-09-21T05:42:29.634627+0000","last_became_active":"2024-09-21T05:42:29.631862+0000","last_became_peered":"2024-09-21T05:42:29.631862+0000","last_unstale":"2024-09-21T05:42:29.634627+0000","last_undegraded":"2024-09-21T05:42:29.634627+0000","last_fullsized":"2024-09-21T05:42:29.634627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:19.183020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629405+0000","last_change":"2024-09-21T05:42:29.624132+0000","last_active":"2024-09-21T05:42:29.629405+0000","last_peered":"2024-09-21T05:42:29.629405+0000","last_clean":"2024-09-21T05:42:29.629405+0000","last_became_active":"2024-09-21T05:42:29.623085+0000","last_became_peered":"2024-09-21T05:42:29.623085+0000","last_unstale":"2024-09-21T05:42:29.629405+0000","last_undegraded":"2024-09-21T05:42:29.629405+0000","last_fullsized":"2024-09-21T05:42:29.629405+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:46.854620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639273+0000","last_change":"2024-09-21T05:42:29.634300+0000","last_active":"2024-09-21T05:42:29.639273+0000","last_peered":"2024-09-21T05:42:29.639273+0000","last_clean":"2024-09-21T05:42:29.639273+0000","last_became_active":"2024-09-21T05:42:29.633093+0000","last_became_peered":"2024-09-21T05:42:29.633093+0000","last_unstale":"2024-09-21T05:42:29.639273+0000","last_undegraded":"2024-09-21T05:42:29.639273+0000","last_fullsized":"2024-09-21T05:42:29.639273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.617989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.648941+0000","last_change":"2024-09-21T05:42:29.647885+0000","last_active":"2024-09-21T05:42:29.648941+0000","last_peered":"2024-09-21T05:42:29.648941+0000","last_clean":"2024-09-21T05:42:29.648941+0000","last_became_active":"2024-09-21T05:42:29.647536+0000","last_became_peered":"2024-09-21T05:42:29.647536+0000","last_unstale":"2024-09-21T05:42:29.648941+0000","last_undegraded":"2024-09-21T05:42:29.648941+0000","last_fullsized":"2024-09-21T05:42:29.648941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.978216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.633718+0000","last_change":"2024-09-21T05:42:29.631184+0000","last_active":"2024-09-21T05:42:29.633718+0000","last_peered":"2024-09-21T05:42:29.633718+0000","last_clean":"2024-09-21T05:42:29.633718+0000","last_became_active":"2024-09-21T05:42:29.630872+0000","last_became_peered":"2024-09-21T05:42:29.630872+0000","last_unstale":"2024-09-21T05:42:29.633718+0000","last_undegraded":"2024-09-21T05:42:29.633718+0000","last_fullsized":"2024-09-21T05:42:29.633718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.570735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641084+0000","last_change":"2024-09-21T05:42:29.639260+0000","last_active":"2024-09-21T05:42:29.641084+0000","last_peered":"2024-09-21T05:42:29.641084+0000","last_clean":"2024-09-21T05:42:29.641084+0000","last_became_active":"2024-09-21T05:42:29.636105+0000","last_became_peered":"2024-09-21T05:42:29.636105+0000","last_unstale":"2024-09-21T05:42:29.641084+0000","last_undegraded":"2024-09-21T05:42:29.641084+0000","last_fullsized":"2024-09-21T05:42:29.641084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:35.048290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.631785+0000","last_change":"2024-09-21T05:42:29.629049+0000","last_active":"2024-09-21T05:42:29.631785+0000","last_peered":"2024-09-21T05:42:29.631785+0000","last_clean":"2024-09-21T05:42:29.631785+0000","last_became_active":"2024-09-21T05:42:29.628201+0000","last_became_peered":"2024-09-21T05:42:29.628201+0000","last_unstale":"2024-09-21T05:42:29.631785+0000","last_undegraded":"2024-09-21T05:42:29.631785+0000","last_fullsized":"2024-09-21T05:42:29.631785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:33.040902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641126+0000","last_change":"2024-09-21T05:42:29.633406+0000","last_active":"2024-09-21T05:42:29.641126+0000","last_peered":"2024-09-21T05:42:29.641126+0000","last_clean":"2024-09-21T05:42:29.641126+0000","last_became_active":"2024-09-21T05:42:29.633102+0000","last_became_peered":"2024-09-21T05:42:29.633102+0000","last_unstale":"2024-09-21T05:42:29.641126+0000","last_undegraded":"2024-09-21T05:42:29.641126+0000","last_fullsized":"2024-09-21T05:42:29.641126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:09.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627538+0000","last_change":"2024-09-21T05:42:29.635366+0000","last_active":"2024-09-21T05:42:30.627538+0000","last_peered":"2024-09-21T05:42:30.627538+0000","last_clean":"2024-09-21T05:42:30.627538+0000","last_became_active":"2024-09-21T05:42:29.634910+0000","last_became_peered":"2024-09-21T05:42:29.634910+0000","last_unstale":"2024-09-21T05:42:30.627538+0000","last_undegraded":"2024-09-21T05:42:30.627538+0000","last_fullsized":"2024-09-21T05:42:30.627538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:18.958160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644655+0000","last_change":"2024-09-21T05:42:29.637406+0000","last_active":"2024-09-21T05:42:29.644655+0000","last_peered":"2024-09-21T05:42:29.644655+0000","last_clean":"2024-09-21T05:42:29.644655+0000","last_became_active":"2024-09-21T05:42:29.636914+0000","last_became_peered":"2024-09-21T05:42:29.636914+0000","last_unstale":"2024-09-21T05:42:29.644655+0000","last_undegraded":"2024-09-21T05:42:29.644655+0000","last_fullsized":"2024-09-21T05:42:29.644655+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:52.467337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635499+0000","last_change":"2024-09-21T05:42:29.629650+0000","last_active":"2024-09-21T05:42:29.635499+0000","last_peered":"2024-09-21T05:42:29.635499+0000","last_clean":"2024-09-21T05:42:29.635499+0000","last_became_active":"2024-09-21T05:42:29.629322+0000","last_became_peered":"2024-09-21T05:42:29.629322+0000","last_unstale":"2024-09-21T05:42:29.635499+0000","last_undegraded":"2024-09-21T05:42:29.635499+0000","last_fullsized":"2024-09-21T05:42:29.635499+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:54.283208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638452+0000","last_change":"2024-09-21T05:42:29.634035+0000","last_active":"2024-09-21T05:42:29.638452+0000","last_peered":"2024-09-21T05:42:29.638452+0000","last_clean":"2024-09-21T05:42:29.638452+0000","last_became_active":"2024-09-21T05:42:29.633711+0000","last_became_peered":"2024-09-21T05:42:29.633711+0000","last_unstale":"2024-09-21T05:42:29.638452+0000","last_undegraded":"2024-09-21T05:42:29.638452+0000","last_fullsized":"2024-09-21T05:42:29.638452+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:34.310526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629453+0000","last_change":"2024-09-21T05:42:29.627807+0000","last_active":"2024-09-21T05:42:29.629453+0000","last_peered":"2024-09-21T05:42:29.629453+0000","last_clean":"2024-09-21T05:42:29.629453+0000","last_became_active":"2024-09-21T05:42:29.627499+0000","last_became_peered":"2024-09-21T05:42:29.627499+0000","last_unstale":"2024-09-21T05:42:29.629453+0000","last_undegraded":"2024-09-21T05:42:29.629453+0000","last_fullsized":"2024-09-21T05:42:29.629453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:15.586141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638393+0000","last_change":"2024-09-21T05:42:29.630500+0000","last_active":"2024-09-21T05:42:29.638393+0000","last_peered":"2024-09-21T05:42:29.638393+0000","last_clean":"2024-09-21T05:42:29.638393+0000","last_became_active":"2024-09-21T05:42:29.630136+0000","last_became_peered":"2024-09-21T05:42:29.630136+0000","last_unstale":"2024-09-21T05:42:29.638393+0000","last_undegraded":"2024-09-21T05:42:29.638393+0000","last_fullsized":"2024-09-21T05:42:29.638393+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:54.129078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626386+0000","last_change":"2024-09-21T05:42:29.632793+0000","last_active":"2024-09-21T05:42:30.626386+0000","last_peered":"2024-09-21T05:42:30.626386+0000","last_clean":"2024-09-21T05:42:30.626386+0000","last_became_active":"2024-09-21T05:42:29.632489+0000","last_became_peered":"2024-09-21T05:42:29.632489+0000","last_unstale":"2024-09-21T05:42:30.626386+0000","last_undegraded":"2024-09-21T05:42:30.626386+0000","last_fullsized":"2024-09-21T05:42:30.626386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:46.220507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641149+0000","last_change":"2024-09-21T05:42:29.634230+0000","last_active":"2024-09-21T05:42:29.641149+0000","last_peered":"2024-09-21T05:42:29.641149+0000","last_clean":"2024-09-21T05:42:29.641149+0000","last_became_active":"2024-09-21T05:42:29.633933+0000","last_became_peered":"2024-09-21T05:42:29.633933+0000","last_unstale":"2024-09-21T05:42:29.641149+0000","last_undegraded":"2024-09-21T05:42:29.641149+0000","last_fullsized":"2024-09-21T05:42:29.641149+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:44.369805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639226+0000","last_change":"2024-09-21T05:42:29.632699+0000","last_active":"2024-09-21T05:42:29.639226+0000","last_peered":"2024-09-21T05:42:29.639226+0000","last_clean":"2024-09-21T05:42:29.639226+0000","last_became_active":"2024-09-21T05:42:29.632399+0000","last_became_peered":"2024-09-21T05:42:29.632399+0000","last_unstale":"2024-09-21T05:42:29.639226+0000","last_undegraded":"2024-09-21T05:42:29.639226+0000","last_fullsized":"2024-09-21T05:42:29.639226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:26.229998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.631798+0000","last_change":"2024-09-21T05:42:29.628643+0000","last_active":"2024-09-21T05:42:29.631798+0000","last_peered":"2024-09-21T05:42:29.631798+0000","last_clean":"2024-09-21T05:42:29.631798+0000","last_became_active":"2024-09-21T05:42:29.628302+0000","last_became_peered":"2024-09-21T05:42:29.628302+0000","last_unstale":"2024-09-21T05:42:29.631798+0000","last_undegraded":"2024-09-21T05:42:29.631798+0000","last_fullsized":"2024-09-21T05:42:29.631798+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:39.979847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638439+0000","last_change":"2024-09-21T05:42:29.629838+0000","last_active":"2024-09-21T05:42:29.638439+0000","last_peered":"2024-09-21T05:42:29.638439+0000","last_clean":"2024-09-21T05:42:29.638439+0000","last_became_active":"2024-09-21T05:42:29.629520+0000","last_became_peered":"2024-09-21T05:42:29.629520+0000","last_unstale":"2024-09-21T05:42:29.638439+0000","last_undegraded":"2024-09-21T05:42:29.638439+0000","last_fullsized":"2024-09-21T05:42:29.638439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:35.048290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627645+0000","last_change":"2024-09-21T05:42:29.630406+0000","last_active":"2024-09-21T05:42:30.627645+0000","last_peered":"2024-09-21T05:42:30.627645+0000","last_clean":"2024-09-21T05:42:30.627645+0000","last_became_active":"2024-09-21T05:42:29.629697+0000","last_became_peered":"2024-09-21T05:42:29.629697+0000","last_unstale":"2024-09-21T05:42:30.627645+0000","last_undegraded":"2024-09-21T05:42:30.627645+0000","last_fullsized":"2024-09-21T05:42:30.627645+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:43.479550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626636+0000","last_change":"2024-09-21T05:42:29.626641+0000","last_active":"2024-09-21T05:42:30.626636+0000","last_peered":"2024-09-21T05:42:30.626636+0000","last_clean":"2024-09-21T05:42:30.626636+0000","last_became_active":"2024-09-21T05:42:29.626245+0000","last_became_peered":"2024-09-21T05:42:29.626245+0000","last_unstale":"2024-09-21T05:42:30.626636+0000","last_undegraded":"2024-09-21T05:42:30.626636+0000","last_fullsized":"2024-09-21T05:42:30.626636+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:35.139583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644198+0000","last_change":"2024-09-21T05:42:29.636812+0000","last_active":"2024-09-21T05:42:29.644198+0000","last_peered":"2024-09-21T05:42:29.644198+0000","last_clean":"2024-09-21T05:42:29.644198+0000","last_became_active":"2024-09-21T05:42:29.636501+0000","last_became_peered":"2024-09-21T05:42:29.636501+0000","last_unstale":"2024-09-21T05:42:29.644198+0000","last_undegraded":"2024-09-21T05:42:29.644198+0000","last_fullsized":"2024-09-21T05:42:29.644198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:44.489386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.646033+0000","last_change":"2024-09-21T05:42:29.640920+0000","last_active":"2024-09-21T05:42:29.646033+0000","last_peered":"2024-09-21T05:42:29.646033+0000","last_clean":"2024-09-21T05:42:29.646033+0000","last_became_active":"2024-09-21T05:42:29.640288+0000","last_became_peered":"2024-09-21T05:42:29.640288+0000","last_unstale":"2024-09-21T05:42:29.646033+0000","last_undegraded":"2024-09-21T05:42:29.646033+0000","last_fullsized":"2024-09-21T05:42:29.646033+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:07.579229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627166+0000","last_change":"2024-09-21T05:42:29.640267+0000","last_active":"2024-09-21T05:42:30.627166+0000","last_peered":"2024-09-21T05:42:30.627166+0000","last_clean":"2024-09-21T05:42:30.627166+0000","last_became_active":"2024-09-21T05:42:29.639969+0000","last_became_peered":"2024-09-21T05:42:29.639969+0000","last_unstale":"2024-09-21T05:42:30.627166+0000","last_undegraded":"2024-09-21T05:42:30.627166+0000","last_fullsized":"2024-09-21T05:42:30.627166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:36.730061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642695+0000","last_change":"2024-09-21T05:42:29.640094+0000","last_active":"2024-09-21T05:42:29.642695+0000","last_peered":"2024-09-21T05:42:29.642695+0000","last_clean":"2024-09-21T05:42:29.642695+0000","last_became_active":"2024-09-21T05:42:29.639786+0000","last_became_peered":"2024-09-21T05:42:29.639786+0000","last_unstale":"2024-09-21T05:42:29.642695+0000","last_undegraded":"2024-09-21T05:42:29.642695+0000","last_fullsized":"2024-09-21T05:42:29.642695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:49.494447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640854+0000","last_change":"2024-09-21T05:42:29.637692+0000","last_active":"2024-09-21T05:42:29.640854+0000","last_peered":"2024-09-21T05:42:29.640854+0000","last_clean":"2024-09-21T05:42:29.640854+0000","last_became_active":"2024-09-21T05:42:29.637367+0000","last_became_peered":"2024-09-21T05:42:29.637367+0000","last_unstale":"2024-09-21T05:42:29.640854+0000","last_undegraded":"2024-09-21T05:42:29.640854+0000","last_fullsized":"2024-09-21T05:42:29.640854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:04.826513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627004+0000","last_change":"2024-09-21T05:42:29.628959+0000","last_active":"2024-09-21T05:42:30.627004+0000","last_peered":"2024-09-21T05:42:30.627004+0000","last_clean":"2024-09-21T05:42:30.627004+0000","last_became_active":"2024-09-21T05:42:29.628654+0000","last_became_peered":"2024-09-21T05:42:29.628654+0000","last_unstale":"2024-09-21T05:42:30.627004+0000","last_undegraded":"2024-09-21T05:42:30.627004+0000","last_fullsized":"2024-09-21T05:42:30.627004+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:20.908369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.630944+0000","last_change":"2024-09-21T05:42:29.626535+0000","last_active":"2024-09-21T05:42:29.630944+0000","last_peered":"2024-09-21T05:42:29.630944+0000","last_clean":"2024-09-21T05:42:29.630944+0000","last_became_active":"2024-09-21T05:42:29.625976+0000","last_became_peered":"2024-09-21T05:42:29.625976+0000","last_unstale":"2024-09-21T05:42:29.630944+0000","last_undegraded":"2024-09-21T05:42:29.630944+0000","last_fullsized":"2024-09-21T05:42:29.630944+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:19.572091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642453+0000","last_change":"2024-09-21T05:42:29.640214+0000","last_active":"2024-09-21T05:42:29.642453+0000","last_peered":"2024-09-21T05:42:29.642453+0000","last_clean":"2024-09-21T05:42:29.642453+0000","last_became_active":"2024-09-21T05:42:29.639302+0000","last_became_peered":"2024-09-21T05:42:29.639302+0000","last_unstale":"2024-09-21T05:42:29.642453+0000","last_undegraded":"2024-09-21T05:42:29.642453+0000","last_fullsized":"2024-09-21T05:42:29.642453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:54.129078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639590+0000","last_change":"2024-09-21T05:42:29.635365+0000","last_active":"2024-09-21T05:42:29.639590+0000","last_peered":"2024-09-21T05:42:29.639590+0000","last_clean":"2024-09-21T05:42:29.639590+0000","last_became_active":"2024-09-21T05:42:29.630284+0000","last_became_peered":"2024-09-21T05:42:29.630284+0000","last_unstale":"2024-09-21T05:42:29.639590+0000","last_undegraded":"2024-09-21T05:42:29.639590+0000","last_fullsized":"2024-09-21T05:42:29.639590+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:28.731480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634728+0000","last_change":"2024-09-21T05:42:29.632611+0000","last_active":"2024-09-21T05:42:29.634728+0000","last_peered":"2024-09-21T05:42:29.634728+0000","last_clean":"2024-09-21T05:42:29.634728+0000","last_became_active":"2024-09-21T05:42:29.632288+0000","last_became_peered":"2024-09-21T05:42:29.632288+0000","last_unstale":"2024-09-21T05:42:29.634728+0000","last_undegraded":"2024-09-21T05:42:29.634728+0000","last_fullsized":"2024-09-21T05:42:29.634728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:28.860484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644678+0000","last_change":"2024-09-21T05:42:29.639226+0000","last_active":"2024-09-21T05:42:29.644678+0000","last_peered":"2024-09-21T05:42:29.644678+0000","last_clean":"2024-09-21T05:42:29.644678+0000","last_became_active":"2024-09-21T05:42:29.638913+0000","last_became_peered":"2024-09-21T05:42:29.638913+0000","last_unstale":"2024-09-21T05:42:29.644678+0000","last_undegraded":"2024-09-21T05:42:29.644678+0000","last_fullsized":"2024-09-21T05:42:29.644678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:23.523706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.645786+0000","last_change":"2024-09-21T05:42:28.632567+0000","last_active":"2024-09-21T05:42:29.645786+0000","last_peered":"2024-09-21T05:42:29.645786+0000","last_clean":"2024-09-21T05:42:29.645786+0000","last_became_active":"2024-09-21T05:42:28.632246+0000","last_became_peered":"2024-09-21T05:42:28.632246+0000","last_unstale":"2024-09-21T05:42:29.645786+0000","last_undegraded":"2024-09-21T05:42:29.645786+0000","last_fullsized":"2024-09-21T05:42:29.645786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:12.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641097+0000","last_change":"2024-09-21T05:42:29.632560+0000","last_active":"2024-09-21T05:42:29.641097+0000","last_peered":"2024-09-21T05:42:29.641097+0000","last_clean":"2024-09-21T05:42:29.641097+0000","last_became_active":"2024-09-21T05:42:29.632255+0000","last_became_peered":"2024-09-21T05:42:29.632255+0000","last_unstale":"2024-09-21T05:42:29.641097+0000","last_undegraded":"2024-09-21T05:42:29.641097+0000","last_fullsized":"2024-09-21T05:42:29.641097+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:53.576314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642657+0000","last_change":"2024-09-21T05:42:28.632770+0000","last_active":"2024-09-21T05:42:29.642657+0000","last_peered":"2024-09-21T05:42:29.642657+0000","last_clean":"2024-09-21T05:42:29.642657+0000","last_became_active":"2024-09-21T05:42:28.632521+0000","last_became_peered":"2024-09-21T05:42:28.632521+0000","last_unstale":"2024-09-21T05:42:29.642657+0000","last_undegraded":"2024-09-21T05:42:29.642657+0000","last_fullsized":"2024-09-21T05:42:29.642657+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:32.946652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644329+0000","last_change":"2024-09-21T05:42:29.642713+0000","last_active":"2024-09-21T05:42:29.644329+0000","last_peered":"2024-09-21T05:42:29.644329+0000","last_clean":"2024-09-21T05:42:29.644329+0000","last_became_active":"2024-09-21T05:42:29.642124+0000","last_became_peered":"2024-09-21T05:42:29.642124+0000","last_unstale":"2024-09-21T05:42:29.644329+0000","last_undegraded":"2024-09-21T05:42:29.644329+0000","last_fullsized":"2024-09-21T05:42:29.644329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:53.994424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627402+0000","last_change":"2024-09-21T05:42:28.632057+0000","last_active":"2024-09-21T05:42:30.627402+0000","last_peered":"2024-09-21T05:42:30.627402+0000","last_clean":"2024-09-21T05:42:30.627402+0000","last_became_active":"2024-09-21T05:42:28.631748+0000","last_became_peered":"2024-09-21T05:42:28.631748+0000","last_unstale":"2024-09-21T05:42:30.627402+0000","last_undegraded":"2024-09-21T05:42:30.627402+0000","last_fullsized":"2024-09-21T05:42:30.627402+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:46.096579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644679+0000","last_change":"2024-09-21T05:42:29.641244+0000","last_active":"2024-09-21T05:42:29.644679+0000","last_peered":"2024-09-21T05:42:29.644679+0000","last_clean":"2024-09-21T05:42:29.644679+0000","last_became_active":"2024-09-21T05:42:29.640893+0000","last_became_peered":"2024-09-21T05:42:29.640893+0000","last_unstale":"2024-09-21T05:42:29.644679+0000","last_undegraded":"2024-09-21T05:42:29.644679+0000","last_fullsized":"2024-09-21T05:42:29.644679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:19.919362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638363+0000","last_change":"2024-09-21T05:42:28.630169+0000","last_active":"2024-09-21T05:42:29.638363+0000","last_peered":"2024-09-21T05:42:29.638363+0000","last_clean":"2024-09-21T05:42:29.638363+0000","last_became_active":"2024-09-21T05:42:28.629794+0000","last_became_peered":"2024-09-21T05:42:28.629794+0000","last_unstale":"2024-09-21T05:42:29.638363+0000","last_undegraded":"2024-09-21T05:42:29.638363+0000","last_fullsized":"2024-09-21T05:42:29.638363+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:49.003565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641092+0000","last_change":"2024-09-21T05:42:29.638130+0000","last_active":"2024-09-21T05:42:29.641092+0000","last_peered":"2024-09-21T05:42:29.641092+0000","last_clean":"2024-09-21T05:42:29.641092+0000","last_became_active":"2024-09-21T05:42:29.637475+0000","last_became_peered":"2024-09-21T05:42:29.637475+0000","last_unstale":"2024-09-21T05:42:29.641092+0000","last_undegraded":"2024-09-21T05:42:29.641092+0000","last_fullsized":"2024-09-21T05:42:29.641092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:33.948700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638311+0000","last_change":"2024-09-21T05:42:28.622204+0000","last_active":"2024-09-21T05:42:29.638311+0000","last_peered":"2024-09-21T05:42:29.638311+0000","last_clean":"2024-09-21T05:42:29.638311+0000","last_became_active":"2024-09-21T05:42:28.621855+0000","last_became_peered":"2024-09-21T05:42:28.621855+0000","last_unstale":"2024-09-21T05:42:29.638311+0000","last_undegraded":"2024-09-21T05:42:29.638311+0000","last_fullsized":"2024-09-21T05:42:29.638311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:33.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627736+0000","last_change":"2024-09-21T05:42:29.644685+0000","last_active":"2024-09-21T05:42:30.627736+0000","last_peered":"2024-09-21T05:42:30.627736+0000","last_clean":"2024-09-21T05:42:30.627736+0000","last_became_active":"2024-09-21T05:42:29.644380+0000","last_became_peered":"2024-09-21T05:42:29.644380+0000","last_unstale":"2024-09-21T05:42:30.627736+0000","last_undegraded":"2024-09-21T05:42:30.627736+0000","last_fullsized":"2024-09-21T05:42:30.627736+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:24.712694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.632612+0000","last_change":"2024-09-21T05:42:28.629832+0000","last_active":"2024-09-21T05:42:29.632612+0000","last_peered":"2024-09-21T05:42:29.632612+0000","last_clean":"2024-09-21T05:42:29.632612+0000","last_became_active":"2024-09-21T05:42:28.629477+0000","last_became_peered":"2024-09-21T05:42:28.629477+0000","last_unstale":"2024-09-21T05:42:29.632612+0000","last_undegraded":"2024-09-21T05:42:29.632612+0000","last_fullsized":"2024-09-21T05:42:29.632612+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:05.070095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640675+0000","last_change":"2024-09-21T05:42:29.631637+0000","last_active":"2024-09-21T05:42:29.640675+0000","last_peered":"2024-09-21T05:42:29.640675+0000","last_clean":"2024-09-21T05:42:29.640675+0000","last_became_active":"2024-09-21T05:42:29.631227+0000","last_became_peered":"2024-09-21T05:42:29.631227+0000","last_unstale":"2024-09-21T05:42:29.640675+0000","last_undegraded":"2024-09-21T05:42:29.640675+0000","last_fullsized":"2024-09-21T05:42:29.640675+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.352029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.612080+0000","last_change":"2024-09-21T05:42:28.622653+0000","last_active":"2024-09-21T05:42:29.612080+0000","last_peered":"2024-09-21T05:42:29.612080+0000","last_clean":"2024-09-21T05:42:29.612080+0000","last_became_active":"2024-09-21T05:42:28.622339+0000","last_became_peered":"2024-09-21T05:42:28.622339+0000","last_unstale":"2024-09-21T05:42:29.612080+0000","last_undegraded":"2024-09-21T05:42:29.612080+0000","last_fullsized":"2024-09-21T05:42:29.612080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:36.292050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639235+0000","last_change":"2024-09-21T05:42:29.638132+0000","last_active":"2024-09-21T05:42:29.639235+0000","last_peered":"2024-09-21T05:42:29.639235+0000","last_clean":"2024-09-21T05:42:29.639235+0000","last_became_active":"2024-09-21T05:42:29.637819+0000","last_became_peered":"2024-09-21T05:42:29.637819+0000","last_unstale":"2024-09-21T05:42:29.639235+0000","last_undegraded":"2024-09-21T05:42:29.639235+0000","last_fullsized":"2024-09-21T05:42:29.639235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:02.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.618538+0000","last_change":"2024-09-21T05:42:28.625390+0000","last_active":"2024-09-21T05:42:29.618538+0000","last_peered":"2024-09-21T05:42:29.618538+0000","last_clean":"2024-09-21T05:42:29.618538+0000","last_became_active":"2024-09-21T05:42:28.625079+0000","last_became_peered":"2024-09-21T05:42:28.625079+0000","last_unstale":"2024-09-21T05:42:29.618538+0000","last_undegraded":"2024-09-21T05:42:29.618538+0000","last_fullsized":"2024-09-21T05:42:29.618538+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:55.163798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.643555+0000","last_change":"2024-09-21T05:42:29.642469+0000","last_active":"2024-09-21T05:42:29.643555+0000","last_peered":"2024-09-21T05:42:29.643555+0000","last_clean":"2024-09-21T05:42:29.643555+0000","last_became_active":"2024-09-21T05:42:29.642137+0000","last_became_peered":"2024-09-21T05:42:29.642137+0000","last_unstale":"2024-09-21T05:42:29.643555+0000","last_undegraded":"2024-09-21T05:42:29.643555+0000","last_fullsized":"2024-09-21T05:42:29.643555+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:34.310526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627293+0000","last_change":"2024-09-21T05:42:28.624051+0000","last_active":"2024-09-21T05:42:30.627293+0000","last_peered":"2024-09-21T05:42:30.627293+0000","last_clean":"2024-09-21T05:42:30.627293+0000","last_became_active":"2024-09-21T05:42:28.623712+0000","last_became_peered":"2024-09-21T05:42:28.623712+0000","last_unstale":"2024-09-21T05:42:30.627293+0000","last_undegraded":"2024-09-21T05:42:30.627293+0000","last_fullsized":"2024-09-21T05:42:30.627293+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:57.106958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.648344+0000","last_change":"2024-09-21T05:42:29.646396+0000","last_active":"2024-09-21T05:42:29.648344+0000","last_peered":"2024-09-21T05:42:29.648344+0000","last_clean":"2024-09-21T05:42:29.648344+0000","last_became_active":"2024-09-21T05:42:29.645976+0000","last_became_peered":"2024-09-21T05:42:29.645976+0000","last_unstale":"2024-09-21T05:42:29.648344+0000","last_undegraded":"2024-09-21T05:42:29.648344+0000","last_fullsized":"2024-09-21T05:42:29.648344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:46.760816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.619816+0000","last_change":"2024-09-21T05:42:28.633366+0000","last_active":"2024-09-21T05:42:29.619816+0000","last_peered":"2024-09-21T05:42:29.619816+0000","last_clean":"2024-09-21T05:42:29.619816+0000","last_became_active":"2024-09-21T05:42:28.630483+0000","last_became_peered":"2024-09-21T05:42:28.630483+0000","last_unstale":"2024-09-21T05:42:29.619816+0000","last_undegraded":"2024-09-21T05:42:29.619816+0000","last_fullsized":"2024-09-21T05:42:29.619816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:21.714458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.633693+0000","last_change":"2024-09-21T05:42:29.631556+0000","last_active":"2024-09-21T05:42:29.633693+0000","last_peered":"2024-09-21T05:42:29.633693+0000","last_clean":"2024-09-21T05:42:29.633693+0000","last_became_active":"2024-09-21T05:42:29.631206+0000","last_became_peered":"2024-09-21T05:42:29.631206+0000","last_unstale":"2024-09-21T05:42:29.633693+0000","last_undegraded":"2024-09-21T05:42:29.633693+0000","last_fullsized":"2024-09-21T05:42:29.633693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:49.346493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.633927+0000","last_change":"2024-09-21T05:42:28.629044+0000","last_active":"2024-09-21T05:42:29.633927+0000","last_peered":"2024-09-21T05:42:29.633927+0000","last_clean":"2024-09-21T05:42:29.633927+0000","last_became_active":"2024-09-21T05:42:28.628656+0000","last_became_peered":"2024-09-21T05:42:28.628656+0000","last_unstale":"2024-09-21T05:42:29.633927+0000","last_undegraded":"2024-09-21T05:42:29.633927+0000","last_fullsized":"2024-09-21T05:42:29.633927+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:04.590995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642071+0000","last_change":"2024-09-21T05:42:29.637374+0000","last_active":"2024-09-21T05:42:29.642071+0000","last_peered":"2024-09-21T05:42:29.642071+0000","last_clean":"2024-09-21T05:42:29.642071+0000","last_became_active":"2024-09-21T05:42:29.637061+0000","last_became_peered":"2024-09-21T05:42:29.637061+0000","last_unstale":"2024-09-21T05:42:29.642071+0000","last_undegraded":"2024-09-21T05:42:29.642071+0000","last_fullsized":"2024-09-21T05:42:29.642071+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:17.710485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.645332+0000","last_change":"2024-09-21T05:42:28.627013+0000","last_active":"2024-09-21T05:42:29.645332+0000","last_peered":"2024-09-21T05:42:29.645332+0000","last_clean":"2024-09-21T05:42:29.645332+0000","last_became_active":"2024-09-21T05:42:28.626276+0000","last_became_peered":"2024-09-21T05:42:28.626276+0000","last_unstale":"2024-09-21T05:42:29.645332+0000","last_undegraded":"2024-09-21T05:42:29.645332+0000","last_fullsized":"2024-09-21T05:42:29.645332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:04.043487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.649369+0000","last_change":"2024-09-21T05:42:29.647994+0000","last_active":"2024-09-21T05:42:29.649369+0000","last_peered":"2024-09-21T05:42:29.649369+0000","last_clean":"2024-09-21T05:42:29.649369+0000","last_became_active":"2024-09-21T05:42:29.647650+0000","last_became_peered":"2024-09-21T05:42:29.647650+0000","last_unstale":"2024-09-21T05:42:29.649369+0000","last_undegraded":"2024-09-21T05:42:29.649369+0000","last_fullsized":"2024-09-21T05:42:29.649369+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:47.133527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.647648+0000","last_change":"2024-09-21T05:42:28.630356+0000","last_active":"2024-09-21T05:42:29.647648+0000","last_peered":"2024-09-21T05:42:29.647648+0000","last_clean":"2024-09-21T05:42:29.647648+0000","last_became_active":"2024-09-21T05:42:28.630008+0000","last_became_peered":"2024-09-21T05:42:28.630008+0000","last_unstale":"2024-09-21T05:42:29.647648+0000","last_undegraded":"2024-09-21T05:42:29.647648+0000","last_fullsized":"2024-09-21T05:42:29.647648+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:35.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.637966+0000","last_change":"2024-09-21T05:42:29.635920+0000","last_active":"2024-09-21T05:42:29.637966+0000","last_peered":"2024-09-21T05:42:29.637966+0000","last_clean":"2024-09-21T05:42:29.637966+0000","last_became_active":"2024-09-21T05:42:29.635596+0000","last_became_peered":"2024-09-21T05:42:29.635596+0000","last_unstale":"2024-09-21T05:42:29.637966+0000","last_undegraded":"2024-09-21T05:42:29.637966+0000","last_fullsized":"2024-09-21T05:42:29.637966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:23.613737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.619061+0000","last_change":"2024-09-21T05:42:28.626898+0000","last_active":"2024-09-21T05:42:29.619061+0000","last_peered":"2024-09-21T05:42:29.619061+0000","last_clean":"2024-09-21T05:42:29.619061+0000","last_became_active":"2024-09-21T05:42:28.626541+0000","last_became_peered":"2024-09-21T05:42:28.626541+0000","last_unstale":"2024-09-21T05:42:29.619061+0000","last_undegraded":"2024-09-21T05:42:29.619061+0000","last_fullsized":"2024-09-21T05:42:29.619061+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:51.649406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638538+0000","last_change":"2024-09-21T05:42:29.634828+0000","last_active":"2024-09-21T05:42:29.638538+0000","last_peered":"2024-09-21T05:42:29.638538+0000","last_clean":"2024-09-21T05:42:29.638538+0000","last_became_active":"2024-09-21T05:42:29.634545+0000","last_became_peered":"2024-09-21T05:42:29.634545+0000","last_unstale":"2024-09-21T05:42:29.638538+0000","last_undegraded":"2024-09-21T05:42:29.638538+0000","last_fullsized":"2024-09-21T05:42:29.638538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:43.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.620493+0000","last_change":"2024-09-21T05:42:28.631613+0000","last_active":"2024-09-21T05:42:29.620493+0000","last_peered":"2024-09-21T05:42:29.620493+0000","last_clean":"2024-09-21T05:42:29.620493+0000","last_became_active":"2024-09-21T05:42:28.631238+0000","last_became_peered":"2024-09-21T05:42:28.631238+0000","last_unstale":"2024-09-21T05:42:29.620493+0000","last_undegraded":"2024-09-21T05:42:29.620493+0000","last_fullsized":"2024-09-21T05:42:29.620493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:51.188981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641301+0000","last_change":"2024-09-21T05:42:29.631762+0000","last_active":"2024-09-21T05:42:29.641301+0000","last_peered":"2024-09-21T05:42:29.641301+0000","last_clean":"2024-09-21T05:42:29.641301+0000","last_became_active":"2024-09-21T05:42:29.631360+0000","last_became_peered":"2024-09-21T05:42:29.631360+0000","last_unstale":"2024-09-21T05:42:29.641301+0000","last_undegraded":"2024-09-21T05:42:29.641301+0000","last_fullsized":"2024-09-21T05:42:29.641301+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:17.095014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":23,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626283+0000","last_change":"2024-09-21T05:42:28.626976+0000","last_active":"2024-09-21T05:42:30.626283+0000","last_peered":"2024-09-21T05:42:30.626283+0000","last_clean":"2024-09-21T05:42:30.626283+0000","last_became_active":"2024-09-21T05:42:28.626680+0000","last_became_peered":"2024-09-21T05:42:28.626680+0000","last_unstale":"2024-09-21T05:42:30.626283+0000","last_undegraded":"2024-09-21T05:42:30.626283+0000","last_fullsized":"2024-09-21T05:42:30.626283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:56.569120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627792+0000","last_change":"2024-09-21T05:42:29.618303+0000","last_active":"2024-09-21T05:42:30.627792+0000","last_peered":"2024-09-21T05:42:30.627792+0000","last_clean":"2024-09-21T05:42:30.627792+0000","last_became_active":"2024-09-21T05:42:29.617966+0000","last_became_peered":"2024-09-21T05:42:29.617966+0000","last_unstale":"2024-09-21T05:42:30.627792+0000","last_undegraded":"2024-09-21T05:42:30.627792+0000","last_fullsized":"2024-09-21T05:42:30.627792+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:51.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634934+0000","last_change":"2024-09-21T05:42:28.630288+0000","last_active":"2024-09-21T05:42:29.634934+0000","last_peered":"2024-09-21T05:42:29.634934+0000","last_clean":"2024-09-21T05:42:29.634934+0000","last_became_active":"2024-09-21T05:42:28.629950+0000","last_became_peered":"2024-09-21T05:42:28.629950+0000","last_unstale":"2024-09-21T05:42:29.634934+0000","last_undegraded":"2024-09-21T05:42:29.634934+0000","last_fullsized":"2024-09-21T05:42:29.634934+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:06.616255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.646943+0000","last_change":"2024-09-21T05:42:29.643672+0000","last_active":"2024-09-21T05:42:29.646943+0000","last_peered":"2024-09-21T05:42:29.646943+0000","last_clean":"2024-09-21T05:42:29.646943+0000","last_became_active":"2024-09-21T05:42:29.643366+0000","last_became_peered":"2024-09-21T05:42:29.643366+0000","last_unstale":"2024-09-21T05:42:29.646943+0000","last_undegraded":"2024-09-21T05:42:29.646943+0000","last_fullsized":"2024-09-21T05:42:29.646943+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:56.026952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627913+0000","last_change":"2024-09-21T05:42:28.633148+0000","last_active":"2024-09-21T05:42:30.627913+0000","last_peered":"2024-09-21T05:42:30.627913+0000","last_clean":"2024-09-21T05:42:30.627913+0000","last_became_active":"2024-09-21T05:42:28.632774+0000","last_became_peered":"2024-09-21T05:42:28.632774+0000","last_unstale":"2024-09-21T05:42:30.627913+0000","last_undegraded":"2024-09-21T05:42:30.627913+0000","last_fullsized":"2024-09-21T05:42:30.627913+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:07.822576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635465+0000","last_change":"2024-09-21T05:42:29.629257+0000","last_active":"2024-09-21T05:42:29.635465+0000","last_peered":"2024-09-21T05:42:29.635465+0000","last_clean":"2024-09-21T05:42:29.635465+0000","last_became_active":"2024-09-21T05:42:29.628858+0000","last_became_peered":"2024-09-21T05:42:29.628858+0000","last_unstale":"2024-09-21T05:42:29.635465+0000","last_undegraded":"2024-09-21T05:42:29.635465+0000","last_fullsized":"2024-09-21T05:42:29.635465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:01.065279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.643076+0000","last_change":"2024-09-21T05:42:28.633926+0000","last_active":"2024-09-21T05:42:29.643076+0000","last_peered":"2024-09-21T05:42:29.643076+0000","last_clean":"2024-09-21T05:42:29.643076+0000","last_became_active":"2024-09-21T05:42:28.630009+0000","last_became_peered":"2024-09-21T05:42:28.630009+0000","last_unstale":"2024-09-21T05:42:29.643076+0000","last_undegraded":"2024-09-21T05:42:29.643076+0000","last_fullsized":"2024-09-21T05:42:29.643076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.697372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642099+0000","last_change":"2024-09-21T05:42:29.639427+0000","last_active":"2024-09-21T05:42:29.642099+0000","last_peered":"2024-09-21T05:42:29.642099+0000","last_clean":"2024-09-21T05:42:29.642099+0000","last_became_active":"2024-09-21T05:42:29.639111+0000","last_became_peered":"2024-09-21T05:42:29.639111+0000","last_unstale":"2024-09-21T05:42:29.642099+0000","last_undegraded":"2024-09-21T05:42:29.642099+0000","last_fullsized":"2024-09-21T05:42:29.642099+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:24.600827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629081+0000","last_change":"2024-09-21T05:42:28.630315+0000","last_active":"2024-09-21T05:42:29.629081+0000","last_peered":"2024-09-21T05:42:29.629081+0000","last_clean":"2024-09-21T05:42:29.629081+0000","last_became_active":"2024-09-21T05:42:28.629991+0000","last_became_peered":"2024-09-21T05:42:28.629991+0000","last_unstale":"2024-09-21T05:42:29.629081+0000","last_undegraded":"2024-09-21T05:42:29.629081+0000","last_fullsized":"2024-09-21T05:42:29.629081+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:59.458821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629563+0000","last_change":"2024-09-21T05:42:29.623748+0000","last_active":"2024-09-21T05:42:29.629563+0000","last_peered":"2024-09-21T05:42:29.629563+0000","last_clean":"2024-09-21T05:42:29.629563+0000","last_became_active":"2024-09-21T05:42:29.623409+0000","last_became_peered":"2024-09-21T05:42:29.623409+0000","last_unstale":"2024-09-21T05:42:29.629563+0000","last_undegraded":"2024-09-21T05:42:29.629563+0000","last_fullsized":"2024-09-21T05:42:29.629563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:21.015474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.619328+0000","last_change":"2024-09-21T05:42:28.634087+0000","last_active":"2024-09-21T05:42:29.619328+0000","last_peered":"2024-09-21T05:42:29.619328+0000","last_clean":"2024-09-21T05:42:29.619328+0000","last_became_active":"2024-09-21T05:42:28.630918+0000","last_became_peered":"2024-09-21T05:42:28.630918+0000","last_unstale":"2024-09-21T05:42:29.619328+0000","last_undegraded":"2024-09-21T05:42:29.619328+0000","last_fullsized":"2024-09-21T05:42:29.619328+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:00.038687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634720+0000","last_change":"2024-09-21T05:42:29.633013+0000","last_active":"2024-09-21T05:42:29.634720+0000","last_peered":"2024-09-21T05:42:29.634720+0000","last_clean":"2024-09-21T05:42:29.634720+0000","last_became_active":"2024-09-21T05:42:29.632694+0000","last_became_peered":"2024-09-21T05:42:29.632694+0000","last_unstale":"2024-09-21T05:42:29.634720+0000","last_undegraded":"2024-09-21T05:42:29.634720+0000","last_fullsized":"2024-09-21T05:42:29.634720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:45.348405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.620731+0000","last_change":"2024-09-21T05:42:28.633790+0000","last_active":"2024-09-21T05:42:29.620731+0000","last_peered":"2024-09-21T05:42:29.620731+0000","last_clean":"2024-09-21T05:42:29.620731+0000","last_became_active":"2024-09-21T05:42:28.633462+0000","last_became_peered":"2024-09-21T05:42:28.633462+0000","last_unstale":"2024-09-21T05:42:29.620731+0000","last_undegraded":"2024-09-21T05:42:29.620731+0000","last_fullsized":"2024-09-21T05:42:29.620731+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:25.645933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635650+0000","last_change":"2024-09-21T05:42:29.634436+0000","last_active":"2024-09-21T05:42:29.635650+0000","last_peered":"2024-09-21T05:42:29.635650+0000","last_clean":"2024-09-21T05:42:29.635650+0000","last_became_active":"2024-09-21T05:42:29.634114+0000","last_became_peered":"2024-09-21T05:42:29.634114+0000","last_unstale":"2024-09-21T05:42:29.635650+0000","last_undegraded":"2024-09-21T05:42:29.635650+0000","last_fullsized":"2024-09-21T05:42:29.635650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:40.179403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.621747+0000","last_change":"2024-09-21T05:42:28.632897+0000","last_active":"2024-09-21T05:42:29.621747+0000","last_peered":"2024-09-21T05:42:29.621747+0000","last_clean":"2024-09-21T05:42:29.621747+0000","last_became_active":"2024-09-21T05:42:28.632577+0000","last_became_peered":"2024-09-21T05:42:28.632577+0000","last_unstale":"2024-09-21T05:42:29.621747+0000","last_undegraded":"2024-09-21T05:42:29.621747+0000","last_fullsized":"2024-09-21T05:42:29.621747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:38.860781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.637044+0000","last_change":"2024-09-21T05:42:29.630369+0000","last_active":"2024-09-21T05:42:29.637044+0000","last_peered":"2024-09-21T05:42:29.637044+0000","last_clean":"2024-09-21T05:42:29.637044+0000","last_became_active":"2024-09-21T05:42:29.630035+0000","last_became_peered":"2024-09-21T05:42:29.630035+0000","last_unstale":"2024-09-21T05:42:29.637044+0000","last_undegraded":"2024-09-21T05:42:29.637044+0000","last_fullsized":"2024-09-21T05:42:29.637044+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:49.494447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634913+0000","last_change":"2024-09-21T05:42:28.622372+0000","last_active":"2024-09-21T05:42:29.634913+0000","last_peered":"2024-09-21T05:42:29.634913+0000","last_clean":"2024-09-21T05:42:29.634913+0000","last_became_active":"2024-09-21T05:42:28.621989+0000","last_became_peered":"2024-09-21T05:42:28.621989+0000","last_unstale":"2024-09-21T05:42:29.634913+0000","last_undegraded":"2024-09-21T05:42:29.634913+0000","last_fullsized":"2024-09-21T05:42:29.634913+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:04.969519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640888+0000","last_change":"2024-09-21T05:42:29.636758+0000","last_active":"2024-09-21T05:42:29.640888+0000","last_peered":"2024-09-21T05:42:29.640888+0000","last_clean":"2024-09-21T05:42:29.640888+0000","last_became_active":"2024-09-21T05:42:29.636424+0000","last_became_peered":"2024-09-21T05:42:29.636424+0000","last_unstale":"2024-09-21T05:42:29.640888+0000","last_undegraded":"2024-09-21T05:42:29.640888+0000","last_fullsized":"2024-09-21T05:42:29.640888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:39.862829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.621745+0000","last_change":"2024-09-21T05:42:28.631698+0000","last_active":"2024-09-21T05:42:29.621745+0000","last_peered":"2024-09-21T05:42:29.621745+0000","last_clean":"2024-09-21T05:42:29.621745+0000","last_became_active":"2024-09-21T05:42:28.631261+0000","last_became_peered":"2024-09-21T05:42:28.631261+0000","last_unstale":"2024-09-21T05:42:29.621745+0000","last_undegraded":"2024-09-21T05:42:29.621745+0000","last_fullsized":"2024-09-21T05:42:29.621745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:37.452860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644642+0000","last_change":"2024-09-21T05:42:29.638689+0000","last_active":"2024-09-21T05:42:29.644642+0000","last_peered":"2024-09-21T05:42:29.644642+0000","last_clean":"2024-09-21T05:42:29.644642+0000","last_became_active":"2024-09-21T05:42:29.638281+0000","last_became_peered":"2024-09-21T05:42:29.638281+0000","last_unstale":"2024-09-21T05:42:29.644642+0000","last_undegraded":"2024-09-21T05:42:29.644642+0000","last_fullsized":"2024-09-21T05:42:29.644642+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:17.246291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.622583+0000","last_change":"2024-09-21T05:42:28.629767+0000","last_active":"2024-09-21T05:42:29.622583+0000","last_peered":"2024-09-21T05:42:29.622583+0000","last_clean":"2024-09-21T05:42:29.622583+0000","last_became_active":"2024-09-21T05:42:28.629379+0000","last_became_peered":"2024-09-21T05:42:28.629379+0000","last_unstale":"2024-09-21T05:42:29.622583+0000","last_undegraded":"2024-09-21T05:42:29.622583+0000","last_fullsized":"2024-09-21T05:42:29.622583+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:32.125509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.646026+0000","last_change":"2024-09-21T05:42:29.644444+0000","last_active":"2024-09-21T05:42:29.646026+0000","last_peered":"2024-09-21T05:42:29.646026+0000","last_clean":"2024-09-21T05:42:29.646026+0000","last_became_active":"2024-09-21T05:42:29.644127+0000","last_became_peered":"2024-09-21T05:42:29.644127+0000","last_unstale":"2024-09-21T05:42:29.646026+0000","last_undegraded":"2024-09-21T05:42:29.646026+0000","last_fullsized":"2024-09-21T05:42:29.646026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:45.068806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.621666+0000","last_change":"2024-09-21T05:42:28.630093+0000","last_active":"2024-09-21T05:42:29.621666+0000","last_peered":"2024-09-21T05:42:29.621666+0000","last_clean":"2024-09-21T05:42:29.621666+0000","last_became_active":"2024-09-21T05:42:28.629759+0000","last_became_peered":"2024-09-21T05:42:28.629759+0000","last_unstale":"2024-09-21T05:42:29.621666+0000","last_undegraded":"2024-09-21T05:42:29.621666+0000","last_fullsized":"2024-09-21T05:42:29.621666+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.611689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640838+0000","last_change":"2024-09-21T05:42:29.631703+0000","last_active":"2024-09-21T05:42:29.640838+0000","last_peered":"2024-09-21T05:42:29.640838+0000","last_clean":"2024-09-21T05:42:29.640838+0000","last_became_active":"2024-09-21T05:42:29.631373+0000","last_became_peered":"2024-09-21T05:42:29.631373+0000","last_unstale":"2024-09-21T05:42:29.640838+0000","last_undegraded":"2024-09-21T05:42:29.640838+0000","last_fullsized":"2024-09-21T05:42:29.640838+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:44.134131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.623472+0000","last_change":"2024-09-21T05:42:28.629732+0000","last_active":"2024-09-21T05:42:29.623472+0000","last_peered":"2024-09-21T05:42:29.623472+0000","last_clean":"2024-09-21T05:42:29.623472+0000","last_became_active":"2024-09-21T05:42:28.629349+0000","last_became_peered":"2024-09-21T05:42:28.629349+0000","last_unstale":"2024-09-21T05:42:29.623472+0000","last_undegraded":"2024-09-21T05:42:29.623472+0000","last_fullsized":"2024-09-21T05:42:29.623472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:12.226276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.648660+0000","last_change":"2024-09-21T05:42:29.646997+0000","last_active":"2024-09-21T05:42:29.648660+0000","last_peered":"2024-09-21T05:42:29.648660+0000","last_clean":"2024-09-21T05:42:29.648660+0000","last_became_active":"2024-09-21T05:42:29.646464+0000","last_became_peered":"2024-09-21T05:42:29.646464+0000","last_unstale":"2024-09-21T05:42:29.648660+0000","last_undegraded":"2024-09-21T05:42:29.648660+0000","last_fullsized":"2024-09-21T05:42:29.648660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:39.925217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.612889+0000","last_change":"2024-09-21T05:42:28.621403+0000","last_active":"2024-09-21T05:42:29.612889+0000","last_peered":"2024-09-21T05:42:29.612889+0000","last_clean":"2024-09-21T05:42:29.612889+0000","last_became_active":"2024-09-21T05:42:28.621065+0000","last_became_peered":"2024-09-21T05:42:28.621065+0000","last_unstale":"2024-09-21T05:42:29.612889+0000","last_undegraded":"2024-09-21T05:42:29.612889+0000","last_fullsized":"2024-09-21T05:42:29.612889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:20.775273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635297+0000","last_change":"2024-09-21T05:42:29.630366+0000","last_active":"2024-09-21T05:42:29.635297+0000","last_peered":"2024-09-21T05:42:29.635297+0000","last_clean":"2024-09-21T05:42:29.635297+0000","last_became_active":"2024-09-21T05:42:29.627873+0000","last_became_peered":"2024-09-21T05:42:29.627873+0000","last_unstale":"2024-09-21T05:42:29.635297+0000","last_undegraded":"2024-09-21T05:42:29.635297+0000","last_fullsized":"2024-09-21T05:42:29.635297+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:55.537195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626503+0000","last_change":"2024-09-21T05:42:28.628131+0000","last_active":"2024-09-21T05:42:30.626503+0000","last_peered":"2024-09-21T05:42:30.626503+0000","last_clean":"2024-09-21T05:42:30.626503+0000","last_became_active":"2024-09-21T05:42:28.627760+0000","last_became_peered":"2024-09-21T05:42:28.627760+0000","last_unstale":"2024-09-21T05:42:30.626503+0000","last_undegraded":"2024-09-21T05:42:30.626503+0000","last_fullsized":"2024-09-21T05:42:30.626503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:12.334459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634559+0000","last_change":"2024-09-21T05:42:29.633371+0000","last_active":"2024-09-21T05:42:29.634559+0000","last_peered":"2024-09-21T05:42:29.634559+0000","last_clean":"2024-09-21T05:42:29.634559+0000","last_became_active":"2024-09-21T05:42:29.633005+0000","last_became_peered":"2024-09-21T05:42:29.633005+0000","last_unstale":"2024-09-21T05:42:29.634559+0000","last_undegraded":"2024-09-21T05:42:29.634559+0000","last_fullsized":"2024-09-21T05:42:29.634559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:06.875521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.618156+0000","last_change":"2024-09-21T05:42:28.630642+0000","last_active":"2024-09-21T05:42:29.618156+0000","last_peered":"2024-09-21T05:42:29.618156+0000","last_clean":"2024-09-21T05:42:29.618156+0000","last_became_active":"2024-09-21T05:42:28.630243+0000","last_became_peered":"2024-09-21T05:42:28.630243+0000","last_unstale":"2024-09-21T05:42:29.618156+0000","last_undegraded":"2024-09-21T05:42:29.618156+0000","last_fullsized":"2024-09-21T05:42:29.618156+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:16.619859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635824+0000","last_change":"2024-09-21T05:42:29.633358+0000","last_active":"2024-09-21T05:42:29.635824+0000","last_peered":"2024-09-21T05:42:29.635824+0000","last_clean":"2024-09-21T05:42:29.635824+0000","last_became_active":"2024-09-21T05:42:29.632181+0000","last_became_peered":"2024-09-21T05:42:29.632181+0000","last_unstale":"2024-09-21T05:42:29.635824+0000","last_undegraded":"2024-09-21T05:42:29.635824+0000","last_fullsized":"2024-09-21T05:42:29.635824+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:53.649003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639838+0000","last_change":"2024-09-21T05:42:28.630733+0000","last_active":"2024-09-21T05:42:29.639838+0000","last_peered":"2024-09-21T05:42:29.639838+0000","last_clean":"2024-09-21T05:42:29.639838+0000","last_became_active":"2024-09-21T05:42:28.630098+0000","last_became_peered":"2024-09-21T05:42:28.630098+0000","last_unstale":"2024-09-21T05:42:29.639838+0000","last_undegraded":"2024-09-21T05:42:29.639838+0000","last_fullsized":"2024-09-21T05:42:29.639838+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:26.343187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":191,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629268+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:42:29.629268+0000","last_peered":"2024-09-21T05:42:29.629268+0000","last_clean":"2024-09-21T05:42:29.629268+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:42:29.629268+0000","last_undegraded":"2024-09-21T05:42:29.629268+0000","last_fullsized":"2024-09-21T05:42:29.629268+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1409024,"data_stored":928462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":811008,"data_stored":337548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":337570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":337568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":337590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36128,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335712,"statfs":{"total":96636764160,"available":96599769088,"internally_reserved":0,"allocated":794624,"data_stored":330703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1388544,"data_stored":921071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:42:33.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:42:33.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 -- 172.21.15.22:0/3714450795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf68003c30 msgr2=0x7faf68004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 --2- 172.21.15.22:0/3714450795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf68003c30 0x7faf68004080 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faf70009b50 tx=0x7faf68005830 comp rx=0 tx=0).stop 2024-09-21T05:42:33.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 -- 172.21.15.22:0/3714450795 shutdown_connections 2024-09-21T05:42:33.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 --2- 172.21.15.22:0/3714450795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf68003c30 0x7faf68004080 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 -- 172.21.15.22:0/3714450795 >> 172.21.15.22:0/3714450795 conn(0x7faf8008bfc0 msgr2=0x7faf8009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:33.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 -- 172.21.15.22:0/3714450795 shutdown_connections 2024-09-21T05:42:33.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.850+0000 7faf8708b640 1 -- 172.21.15.22:0/3714450795 wait complete. 2024-09-21T05:42:33.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.852+0000 7faf8708b640 1 Processor -- start 2024-09-21T05:42:33.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.852+0000 7faf8708b640 1 -- start start 2024-09-21T05:42:33.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf8708b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf68003c30 0x7faf800ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf8708b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf801607e0 0x7faf8015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf8708b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 0x7faf800b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf8708b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faf800b36c0 con 0x7faf800b2570 2024-09-21T05:42:33.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf8708b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faf800b3540 con 0x7faf68003c30 2024-09-21T05:42:33.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf8708b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faf800b3840 con 0x7faf801607e0 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 0x7faf800b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 0x7faf800b2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59040/0 (socket says 172.21.15.22:59040) 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 -- 172.21.15.22:0/3626519509 learned_addr learned my addr 172.21.15.22:0/3626519509 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf7ffff640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf801607e0 0x7faf8015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf84e27640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf68003c30 0x7faf800ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 -- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf801607e0 msgr2=0x7faf8015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf801607e0 0x7faf8015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 -- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf68003c30 msgr2=0x7faf800ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf68003c30 0x7faf800ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.853+0000 7faf85628640 1 -- 172.21.15.22:0/3626519509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf800b2ec0 con 0x7faf800b2570 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.854+0000 7faf85628640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 0x7faf800b2980 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faf800d0410 tx=0x7faf7800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.854+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7800c910 con 0x7faf800b2570 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.854+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf78017070 con 0x7faf800b2570 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.854+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf800cb180 con 0x7faf800b2570 2024-09-21T05:42:33.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.854+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7800ce80 con 0x7faf800b2570 2024-09-21T05:42:33.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.854+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf800ccef0 con 0x7faf800b2570 2024-09-21T05:42:33.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.855+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf800cf110 con 0x7faf800b2570 2024-09-21T05:42:33.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.856+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faf7801c460 con 0x7faf800b2570 2024-09-21T05:42:33.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.856+0000 7faf7d7fa640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faf6c078bc0 0x7faf6c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:33.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.857+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7faf7809aeb0 con 0x7faf800b2570 2024-09-21T05:42:33.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.857+0000 7faf84e27640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faf6c078bc0 0x7faf6c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:33.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.857+0000 7faf84e27640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faf6c078bc0 0x7faf6c07b0d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faf70035000 tx=0x7faf700046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:33.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:33.861+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf800b2ec0 con 0x7faf800b2570 2024-09-21T05:42:34.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.044+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf800b83c0 con 0x7faf6c078bc0 2024-09-21T05:42:34.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.049+0000 7faf7d7fa640 1 -- 172.21.15.22:0/3626519509 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252056 (secure 0 0 0) 0x7faf800b83c0 con 0x7faf6c078bc0 2024-09-21T05:42:34.054 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:34.055 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.052+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faf6c078bc0 msgr2=0x7faf6c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.052+0000 7faf8708b640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faf6c078bc0 0x7faf6c07b0d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faf70035000 tx=0x7faf700046c0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.052+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 msgr2=0x7faf800b2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.052+0000 7faf8708b640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 0x7faf800b2980 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faf800d0410 tx=0x7faf7800ea90 comp rx=0 tx=0).stop 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 shutdown_connections 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faf800b2570 0x7faf800b2980 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faf6c078bc0 0x7faf6c07b0d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faf801607e0 0x7faf8015fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 --2- 172.21.15.22:0/3626519509 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faf68003c30 0x7faf800ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 >> 172.21.15.22:0/3626519509 conn(0x7faf8008bfc0 msgr2=0x7faf801618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 shutdown_connections 2024-09-21T05:42:34.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.053+0000 7faf8708b640 1 -- 172.21.15.22:0/3626519509 wait complete. 2024-09-21T05:42:34.066 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-21T05:42:33.785667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257036,"kb_used_data":7484,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717684,"statfs":{"total":773094113280,"available":772830908416,"internally_reserved":0,"allocated":7663616,"data_stored":3869602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001271"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.649381+0000","last_change":"2024-09-21T05:42:29.648038+0000","last_active":"2024-09-21T05:42:29.649381+0000","last_peered":"2024-09-21T05:42:29.649381+0000","last_clean":"2024-09-21T05:42:29.649381+0000","last_became_active":"2024-09-21T05:42:29.647589+0000","last_became_peered":"2024-09-21T05:42:29.647589+0000","last_unstale":"2024-09-21T05:42:29.649381+0000","last_undegraded":"2024-09-21T05:42:29.649381+0000","last_fullsized":"2024-09-21T05:42:29.649381+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:34.190668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634627+0000","last_change":"2024-09-21T05:42:29.632172+0000","last_active":"2024-09-21T05:42:29.634627+0000","last_peered":"2024-09-21T05:42:29.634627+0000","last_clean":"2024-09-21T05:42:29.634627+0000","last_became_active":"2024-09-21T05:42:29.631862+0000","last_became_peered":"2024-09-21T05:42:29.631862+0000","last_unstale":"2024-09-21T05:42:29.634627+0000","last_undegraded":"2024-09-21T05:42:29.634627+0000","last_fullsized":"2024-09-21T05:42:29.634627+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:16:19.183020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629405+0000","last_change":"2024-09-21T05:42:29.624132+0000","last_active":"2024-09-21T05:42:29.629405+0000","last_peered":"2024-09-21T05:42:29.629405+0000","last_clean":"2024-09-21T05:42:29.629405+0000","last_became_active":"2024-09-21T05:42:29.623085+0000","last_became_peered":"2024-09-21T05:42:29.623085+0000","last_unstale":"2024-09-21T05:42:29.629405+0000","last_undegraded":"2024-09-21T05:42:29.629405+0000","last_fullsized":"2024-09-21T05:42:29.629405+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:46.854620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639273+0000","last_change":"2024-09-21T05:42:29.634300+0000","last_active":"2024-09-21T05:42:29.639273+0000","last_peered":"2024-09-21T05:42:29.639273+0000","last_clean":"2024-09-21T05:42:29.639273+0000","last_became_active":"2024-09-21T05:42:29.633093+0000","last_became_peered":"2024-09-21T05:42:29.633093+0000","last_unstale":"2024-09-21T05:42:29.639273+0000","last_undegraded":"2024-09-21T05:42:29.639273+0000","last_fullsized":"2024-09-21T05:42:29.639273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.617989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.648941+0000","last_change":"2024-09-21T05:42:29.647885+0000","last_active":"2024-09-21T05:42:29.648941+0000","last_peered":"2024-09-21T05:42:29.648941+0000","last_clean":"2024-09-21T05:42:29.648941+0000","last_became_active":"2024-09-21T05:42:29.647536+0000","last_became_peered":"2024-09-21T05:42:29.647536+0000","last_unstale":"2024-09-21T05:42:29.648941+0000","last_undegraded":"2024-09-21T05:42:29.648941+0000","last_fullsized":"2024-09-21T05:42:29.648941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:39.978216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.633718+0000","last_change":"2024-09-21T05:42:29.631184+0000","last_active":"2024-09-21T05:42:29.633718+0000","last_peered":"2024-09-21T05:42:29.633718+0000","last_clean":"2024-09-21T05:42:29.633718+0000","last_became_active":"2024-09-21T05:42:29.630872+0000","last_became_peered":"2024-09-21T05:42:29.630872+0000","last_unstale":"2024-09-21T05:42:29.633718+0000","last_undegraded":"2024-09-21T05:42:29.633718+0000","last_fullsized":"2024-09-21T05:42:29.633718+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.570735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641084+0000","last_change":"2024-09-21T05:42:29.639260+0000","last_active":"2024-09-21T05:42:29.641084+0000","last_peered":"2024-09-21T05:42:29.641084+0000","last_clean":"2024-09-21T05:42:29.641084+0000","last_became_active":"2024-09-21T05:42:29.636105+0000","last_became_peered":"2024-09-21T05:42:29.636105+0000","last_unstale":"2024-09-21T05:42:29.641084+0000","last_undegraded":"2024-09-21T05:42:29.641084+0000","last_fullsized":"2024-09-21T05:42:29.641084+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:35.048290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.631785+0000","last_change":"2024-09-21T05:42:29.629049+0000","last_active":"2024-09-21T05:42:29.631785+0000","last_peered":"2024-09-21T05:42:29.631785+0000","last_clean":"2024-09-21T05:42:29.631785+0000","last_became_active":"2024-09-21T05:42:29.628201+0000","last_became_peered":"2024-09-21T05:42:29.628201+0000","last_unstale":"2024-09-21T05:42:29.631785+0000","last_undegraded":"2024-09-21T05:42:29.631785+0000","last_fullsized":"2024-09-21T05:42:29.631785+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:33.040902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641126+0000","last_change":"2024-09-21T05:42:29.633406+0000","last_active":"2024-09-21T05:42:29.641126+0000","last_peered":"2024-09-21T05:42:29.641126+0000","last_clean":"2024-09-21T05:42:29.641126+0000","last_became_active":"2024-09-21T05:42:29.633102+0000","last_became_peered":"2024-09-21T05:42:29.633102+0000","last_unstale":"2024-09-21T05:42:29.641126+0000","last_undegraded":"2024-09-21T05:42:29.641126+0000","last_fullsized":"2024-09-21T05:42:29.641126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:09.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627538+0000","last_change":"2024-09-21T05:42:29.635366+0000","last_active":"2024-09-21T05:42:30.627538+0000","last_peered":"2024-09-21T05:42:30.627538+0000","last_clean":"2024-09-21T05:42:30.627538+0000","last_became_active":"2024-09-21T05:42:29.634910+0000","last_became_peered":"2024-09-21T05:42:29.634910+0000","last_unstale":"2024-09-21T05:42:30.627538+0000","last_undegraded":"2024-09-21T05:42:30.627538+0000","last_fullsized":"2024-09-21T05:42:30.627538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:18.958160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644655+0000","last_change":"2024-09-21T05:42:29.637406+0000","last_active":"2024-09-21T05:42:29.644655+0000","last_peered":"2024-09-21T05:42:29.644655+0000","last_clean":"2024-09-21T05:42:29.644655+0000","last_became_active":"2024-09-21T05:42:29.636914+0000","last_became_peered":"2024-09-21T05:42:29.636914+0000","last_unstale":"2024-09-21T05:42:29.644655+0000","last_undegraded":"2024-09-21T05:42:29.644655+0000","last_fullsized":"2024-09-21T05:42:29.644655+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:52.467337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635499+0000","last_change":"2024-09-21T05:42:29.629650+0000","last_active":"2024-09-21T05:42:29.635499+0000","last_peered":"2024-09-21T05:42:29.635499+0000","last_clean":"2024-09-21T05:42:29.635499+0000","last_became_active":"2024-09-21T05:42:29.629322+0000","last_became_peered":"2024-09-21T05:42:29.629322+0000","last_unstale":"2024-09-21T05:42:29.635499+0000","last_undegraded":"2024-09-21T05:42:29.635499+0000","last_fullsized":"2024-09-21T05:42:29.635499+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:54.283208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638452+0000","last_change":"2024-09-21T05:42:29.634035+0000","last_active":"2024-09-21T05:42:29.638452+0000","last_peered":"2024-09-21T05:42:29.638452+0000","last_clean":"2024-09-21T05:42:29.638452+0000","last_became_active":"2024-09-21T05:42:29.633711+0000","last_became_peered":"2024-09-21T05:42:29.633711+0000","last_unstale":"2024-09-21T05:42:29.638452+0000","last_undegraded":"2024-09-21T05:42:29.638452+0000","last_fullsized":"2024-09-21T05:42:29.638452+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:34.310526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629453+0000","last_change":"2024-09-21T05:42:29.627807+0000","last_active":"2024-09-21T05:42:29.629453+0000","last_peered":"2024-09-21T05:42:29.629453+0000","last_clean":"2024-09-21T05:42:29.629453+0000","last_became_active":"2024-09-21T05:42:29.627499+0000","last_became_peered":"2024-09-21T05:42:29.627499+0000","last_unstale":"2024-09-21T05:42:29.629453+0000","last_undegraded":"2024-09-21T05:42:29.629453+0000","last_fullsized":"2024-09-21T05:42:29.629453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:15.586141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638393+0000","last_change":"2024-09-21T05:42:29.630500+0000","last_active":"2024-09-21T05:42:29.638393+0000","last_peered":"2024-09-21T05:42:29.638393+0000","last_clean":"2024-09-21T05:42:29.638393+0000","last_became_active":"2024-09-21T05:42:29.630136+0000","last_became_peered":"2024-09-21T05:42:29.630136+0000","last_unstale":"2024-09-21T05:42:29.638393+0000","last_undegraded":"2024-09-21T05:42:29.638393+0000","last_fullsized":"2024-09-21T05:42:29.638393+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:54.129078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626386+0000","last_change":"2024-09-21T05:42:29.632793+0000","last_active":"2024-09-21T05:42:30.626386+0000","last_peered":"2024-09-21T05:42:30.626386+0000","last_clean":"2024-09-21T05:42:30.626386+0000","last_became_active":"2024-09-21T05:42:29.632489+0000","last_became_peered":"2024-09-21T05:42:29.632489+0000","last_unstale":"2024-09-21T05:42:30.626386+0000","last_undegraded":"2024-09-21T05:42:30.626386+0000","last_fullsized":"2024-09-21T05:42:30.626386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:46.220507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641149+0000","last_change":"2024-09-21T05:42:29.634230+0000","last_active":"2024-09-21T05:42:29.641149+0000","last_peered":"2024-09-21T05:42:29.641149+0000","last_clean":"2024-09-21T05:42:29.641149+0000","last_became_active":"2024-09-21T05:42:29.633933+0000","last_became_peered":"2024-09-21T05:42:29.633933+0000","last_unstale":"2024-09-21T05:42:29.641149+0000","last_undegraded":"2024-09-21T05:42:29.641149+0000","last_fullsized":"2024-09-21T05:42:29.641149+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:44.369805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639226+0000","last_change":"2024-09-21T05:42:29.632699+0000","last_active":"2024-09-21T05:42:29.639226+0000","last_peered":"2024-09-21T05:42:29.639226+0000","last_clean":"2024-09-21T05:42:29.639226+0000","last_became_active":"2024-09-21T05:42:29.632399+0000","last_became_peered":"2024-09-21T05:42:29.632399+0000","last_unstale":"2024-09-21T05:42:29.639226+0000","last_undegraded":"2024-09-21T05:42:29.639226+0000","last_fullsized":"2024-09-21T05:42:29.639226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:26.229998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.631798+0000","last_change":"2024-09-21T05:42:29.628643+0000","last_active":"2024-09-21T05:42:29.631798+0000","last_peered":"2024-09-21T05:42:29.631798+0000","last_clean":"2024-09-21T05:42:29.631798+0000","last_became_active":"2024-09-21T05:42:29.628302+0000","last_became_peered":"2024-09-21T05:42:29.628302+0000","last_unstale":"2024-09-21T05:42:29.631798+0000","last_undegraded":"2024-09-21T05:42:29.631798+0000","last_fullsized":"2024-09-21T05:42:29.631798+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:39.979847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638439+0000","last_change":"2024-09-21T05:42:29.629838+0000","last_active":"2024-09-21T05:42:29.638439+0000","last_peered":"2024-09-21T05:42:29.638439+0000","last_clean":"2024-09-21T05:42:29.638439+0000","last_became_active":"2024-09-21T05:42:29.629520+0000","last_became_peered":"2024-09-21T05:42:29.629520+0000","last_unstale":"2024-09-21T05:42:29.638439+0000","last_undegraded":"2024-09-21T05:42:29.638439+0000","last_fullsized":"2024-09-21T05:42:29.638439+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:35.048290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627645+0000","last_change":"2024-09-21T05:42:29.630406+0000","last_active":"2024-09-21T05:42:30.627645+0000","last_peered":"2024-09-21T05:42:30.627645+0000","last_clean":"2024-09-21T05:42:30.627645+0000","last_became_active":"2024-09-21T05:42:29.629697+0000","last_became_peered":"2024-09-21T05:42:29.629697+0000","last_unstale":"2024-09-21T05:42:30.627645+0000","last_undegraded":"2024-09-21T05:42:30.627645+0000","last_fullsized":"2024-09-21T05:42:30.627645+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:43.479550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626636+0000","last_change":"2024-09-21T05:42:29.626641+0000","last_active":"2024-09-21T05:42:30.626636+0000","last_peered":"2024-09-21T05:42:30.626636+0000","last_clean":"2024-09-21T05:42:30.626636+0000","last_became_active":"2024-09-21T05:42:29.626245+0000","last_became_peered":"2024-09-21T05:42:29.626245+0000","last_unstale":"2024-09-21T05:42:30.626636+0000","last_undegraded":"2024-09-21T05:42:30.626636+0000","last_fullsized":"2024-09-21T05:42:30.626636+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:35.139583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644198+0000","last_change":"2024-09-21T05:42:29.636812+0000","last_active":"2024-09-21T05:42:29.644198+0000","last_peered":"2024-09-21T05:42:29.644198+0000","last_clean":"2024-09-21T05:42:29.644198+0000","last_became_active":"2024-09-21T05:42:29.636501+0000","last_became_peered":"2024-09-21T05:42:29.636501+0000","last_unstale":"2024-09-21T05:42:29.644198+0000","last_undegraded":"2024-09-21T05:42:29.644198+0000","last_fullsized":"2024-09-21T05:42:29.644198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:09:44.489386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.646033+0000","last_change":"2024-09-21T05:42:29.640920+0000","last_active":"2024-09-21T05:42:29.646033+0000","last_peered":"2024-09-21T05:42:29.646033+0000","last_clean":"2024-09-21T05:42:29.646033+0000","last_became_active":"2024-09-21T05:42:29.640288+0000","last_became_peered":"2024-09-21T05:42:29.640288+0000","last_unstale":"2024-09-21T05:42:29.646033+0000","last_undegraded":"2024-09-21T05:42:29.646033+0000","last_fullsized":"2024-09-21T05:42:29.646033+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:07.579229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627166+0000","last_change":"2024-09-21T05:42:29.640267+0000","last_active":"2024-09-21T05:42:30.627166+0000","last_peered":"2024-09-21T05:42:30.627166+0000","last_clean":"2024-09-21T05:42:30.627166+0000","last_became_active":"2024-09-21T05:42:29.639969+0000","last_became_peered":"2024-09-21T05:42:29.639969+0000","last_unstale":"2024-09-21T05:42:30.627166+0000","last_undegraded":"2024-09-21T05:42:30.627166+0000","last_fullsized":"2024-09-21T05:42:30.627166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:36.730061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642695+0000","last_change":"2024-09-21T05:42:29.640094+0000","last_active":"2024-09-21T05:42:29.642695+0000","last_peered":"2024-09-21T05:42:29.642695+0000","last_clean":"2024-09-21T05:42:29.642695+0000","last_became_active":"2024-09-21T05:42:29.639786+0000","last_became_peered":"2024-09-21T05:42:29.639786+0000","last_unstale":"2024-09-21T05:42:29.642695+0000","last_undegraded":"2024-09-21T05:42:29.642695+0000","last_fullsized":"2024-09-21T05:42:29.642695+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:49.494447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640854+0000","last_change":"2024-09-21T05:42:29.637692+0000","last_active":"2024-09-21T05:42:29.640854+0000","last_peered":"2024-09-21T05:42:29.640854+0000","last_clean":"2024-09-21T05:42:29.640854+0000","last_became_active":"2024-09-21T05:42:29.637367+0000","last_became_peered":"2024-09-21T05:42:29.637367+0000","last_unstale":"2024-09-21T05:42:29.640854+0000","last_undegraded":"2024-09-21T05:42:29.640854+0000","last_fullsized":"2024-09-21T05:42:29.640854+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:04.826513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627004+0000","last_change":"2024-09-21T05:42:29.628959+0000","last_active":"2024-09-21T05:42:30.627004+0000","last_peered":"2024-09-21T05:42:30.627004+0000","last_clean":"2024-09-21T05:42:30.627004+0000","last_became_active":"2024-09-21T05:42:29.628654+0000","last_became_peered":"2024-09-21T05:42:29.628654+0000","last_unstale":"2024-09-21T05:42:30.627004+0000","last_undegraded":"2024-09-21T05:42:30.627004+0000","last_fullsized":"2024-09-21T05:42:30.627004+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:20.908369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.630944+0000","last_change":"2024-09-21T05:42:29.626535+0000","last_active":"2024-09-21T05:42:29.630944+0000","last_peered":"2024-09-21T05:42:29.630944+0000","last_clean":"2024-09-21T05:42:29.630944+0000","last_became_active":"2024-09-21T05:42:29.625976+0000","last_became_peered":"2024-09-21T05:42:29.625976+0000","last_unstale":"2024-09-21T05:42:29.630944+0000","last_undegraded":"2024-09-21T05:42:29.630944+0000","last_fullsized":"2024-09-21T05:42:29.630944+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:19.572091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642453+0000","last_change":"2024-09-21T05:42:29.640214+0000","last_active":"2024-09-21T05:42:29.642453+0000","last_peered":"2024-09-21T05:42:29.642453+0000","last_clean":"2024-09-21T05:42:29.642453+0000","last_became_active":"2024-09-21T05:42:29.639302+0000","last_became_peered":"2024-09-21T05:42:29.639302+0000","last_unstale":"2024-09-21T05:42:29.642453+0000","last_undegraded":"2024-09-21T05:42:29.642453+0000","last_fullsized":"2024-09-21T05:42:29.642453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:54.129078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639590+0000","last_change":"2024-09-21T05:42:29.635365+0000","last_active":"2024-09-21T05:42:29.639590+0000","last_peered":"2024-09-21T05:42:29.639590+0000","last_clean":"2024-09-21T05:42:29.639590+0000","last_became_active":"2024-09-21T05:42:29.630284+0000","last_became_peered":"2024-09-21T05:42:29.630284+0000","last_unstale":"2024-09-21T05:42:29.639590+0000","last_undegraded":"2024-09-21T05:42:29.639590+0000","last_fullsized":"2024-09-21T05:42:29.639590+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:34:28.731480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634728+0000","last_change":"2024-09-21T05:42:29.632611+0000","last_active":"2024-09-21T05:42:29.634728+0000","last_peered":"2024-09-21T05:42:29.634728+0000","last_clean":"2024-09-21T05:42:29.634728+0000","last_became_active":"2024-09-21T05:42:29.632288+0000","last_became_peered":"2024-09-21T05:42:29.632288+0000","last_unstale":"2024-09-21T05:42:29.634728+0000","last_undegraded":"2024-09-21T05:42:29.634728+0000","last_fullsized":"2024-09-21T05:42:29.634728+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:28.860484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644678+0000","last_change":"2024-09-21T05:42:29.639226+0000","last_active":"2024-09-21T05:42:29.644678+0000","last_peered":"2024-09-21T05:42:29.644678+0000","last_clean":"2024-09-21T05:42:29.644678+0000","last_became_active":"2024-09-21T05:42:29.638913+0000","last_became_peered":"2024-09-21T05:42:29.638913+0000","last_unstale":"2024-09-21T05:42:29.644678+0000","last_undegraded":"2024-09-21T05:42:29.644678+0000","last_fullsized":"2024-09-21T05:42:29.644678+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:18:23.523706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"67'4","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.645786+0000","last_change":"2024-09-21T05:42:28.632567+0000","last_active":"2024-09-21T05:42:29.645786+0000","last_peered":"2024-09-21T05:42:29.645786+0000","last_clean":"2024-09-21T05:42:29.645786+0000","last_became_active":"2024-09-21T05:42:28.632246+0000","last_became_peered":"2024-09-21T05:42:28.632246+0000","last_unstale":"2024-09-21T05:42:29.645786+0000","last_undegraded":"2024-09-21T05:42:29.645786+0000","last_fullsized":"2024-09-21T05:42:29.645786+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:12.969812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641097+0000","last_change":"2024-09-21T05:42:29.632560+0000","last_active":"2024-09-21T05:42:29.641097+0000","last_peered":"2024-09-21T05:42:29.641097+0000","last_clean":"2024-09-21T05:42:29.641097+0000","last_became_active":"2024-09-21T05:42:29.632255+0000","last_became_peered":"2024-09-21T05:42:29.632255+0000","last_unstale":"2024-09-21T05:42:29.641097+0000","last_undegraded":"2024-09-21T05:42:29.641097+0000","last_fullsized":"2024-09-21T05:42:29.641097+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:53.576314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642657+0000","last_change":"2024-09-21T05:42:28.632770+0000","last_active":"2024-09-21T05:42:29.642657+0000","last_peered":"2024-09-21T05:42:29.642657+0000","last_clean":"2024-09-21T05:42:29.642657+0000","last_became_active":"2024-09-21T05:42:28.632521+0000","last_became_peered":"2024-09-21T05:42:28.632521+0000","last_unstale":"2024-09-21T05:42:29.642657+0000","last_undegraded":"2024-09-21T05:42:29.642657+0000","last_fullsized":"2024-09-21T05:42:29.642657+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:32.946652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644329+0000","last_change":"2024-09-21T05:42:29.642713+0000","last_active":"2024-09-21T05:42:29.644329+0000","last_peered":"2024-09-21T05:42:29.644329+0000","last_clean":"2024-09-21T05:42:29.644329+0000","last_became_active":"2024-09-21T05:42:29.642124+0000","last_became_peered":"2024-09-21T05:42:29.642124+0000","last_unstale":"2024-09-21T05:42:29.644329+0000","last_undegraded":"2024-09-21T05:42:29.644329+0000","last_fullsized":"2024-09-21T05:42:29.644329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:53.994424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"67'2","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627402+0000","last_change":"2024-09-21T05:42:28.632057+0000","last_active":"2024-09-21T05:42:30.627402+0000","last_peered":"2024-09-21T05:42:30.627402+0000","last_clean":"2024-09-21T05:42:30.627402+0000","last_became_active":"2024-09-21T05:42:28.631748+0000","last_became_peered":"2024-09-21T05:42:28.631748+0000","last_unstale":"2024-09-21T05:42:30.627402+0000","last_undegraded":"2024-09-21T05:42:30.627402+0000","last_fullsized":"2024-09-21T05:42:30.627402+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:46.096579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644679+0000","last_change":"2024-09-21T05:42:29.641244+0000","last_active":"2024-09-21T05:42:29.644679+0000","last_peered":"2024-09-21T05:42:29.644679+0000","last_clean":"2024-09-21T05:42:29.644679+0000","last_became_active":"2024-09-21T05:42:29.640893+0000","last_became_peered":"2024-09-21T05:42:29.640893+0000","last_unstale":"2024-09-21T05:42:29.644679+0000","last_undegraded":"2024-09-21T05:42:29.644679+0000","last_fullsized":"2024-09-21T05:42:29.644679+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:19.919362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638363+0000","last_change":"2024-09-21T05:42:28.630169+0000","last_active":"2024-09-21T05:42:29.638363+0000","last_peered":"2024-09-21T05:42:29.638363+0000","last_clean":"2024-09-21T05:42:29.638363+0000","last_became_active":"2024-09-21T05:42:28.629794+0000","last_became_peered":"2024-09-21T05:42:28.629794+0000","last_unstale":"2024-09-21T05:42:29.638363+0000","last_undegraded":"2024-09-21T05:42:29.638363+0000","last_fullsized":"2024-09-21T05:42:29.638363+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:49.003565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641092+0000","last_change":"2024-09-21T05:42:29.638130+0000","last_active":"2024-09-21T05:42:29.641092+0000","last_peered":"2024-09-21T05:42:29.641092+0000","last_clean":"2024-09-21T05:42:29.641092+0000","last_became_active":"2024-09-21T05:42:29.637475+0000","last_became_peered":"2024-09-21T05:42:29.637475+0000","last_unstale":"2024-09-21T05:42:29.641092+0000","last_undegraded":"2024-09-21T05:42:29.641092+0000","last_fullsized":"2024-09-21T05:42:29.641092+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:23:33.948700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638311+0000","last_change":"2024-09-21T05:42:28.622204+0000","last_active":"2024-09-21T05:42:29.638311+0000","last_peered":"2024-09-21T05:42:29.638311+0000","last_clean":"2024-09-21T05:42:29.638311+0000","last_became_active":"2024-09-21T05:42:28.621855+0000","last_became_peered":"2024-09-21T05:42:28.621855+0000","last_unstale":"2024-09-21T05:42:29.638311+0000","last_undegraded":"2024-09-21T05:42:29.638311+0000","last_fullsized":"2024-09-21T05:42:29.638311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:33.437219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627736+0000","last_change":"2024-09-21T05:42:29.644685+0000","last_active":"2024-09-21T05:42:30.627736+0000","last_peered":"2024-09-21T05:42:30.627736+0000","last_clean":"2024-09-21T05:42:30.627736+0000","last_became_active":"2024-09-21T05:42:29.644380+0000","last_became_peered":"2024-09-21T05:42:29.644380+0000","last_unstale":"2024-09-21T05:42:30.627736+0000","last_undegraded":"2024-09-21T05:42:30.627736+0000","last_fullsized":"2024-09-21T05:42:30.627736+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:24.712694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.632612+0000","last_change":"2024-09-21T05:42:28.629832+0000","last_active":"2024-09-21T05:42:29.632612+0000","last_peered":"2024-09-21T05:42:29.632612+0000","last_clean":"2024-09-21T05:42:29.632612+0000","last_became_active":"2024-09-21T05:42:28.629477+0000","last_became_peered":"2024-09-21T05:42:28.629477+0000","last_unstale":"2024-09-21T05:42:29.632612+0000","last_undegraded":"2024-09-21T05:42:29.632612+0000","last_fullsized":"2024-09-21T05:42:29.632612+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:05.070095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640675+0000","last_change":"2024-09-21T05:42:29.631637+0000","last_active":"2024-09-21T05:42:29.640675+0000","last_peered":"2024-09-21T05:42:29.640675+0000","last_clean":"2024-09-21T05:42:29.640675+0000","last_became_active":"2024-09-21T05:42:29.631227+0000","last_became_peered":"2024-09-21T05:42:29.631227+0000","last_unstale":"2024-09-21T05:42:29.640675+0000","last_undegraded":"2024-09-21T05:42:29.640675+0000","last_fullsized":"2024-09-21T05:42:29.640675+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:49.352029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.612080+0000","last_change":"2024-09-21T05:42:28.622653+0000","last_active":"2024-09-21T05:42:29.612080+0000","last_peered":"2024-09-21T05:42:29.612080+0000","last_clean":"2024-09-21T05:42:29.612080+0000","last_became_active":"2024-09-21T05:42:28.622339+0000","last_became_peered":"2024-09-21T05:42:28.622339+0000","last_unstale":"2024-09-21T05:42:29.612080+0000","last_undegraded":"2024-09-21T05:42:29.612080+0000","last_fullsized":"2024-09-21T05:42:29.612080+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:11:36.292050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639235+0000","last_change":"2024-09-21T05:42:29.638132+0000","last_active":"2024-09-21T05:42:29.639235+0000","last_peered":"2024-09-21T05:42:29.639235+0000","last_clean":"2024-09-21T05:42:29.639235+0000","last_became_active":"2024-09-21T05:42:29.637819+0000","last_became_peered":"2024-09-21T05:42:29.637819+0000","last_unstale":"2024-09-21T05:42:29.639235+0000","last_undegraded":"2024-09-21T05:42:29.639235+0000","last_fullsized":"2024-09-21T05:42:29.639235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:02.013378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.618538+0000","last_change":"2024-09-21T05:42:28.625390+0000","last_active":"2024-09-21T05:42:29.618538+0000","last_peered":"2024-09-21T05:42:29.618538+0000","last_clean":"2024-09-21T05:42:29.618538+0000","last_became_active":"2024-09-21T05:42:28.625079+0000","last_became_peered":"2024-09-21T05:42:28.625079+0000","last_unstale":"2024-09-21T05:42:29.618538+0000","last_undegraded":"2024-09-21T05:42:29.618538+0000","last_fullsized":"2024-09-21T05:42:29.618538+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:55.163798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.643555+0000","last_change":"2024-09-21T05:42:29.642469+0000","last_active":"2024-09-21T05:42:29.643555+0000","last_peered":"2024-09-21T05:42:29.643555+0000","last_clean":"2024-09-21T05:42:29.643555+0000","last_became_active":"2024-09-21T05:42:29.642137+0000","last_became_peered":"2024-09-21T05:42:29.642137+0000","last_unstale":"2024-09-21T05:42:29.643555+0000","last_undegraded":"2024-09-21T05:42:29.643555+0000","last_fullsized":"2024-09-21T05:42:29.643555+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:34.310526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627293+0000","last_change":"2024-09-21T05:42:28.624051+0000","last_active":"2024-09-21T05:42:30.627293+0000","last_peered":"2024-09-21T05:42:30.627293+0000","last_clean":"2024-09-21T05:42:30.627293+0000","last_became_active":"2024-09-21T05:42:28.623712+0000","last_became_peered":"2024-09-21T05:42:28.623712+0000","last_unstale":"2024-09-21T05:42:30.627293+0000","last_undegraded":"2024-09-21T05:42:30.627293+0000","last_fullsized":"2024-09-21T05:42:30.627293+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:33:57.106958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.648344+0000","last_change":"2024-09-21T05:42:29.646396+0000","last_active":"2024-09-21T05:42:29.648344+0000","last_peered":"2024-09-21T05:42:29.648344+0000","last_clean":"2024-09-21T05:42:29.648344+0000","last_became_active":"2024-09-21T05:42:29.645976+0000","last_became_peered":"2024-09-21T05:42:29.645976+0000","last_unstale":"2024-09-21T05:42:29.648344+0000","last_undegraded":"2024-09-21T05:42:29.648344+0000","last_fullsized":"2024-09-21T05:42:29.648344+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:46.760816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.619816+0000","last_change":"2024-09-21T05:42:28.633366+0000","last_active":"2024-09-21T05:42:29.619816+0000","last_peered":"2024-09-21T05:42:29.619816+0000","last_clean":"2024-09-21T05:42:29.619816+0000","last_became_active":"2024-09-21T05:42:28.630483+0000","last_became_peered":"2024-09-21T05:42:28.630483+0000","last_unstale":"2024-09-21T05:42:29.619816+0000","last_undegraded":"2024-09-21T05:42:29.619816+0000","last_fullsized":"2024-09-21T05:42:29.619816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:21.714458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.633693+0000","last_change":"2024-09-21T05:42:29.631556+0000","last_active":"2024-09-21T05:42:29.633693+0000","last_peered":"2024-09-21T05:42:29.633693+0000","last_clean":"2024-09-21T05:42:29.633693+0000","last_became_active":"2024-09-21T05:42:29.631206+0000","last_became_peered":"2024-09-21T05:42:29.631206+0000","last_unstale":"2024-09-21T05:42:29.633693+0000","last_undegraded":"2024-09-21T05:42:29.633693+0000","last_fullsized":"2024-09-21T05:42:29.633693+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:05:49.346493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"67'3","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.633927+0000","last_change":"2024-09-21T05:42:28.629044+0000","last_active":"2024-09-21T05:42:29.633927+0000","last_peered":"2024-09-21T05:42:29.633927+0000","last_clean":"2024-09-21T05:42:29.633927+0000","last_became_active":"2024-09-21T05:42:28.628656+0000","last_became_peered":"2024-09-21T05:42:28.628656+0000","last_unstale":"2024-09-21T05:42:29.633927+0000","last_undegraded":"2024-09-21T05:42:29.633927+0000","last_fullsized":"2024-09-21T05:42:29.633927+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:03:04.590995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642071+0000","last_change":"2024-09-21T05:42:29.637374+0000","last_active":"2024-09-21T05:42:29.642071+0000","last_peered":"2024-09-21T05:42:29.642071+0000","last_clean":"2024-09-21T05:42:29.642071+0000","last_became_active":"2024-09-21T05:42:29.637061+0000","last_became_peered":"2024-09-21T05:42:29.637061+0000","last_unstale":"2024-09-21T05:42:29.642071+0000","last_undegraded":"2024-09-21T05:42:29.642071+0000","last_fullsized":"2024-09-21T05:42:29.642071+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:17.710485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"67'6","reported_seq":19,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.645332+0000","last_change":"2024-09-21T05:42:28.627013+0000","last_active":"2024-09-21T05:42:29.645332+0000","last_peered":"2024-09-21T05:42:29.645332+0000","last_clean":"2024-09-21T05:42:29.645332+0000","last_became_active":"2024-09-21T05:42:28.626276+0000","last_became_peered":"2024-09-21T05:42:28.626276+0000","last_unstale":"2024-09-21T05:42:29.645332+0000","last_undegraded":"2024-09-21T05:42:29.645332+0000","last_fullsized":"2024-09-21T05:42:29.645332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:19:04.043487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.649369+0000","last_change":"2024-09-21T05:42:29.647994+0000","last_active":"2024-09-21T05:42:29.649369+0000","last_peered":"2024-09-21T05:42:29.649369+0000","last_clean":"2024-09-21T05:42:29.649369+0000","last_became_active":"2024-09-21T05:42:29.647650+0000","last_became_peered":"2024-09-21T05:42:29.647650+0000","last_unstale":"2024-09-21T05:42:29.649369+0000","last_undegraded":"2024-09-21T05:42:29.649369+0000","last_fullsized":"2024-09-21T05:42:29.649369+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:26:47.133527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.647648+0000","last_change":"2024-09-21T05:42:28.630356+0000","last_active":"2024-09-21T05:42:29.647648+0000","last_peered":"2024-09-21T05:42:29.647648+0000","last_clean":"2024-09-21T05:42:29.647648+0000","last_became_active":"2024-09-21T05:42:28.630008+0000","last_became_peered":"2024-09-21T05:42:28.630008+0000","last_unstale":"2024-09-21T05:42:29.647648+0000","last_undegraded":"2024-09-21T05:42:29.647648+0000","last_fullsized":"2024-09-21T05:42:29.647648+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:47:35.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.637966+0000","last_change":"2024-09-21T05:42:29.635920+0000","last_active":"2024-09-21T05:42:29.637966+0000","last_peered":"2024-09-21T05:42:29.637966+0000","last_clean":"2024-09-21T05:42:29.637966+0000","last_became_active":"2024-09-21T05:42:29.635596+0000","last_became_peered":"2024-09-21T05:42:29.635596+0000","last_unstale":"2024-09-21T05:42:29.637966+0000","last_undegraded":"2024-09-21T05:42:29.637966+0000","last_fullsized":"2024-09-21T05:42:29.637966+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:23.613737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.619061+0000","last_change":"2024-09-21T05:42:28.626898+0000","last_active":"2024-09-21T05:42:29.619061+0000","last_peered":"2024-09-21T05:42:29.619061+0000","last_clean":"2024-09-21T05:42:29.619061+0000","last_became_active":"2024-09-21T05:42:28.626541+0000","last_became_peered":"2024-09-21T05:42:28.626541+0000","last_unstale":"2024-09-21T05:42:29.619061+0000","last_undegraded":"2024-09-21T05:42:29.619061+0000","last_fullsized":"2024-09-21T05:42:29.619061+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:51.649406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.638538+0000","last_change":"2024-09-21T05:42:29.634828+0000","last_active":"2024-09-21T05:42:29.638538+0000","last_peered":"2024-09-21T05:42:29.638538+0000","last_clean":"2024-09-21T05:42:29.638538+0000","last_became_active":"2024-09-21T05:42:29.634545+0000","last_became_peered":"2024-09-21T05:42:29.634545+0000","last_unstale":"2024-09-21T05:42:29.638538+0000","last_undegraded":"2024-09-21T05:42:29.638538+0000","last_fullsized":"2024-09-21T05:42:29.638538+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:46:43.413048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.620493+0000","last_change":"2024-09-21T05:42:28.631613+0000","last_active":"2024-09-21T05:42:29.620493+0000","last_peered":"2024-09-21T05:42:29.620493+0000","last_clean":"2024-09-21T05:42:29.620493+0000","last_became_active":"2024-09-21T05:42:28.631238+0000","last_became_peered":"2024-09-21T05:42:28.631238+0000","last_unstale":"2024-09-21T05:42:29.620493+0000","last_undegraded":"2024-09-21T05:42:29.620493+0000","last_fullsized":"2024-09-21T05:42:29.620493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:51.188981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.641301+0000","last_change":"2024-09-21T05:42:29.631762+0000","last_active":"2024-09-21T05:42:29.641301+0000","last_peered":"2024-09-21T05:42:29.641301+0000","last_clean":"2024-09-21T05:42:29.641301+0000","last_became_active":"2024-09-21T05:42:29.631360+0000","last_became_peered":"2024-09-21T05:42:29.631360+0000","last_unstale":"2024-09-21T05:42:29.641301+0000","last_undegraded":"2024-09-21T05:42:29.641301+0000","last_fullsized":"2024-09-21T05:42:29.641301+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:17.095014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"67'8","reported_seq":23,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626283+0000","last_change":"2024-09-21T05:42:28.626976+0000","last_active":"2024-09-21T05:42:30.626283+0000","last_peered":"2024-09-21T05:42:30.626283+0000","last_clean":"2024-09-21T05:42:30.626283+0000","last_became_active":"2024-09-21T05:42:28.626680+0000","last_became_peered":"2024-09-21T05:42:28.626680+0000","last_unstale":"2024-09-21T05:42:30.626283+0000","last_undegraded":"2024-09-21T05:42:30.626283+0000","last_fullsized":"2024-09-21T05:42:30.626283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:56.569120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627792+0000","last_change":"2024-09-21T05:42:29.618303+0000","last_active":"2024-09-21T05:42:30.627792+0000","last_peered":"2024-09-21T05:42:30.627792+0000","last_clean":"2024-09-21T05:42:30.627792+0000","last_became_active":"2024-09-21T05:42:29.617966+0000","last_became_peered":"2024-09-21T05:42:29.617966+0000","last_unstale":"2024-09-21T05:42:30.627792+0000","last_undegraded":"2024-09-21T05:42:30.627792+0000","last_fullsized":"2024-09-21T05:42:30.627792+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:48:51.658406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634934+0000","last_change":"2024-09-21T05:42:28.630288+0000","last_active":"2024-09-21T05:42:29.634934+0000","last_peered":"2024-09-21T05:42:29.634934+0000","last_clean":"2024-09-21T05:42:29.634934+0000","last_became_active":"2024-09-21T05:42:28.629950+0000","last_became_peered":"2024-09-21T05:42:28.629950+0000","last_unstale":"2024-09-21T05:42:29.634934+0000","last_undegraded":"2024-09-21T05:42:29.634934+0000","last_fullsized":"2024-09-21T05:42:29.634934+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:06.616255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.646943+0000","last_change":"2024-09-21T05:42:29.643672+0000","last_active":"2024-09-21T05:42:29.646943+0000","last_peered":"2024-09-21T05:42:29.646943+0000","last_clean":"2024-09-21T05:42:29.646943+0000","last_became_active":"2024-09-21T05:42:29.643366+0000","last_became_peered":"2024-09-21T05:42:29.643366+0000","last_unstale":"2024-09-21T05:42:29.646943+0000","last_undegraded":"2024-09-21T05:42:29.646943+0000","last_fullsized":"2024-09-21T05:42:29.646943+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:56.026952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"67'2","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.627913+0000","last_change":"2024-09-21T05:42:28.633148+0000","last_active":"2024-09-21T05:42:30.627913+0000","last_peered":"2024-09-21T05:42:30.627913+0000","last_clean":"2024-09-21T05:42:30.627913+0000","last_became_active":"2024-09-21T05:42:28.632774+0000","last_became_peered":"2024-09-21T05:42:28.632774+0000","last_unstale":"2024-09-21T05:42:30.627913+0000","last_undegraded":"2024-09-21T05:42:30.627913+0000","last_fullsized":"2024-09-21T05:42:30.627913+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:07.822576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635465+0000","last_change":"2024-09-21T05:42:29.629257+0000","last_active":"2024-09-21T05:42:29.635465+0000","last_peered":"2024-09-21T05:42:29.635465+0000","last_clean":"2024-09-21T05:42:29.635465+0000","last_became_active":"2024-09-21T05:42:29.628858+0000","last_became_peered":"2024-09-21T05:42:29.628858+0000","last_unstale":"2024-09-21T05:42:29.635465+0000","last_undegraded":"2024-09-21T05:42:29.635465+0000","last_fullsized":"2024-09-21T05:42:29.635465+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:01.065279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"67'1","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.643076+0000","last_change":"2024-09-21T05:42:28.633926+0000","last_active":"2024-09-21T05:42:29.643076+0000","last_peered":"2024-09-21T05:42:29.643076+0000","last_clean":"2024-09-21T05:42:29.643076+0000","last_became_active":"2024-09-21T05:42:28.630009+0000","last_became_peered":"2024-09-21T05:42:28.630009+0000","last_unstale":"2024-09-21T05:42:29.643076+0000","last_undegraded":"2024-09-21T05:42:29.643076+0000","last_fullsized":"2024-09-21T05:42:29.643076+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:44.697372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.642099+0000","last_change":"2024-09-21T05:42:29.639427+0000","last_active":"2024-09-21T05:42:29.642099+0000","last_peered":"2024-09-21T05:42:29.642099+0000","last_clean":"2024-09-21T05:42:29.642099+0000","last_became_active":"2024-09-21T05:42:29.639111+0000","last_became_peered":"2024-09-21T05:42:29.639111+0000","last_unstale":"2024-09-21T05:42:29.642099+0000","last_undegraded":"2024-09-21T05:42:29.642099+0000","last_fullsized":"2024-09-21T05:42:29.642099+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:21:24.600827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629081+0000","last_change":"2024-09-21T05:42:28.630315+0000","last_active":"2024-09-21T05:42:29.629081+0000","last_peered":"2024-09-21T05:42:29.629081+0000","last_clean":"2024-09-21T05:42:29.629081+0000","last_became_active":"2024-09-21T05:42:28.629991+0000","last_became_peered":"2024-09-21T05:42:28.629991+0000","last_unstale":"2024-09-21T05:42:29.629081+0000","last_undegraded":"2024-09-21T05:42:29.629081+0000","last_fullsized":"2024-09-21T05:42:29.629081+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:59.458821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629563+0000","last_change":"2024-09-21T05:42:29.623748+0000","last_active":"2024-09-21T05:42:29.629563+0000","last_peered":"2024-09-21T05:42:29.629563+0000","last_clean":"2024-09-21T05:42:29.629563+0000","last_became_active":"2024-09-21T05:42:29.623409+0000","last_became_peered":"2024-09-21T05:42:29.623409+0000","last_unstale":"2024-09-21T05:42:29.629563+0000","last_undegraded":"2024-09-21T05:42:29.629563+0000","last_fullsized":"2024-09-21T05:42:29.629563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:21.015474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.619328+0000","last_change":"2024-09-21T05:42:28.634087+0000","last_active":"2024-09-21T05:42:29.619328+0000","last_peered":"2024-09-21T05:42:29.619328+0000","last_clean":"2024-09-21T05:42:29.619328+0000","last_became_active":"2024-09-21T05:42:28.630918+0000","last_became_peered":"2024-09-21T05:42:28.630918+0000","last_unstale":"2024-09-21T05:42:29.619328+0000","last_undegraded":"2024-09-21T05:42:29.619328+0000","last_fullsized":"2024-09-21T05:42:29.619328+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:00.038687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634720+0000","last_change":"2024-09-21T05:42:29.633013+0000","last_active":"2024-09-21T05:42:29.634720+0000","last_peered":"2024-09-21T05:42:29.634720+0000","last_clean":"2024-09-21T05:42:29.634720+0000","last_became_active":"2024-09-21T05:42:29.632694+0000","last_became_peered":"2024-09-21T05:42:29.632694+0000","last_unstale":"2024-09-21T05:42:29.634720+0000","last_undegraded":"2024-09-21T05:42:29.634720+0000","last_fullsized":"2024-09-21T05:42:29.634720+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:45.348405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.620731+0000","last_change":"2024-09-21T05:42:28.633790+0000","last_active":"2024-09-21T05:42:29.620731+0000","last_peered":"2024-09-21T05:42:29.620731+0000","last_clean":"2024-09-21T05:42:29.620731+0000","last_became_active":"2024-09-21T05:42:28.633462+0000","last_became_peered":"2024-09-21T05:42:28.633462+0000","last_unstale":"2024-09-21T05:42:29.620731+0000","last_undegraded":"2024-09-21T05:42:29.620731+0000","last_fullsized":"2024-09-21T05:42:29.620731+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:25.645933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635650+0000","last_change":"2024-09-21T05:42:29.634436+0000","last_active":"2024-09-21T05:42:29.635650+0000","last_peered":"2024-09-21T05:42:29.635650+0000","last_clean":"2024-09-21T05:42:29.635650+0000","last_became_active":"2024-09-21T05:42:29.634114+0000","last_became_peered":"2024-09-21T05:42:29.634114+0000","last_unstale":"2024-09-21T05:42:29.635650+0000","last_undegraded":"2024-09-21T05:42:29.635650+0000","last_fullsized":"2024-09-21T05:42:29.635650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:40.179403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.621747+0000","last_change":"2024-09-21T05:42:28.632897+0000","last_active":"2024-09-21T05:42:29.621747+0000","last_peered":"2024-09-21T05:42:29.621747+0000","last_clean":"2024-09-21T05:42:29.621747+0000","last_became_active":"2024-09-21T05:42:28.632577+0000","last_became_peered":"2024-09-21T05:42:28.632577+0000","last_unstale":"2024-09-21T05:42:29.621747+0000","last_undegraded":"2024-09-21T05:42:29.621747+0000","last_fullsized":"2024-09-21T05:42:29.621747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:38.860781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.637044+0000","last_change":"2024-09-21T05:42:29.630369+0000","last_active":"2024-09-21T05:42:29.637044+0000","last_peered":"2024-09-21T05:42:29.637044+0000","last_clean":"2024-09-21T05:42:29.637044+0000","last_became_active":"2024-09-21T05:42:29.630035+0000","last_became_peered":"2024-09-21T05:42:29.630035+0000","last_unstale":"2024-09-21T05:42:29.637044+0000","last_undegraded":"2024-09-21T05:42:29.637044+0000","last_fullsized":"2024-09-21T05:42:29.637044+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:42:49.494447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634913+0000","last_change":"2024-09-21T05:42:28.622372+0000","last_active":"2024-09-21T05:42:29.634913+0000","last_peered":"2024-09-21T05:42:29.634913+0000","last_clean":"2024-09-21T05:42:29.634913+0000","last_became_active":"2024-09-21T05:42:28.621989+0000","last_became_peered":"2024-09-21T05:42:28.621989+0000","last_unstale":"2024-09-21T05:42:29.634913+0000","last_undegraded":"2024-09-21T05:42:29.634913+0000","last_fullsized":"2024-09-21T05:42:29.634913+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:39:04.969519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640888+0000","last_change":"2024-09-21T05:42:29.636758+0000","last_active":"2024-09-21T05:42:29.640888+0000","last_peered":"2024-09-21T05:42:29.640888+0000","last_clean":"2024-09-21T05:42:29.640888+0000","last_became_active":"2024-09-21T05:42:29.636424+0000","last_became_peered":"2024-09-21T05:42:29.636424+0000","last_unstale":"2024-09-21T05:42:29.640888+0000","last_undegraded":"2024-09-21T05:42:29.640888+0000","last_fullsized":"2024-09-21T05:42:29.640888+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:39.862829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.621745+0000","last_change":"2024-09-21T05:42:28.631698+0000","last_active":"2024-09-21T05:42:29.621745+0000","last_peered":"2024-09-21T05:42:29.621745+0000","last_clean":"2024-09-21T05:42:29.621745+0000","last_became_active":"2024-09-21T05:42:28.631261+0000","last_became_peered":"2024-09-21T05:42:28.631261+0000","last_unstale":"2024-09-21T05:42:29.621745+0000","last_undegraded":"2024-09-21T05:42:29.621745+0000","last_fullsized":"2024-09-21T05:42:29.621745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:37.452860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.644642+0000","last_change":"2024-09-21T05:42:29.638689+0000","last_active":"2024-09-21T05:42:29.644642+0000","last_peered":"2024-09-21T05:42:29.644642+0000","last_clean":"2024-09-21T05:42:29.644642+0000","last_became_active":"2024-09-21T05:42:29.638281+0000","last_became_peered":"2024-09-21T05:42:29.638281+0000","last_unstale":"2024-09-21T05:42:29.644642+0000","last_undegraded":"2024-09-21T05:42:29.644642+0000","last_fullsized":"2024-09-21T05:42:29.644642+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:52:17.246291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.622583+0000","last_change":"2024-09-21T05:42:28.629767+0000","last_active":"2024-09-21T05:42:29.622583+0000","last_peered":"2024-09-21T05:42:29.622583+0000","last_clean":"2024-09-21T05:42:29.622583+0000","last_became_active":"2024-09-21T05:42:28.629379+0000","last_became_peered":"2024-09-21T05:42:28.629379+0000","last_unstale":"2024-09-21T05:42:29.622583+0000","last_undegraded":"2024-09-21T05:42:29.622583+0000","last_fullsized":"2024-09-21T05:42:29.622583+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:32.125509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.646026+0000","last_change":"2024-09-21T05:42:29.644444+0000","last_active":"2024-09-21T05:42:29.646026+0000","last_peered":"2024-09-21T05:42:29.646026+0000","last_clean":"2024-09-21T05:42:29.646026+0000","last_became_active":"2024-09-21T05:42:29.644127+0000","last_became_peered":"2024-09-21T05:42:29.644127+0000","last_unstale":"2024-09-21T05:42:29.646026+0000","last_undegraded":"2024-09-21T05:42:29.646026+0000","last_fullsized":"2024-09-21T05:42:29.646026+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:45.068806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.621666+0000","last_change":"2024-09-21T05:42:28.630093+0000","last_active":"2024-09-21T05:42:29.621666+0000","last_peered":"2024-09-21T05:42:29.621666+0000","last_clean":"2024-09-21T05:42:29.621666+0000","last_became_active":"2024-09-21T05:42:28.629759+0000","last_became_peered":"2024-09-21T05:42:28.629759+0000","last_unstale":"2024-09-21T05:42:29.621666+0000","last_undegraded":"2024-09-21T05:42:29.621666+0000","last_fullsized":"2024-09-21T05:42:29.621666+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:22.611689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.640838+0000","last_change":"2024-09-21T05:42:29.631703+0000","last_active":"2024-09-21T05:42:29.640838+0000","last_peered":"2024-09-21T05:42:29.640838+0000","last_clean":"2024-09-21T05:42:29.640838+0000","last_became_active":"2024-09-21T05:42:29.631373+0000","last_became_peered":"2024-09-21T05:42:29.631373+0000","last_unstale":"2024-09-21T05:42:29.640838+0000","last_undegraded":"2024-09-21T05:42:29.640838+0000","last_fullsized":"2024-09-21T05:42:29.640838+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:44.134131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.623472+0000","last_change":"2024-09-21T05:42:28.629732+0000","last_active":"2024-09-21T05:42:29.623472+0000","last_peered":"2024-09-21T05:42:29.623472+0000","last_clean":"2024-09-21T05:42:29.623472+0000","last_became_active":"2024-09-21T05:42:28.629349+0000","last_became_peered":"2024-09-21T05:42:28.629349+0000","last_unstale":"2024-09-21T05:42:29.623472+0000","last_undegraded":"2024-09-21T05:42:29.623472+0000","last_fullsized":"2024-09-21T05:42:29.623472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:12.226276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.648660+0000","last_change":"2024-09-21T05:42:29.646997+0000","last_active":"2024-09-21T05:42:29.648660+0000","last_peered":"2024-09-21T05:42:29.648660+0000","last_clean":"2024-09-21T05:42:29.648660+0000","last_became_active":"2024-09-21T05:42:29.646464+0000","last_became_peered":"2024-09-21T05:42:29.646464+0000","last_unstale":"2024-09-21T05:42:29.648660+0000","last_undegraded":"2024-09-21T05:42:29.648660+0000","last_fullsized":"2024-09-21T05:42:29.648660+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:39.925217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.612889+0000","last_change":"2024-09-21T05:42:28.621403+0000","last_active":"2024-09-21T05:42:29.612889+0000","last_peered":"2024-09-21T05:42:29.612889+0000","last_clean":"2024-09-21T05:42:29.612889+0000","last_became_active":"2024-09-21T05:42:28.621065+0000","last_became_peered":"2024-09-21T05:42:28.621065+0000","last_unstale":"2024-09-21T05:42:29.612889+0000","last_undegraded":"2024-09-21T05:42:29.612889+0000","last_fullsized":"2024-09-21T05:42:29.612889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:20.775273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635297+0000","last_change":"2024-09-21T05:42:29.630366+0000","last_active":"2024-09-21T05:42:29.635297+0000","last_peered":"2024-09-21T05:42:29.635297+0000","last_clean":"2024-09-21T05:42:29.635297+0000","last_became_active":"2024-09-21T05:42:29.627873+0000","last_became_peered":"2024-09-21T05:42:29.627873+0000","last_unstale":"2024-09-21T05:42:29.635297+0000","last_undegraded":"2024-09-21T05:42:29.635297+0000","last_fullsized":"2024-09-21T05:42:29.635297+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:00:55.537195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-21T05:42:30.626503+0000","last_change":"2024-09-21T05:42:28.628131+0000","last_active":"2024-09-21T05:42:30.626503+0000","last_peered":"2024-09-21T05:42:30.626503+0000","last_clean":"2024-09-21T05:42:30.626503+0000","last_became_active":"2024-09-21T05:42:28.627760+0000","last_became_peered":"2024-09-21T05:42:28.627760+0000","last_unstale":"2024-09-21T05:42:30.626503+0000","last_undegraded":"2024-09-21T05:42:30.626503+0000","last_fullsized":"2024-09-21T05:42:30.626503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:32:12.334459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.634559+0000","last_change":"2024-09-21T05:42:29.633371+0000","last_active":"2024-09-21T05:42:29.634559+0000","last_peered":"2024-09-21T05:42:29.634559+0000","last_clean":"2024-09-21T05:42:29.634559+0000","last_became_active":"2024-09-21T05:42:29.633005+0000","last_became_peered":"2024-09-21T05:42:29.633005+0000","last_unstale":"2024-09-21T05:42:29.634559+0000","last_undegraded":"2024-09-21T05:42:29.634559+0000","last_fullsized":"2024-09-21T05:42:29.634559+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:06.875521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.618156+0000","last_change":"2024-09-21T05:42:28.630642+0000","last_active":"2024-09-21T05:42:29.618156+0000","last_peered":"2024-09-21T05:42:29.618156+0000","last_clean":"2024-09-21T05:42:29.618156+0000","last_became_active":"2024-09-21T05:42:28.630243+0000","last_became_peered":"2024-09-21T05:42:28.630243+0000","last_unstale":"2024-09-21T05:42:29.618156+0000","last_undegraded":"2024-09-21T05:42:29.618156+0000","last_fullsized":"2024-09-21T05:42:29.618156+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:44:16.619859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.635824+0000","last_change":"2024-09-21T05:42:29.633358+0000","last_active":"2024-09-21T05:42:29.635824+0000","last_peered":"2024-09-21T05:42:29.635824+0000","last_clean":"2024-09-21T05:42:29.635824+0000","last_became_active":"2024-09-21T05:42:29.632181+0000","last_became_peered":"2024-09-21T05:42:29.632181+0000","last_unstale":"2024-09-21T05:42:29.635824+0000","last_undegraded":"2024-09-21T05:42:29.635824+0000","last_fullsized":"2024-09-21T05:42:29.635824+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:28.601316+0000","last_clean_scrub_stamp":"2024-09-21T05:42:28.601316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:53.649003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"67'2","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.639838+0000","last_change":"2024-09-21T05:42:28.630733+0000","last_active":"2024-09-21T05:42:29.639838+0000","last_peered":"2024-09-21T05:42:29.639838+0000","last_clean":"2024-09-21T05:42:29.639838+0000","last_became_active":"2024-09-21T05:42:28.630098+0000","last_became_peered":"2024-09-21T05:42:28.630098+0000","last_unstale":"2024-09-21T05:42:29.639838+0000","last_undegraded":"2024-09-21T05:42:29.639838+0000","last_fullsized":"2024-09-21T05:42:29.639838+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:42:27.599268+0000","last_clean_scrub_stamp":"2024-09-21T05:42:27.599268+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:26.343187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":191,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-21T05:42:29.629268+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:42:29.629268+0000","last_peered":"2024-09-21T05:42:29.629268+0000","last_clean":"2024-09-21T05:42:29.629268+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:42:29.629268+0000","last_undegraded":"2024-09-21T05:42:29.629268+0000","last_fullsized":"2024-09-21T05:42:29.629268+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509484,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32568,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339272,"statfs":{"total":96636764160,"available":96603414528,"internally_reserved":0,"allocated":1409024,"data_stored":928462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":811008,"data_stored":337548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":337570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":337568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542186,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31992,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339848,"statfs":{"total":96636764160,"available":96604004352,"internally_reserved":0,"allocated":815104,"data_stored":337590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31996,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339844,"statfs":{"total":96636764160,"available":96604000256,"internally_reserved":0,"allocated":819200,"data_stored":339090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36128,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335712,"statfs":{"total":96636764160,"available":96599769088,"internally_reserved":0,"allocated":794624,"data_stored":330703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1388544,"data_stored":921071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:42:34.070 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:42:34.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:42:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 -- 172.21.15.22:0/1709643713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406e170 msgr2=0x7fbe140b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 --2- 172.21.15.22:0/1709643713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406e170 0x7fbe140b3540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009830 tx=0x7fbe1002a480 comp rx=0 tx=0).stop 2024-09-21T05:42:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 -- 172.21.15.22:0/1709643713 shutdown_connections 2024-09-21T05:42:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 --2- 172.21.15.22:0/1709643713 >> v2:172.21.15.134:3300/0 conn(0x7fbe140ceb40 0x7fbe140cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 --2- 172.21.15.22:0/1709643713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406e170 0x7fbe140b3540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 -- 172.21.15.22:0/1709643713 >> 172.21.15.22:0/1709643713 conn(0x7fbe1408bfc0 msgr2=0x7fbe1409d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:34.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 -- 172.21.15.22:0/1709643713 shutdown_connections 2024-09-21T05:42:34.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.230+0000 7fbe1c550640 1 -- 172.21.15.22:0/1709643713 wait complete. 2024-09-21T05:42:34.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.232+0000 7fbe1c550640 1 Processor -- start 2024-09-21T05:42:34.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.232+0000 7fbe1c550640 1 -- start start 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1c550640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406f4b0 0x7fbe14151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1c550640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 0x7fbe141511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1c550640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe141544c0 0x7fbe140cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1c550640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140d0e60 con 0x7fbe140ceb40 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1c550640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbe140d0ce0 con 0x7fbe141544c0 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1c550640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbe140d0fe0 con 0x7fbe1406f4b0 2024-09-21T05:42:34.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 0x7fbe141511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 0x7fbe141511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34528/0 (socket says 172.21.15.22:34528) 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 -- 172.21.15.22:0/2815008890 learned_addr learned my addr 172.21.15.22:0/2815008890 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1a2ec640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406f4b0 0x7fbe14151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe1aaed640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe141544c0 0x7fbe140cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 -- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406f4b0 msgr2=0x7fbe14151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406f4b0 0x7fbe14151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 -- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe141544c0 msgr2=0x7fbe140cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.233+0000 7fbe19aeb640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe141544c0 0x7fbe140cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe19aeb640 1 -- 172.21.15.22:0/2815008890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe140cc7e0 con 0x7fbe140ceb40 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe19aeb640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 0x7fbe141511a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbe04007fd0 tx=0x7fbe0400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04011790 con 0x7fbe140ceb40 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe04017070 con 0x7fbe140ceb40 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe140ccac0 con 0x7fbe140ceb40 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04011d00 con 0x7fbe140ceb40 2024-09-21T05:42:34.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.234+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe140cd080 con 0x7fbe140ceb40 2024-09-21T05:42:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.235+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe140b9d60 con 0x7fbe140ceb40 2024-09-21T05:42:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.236+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbe0401c460 con 0x7fbe140ceb40 2024-09-21T05:42:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.236+0000 7fbe0affd640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdf8078bc0 0x7fbdf807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.236+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fbe0409aa40 con 0x7fbe140ceb40 2024-09-21T05:42:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.237+0000 7fbe1a2ec640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdf8078bc0 0x7fbdf807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.237+0000 7fbe1a2ec640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdf8078bc0 0x7fbdf807b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe10034000 tx=0x7fbe10004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:34.243 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.241+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbe140b9d60 con 0x7fbe140ceb40 2024-09-21T05:42:34.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.501+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbe14154a20 con 0x7fbe140ceb40 2024-09-21T05:42:34.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.504+0000 7fbe0affd640 1 -- 172.21.15.22:0/2815008890 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7fbe04062e90 con 0x7fbe140ceb40 2024-09-21T05:42:34.506 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:42:34.506 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:42:34.507 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:42:34.507 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:42:34.507 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:42:34.507 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:42:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdf8078bc0 msgr2=0x7fbdf807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdf8078bc0 0x7fbdf807b0d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbe10034000 tx=0x7fbe10004730 comp rx=0 tx=0).stop 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 msgr2=0x7fbe141511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 0x7fbe141511a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbe04007fd0 tx=0x7fbe0400c820 comp rx=0 tx=0).stop 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 shutdown_connections 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbe141544c0 0x7fbe140cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbe140ceb40 0x7fbe141511a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbdf8078bc0 0x7fbdf807b0d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 --2- 172.21.15.22:0/2815008890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbe1406f4b0 0x7fbe14151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 >> 172.21.15.22:0/2815008890 conn(0x7fbe1408bfc0 msgr2=0x7fbe1406eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 shutdown_connections 2024-09-21T05:42:34.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.506+0000 7fbe1c550640 1 -- 172.21.15.22:0/2815008890 wait complete. 2024-09-21T05:42:34.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:34.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 -- 172.21.15.22:0/4146436873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c160c90 msgr2=0x7f0b1c0b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 --2- 172.21.15.22:0/4146436873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c160c90 0x7f0b1c0b3e20 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c073b20 tx=0x7f0b0402a4d0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 -- 172.21.15.22:0/4146436873 shutdown_connections 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 --2- 172.21.15.22:0/4146436873 >> v2:172.21.15.134:3300/0 conn(0x7f0b1c0b4360 0x7f0b1c0727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 --2- 172.21.15.22:0/4146436873 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c160c90 0x7f0b1c0b3e20 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 --2- 172.21.15.22:0/4146436873 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b1c0cff70 0x7f0b1c15c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 -- 172.21.15.22:0/4146436873 >> 172.21.15.22:0/4146436873 conn(0x7f0b1c0cee20 msgr2=0x7f0b1c0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 -- 172.21.15.22:0/4146436873 shutdown_connections 2024-09-21T05:42:34.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.634+0000 7f0b22615640 1 -- 172.21.15.22:0/4146436873 wait complete. 2024-09-21T05:42:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.636+0000 7f0b22615640 1 Processor -- start 2024-09-21T05:42:34.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.636+0000 7f0b22615640 1 -- start start 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.637+0000 7f0b22615640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c0b4360 0x7f0b1c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.637+0000 7f0b22615640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 0x7f0b1c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.637+0000 7f0b22615640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b1c160c90 0x7f0b1c06eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.637+0000 7f0b22615640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c073160 con 0x7f0b1c0cff70 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b22615640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c072fe0 con 0x7f0b1c0b4360 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b22615640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1c0732e0 con 0x7f0b1c160c90 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 0x7f0b1c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 0x7f0b1c06eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34550/0 (socket says 172.21.15.22:34550) 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 -- 172.21.15.22:0/1438869150 learned_addr learned my addr 172.21.15.22:0/1438869150 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1bfff640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c0b4360 0x7f0b1c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b20bb2640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b1c160c90 0x7f0b1c06eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 -- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b1c160c90 msgr2=0x7f0b1c06eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b1c160c90 0x7f0b1c06eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 -- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c0b4360 msgr2=0x7f0b1c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c0b4360 0x7f0b1c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 -- 172.21.15.22:0/1438869150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c06e180 con 0x7f0b1c0cff70 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b1b7fe640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 0x7f0b1c06eab0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0b04034040 tx=0x7f0b040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.638+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0403e030 con 0x7f0b1c0cff70 2024-09-21T05:42:34.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.639+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c06e460 con 0x7f0b1c0cff70 2024-09-21T05:42:34.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.639+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b04003ca0 con 0x7f0b1c0cff70 2024-09-21T05:42:34.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.639+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b04035040 con 0x7f0b1c0cff70 2024-09-21T05:42:34.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.639+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c06f530 con 0x7f0b1c0cff70 2024-09-21T05:42:34.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.640+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c0727e0 con 0x7f0b1c0cff70 2024-09-21T05:42:34.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.640+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b04041030 con 0x7f0b1c0cff70 2024-09-21T05:42:34.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.641+0000 7f0b18ff9640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0afc078bc0 0x7f0afc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:34.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.641+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f0b040bb610 con 0x7f0b1c0cff70 2024-09-21T05:42:34.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.641+0000 7f0b1bfff640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0afc078bc0 0x7f0afc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:34.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.642+0000 7f0b1bfff640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0afc078bc0 0x7f0afc07b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009b30 tx=0x7f0b0c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:34.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.647+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b04083410 con 0x7f0b1c0cff70 2024-09-21T05:42:34.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.871+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b1c0d1b70 con 0x7f0b1c0cff70 2024-09-21T05:42:34.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.871+0000 7f0b18ff9640 1 -- 172.21.15.22:0/1438869150 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5228 (secure 0 0 0) 0x7f0b04087e40 con 0x7f0b1c0cff70 2024-09-21T05:42:34.873 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:34.874 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":45,"btime":"2024-09-21T05:42:32:630314+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:32.016336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/1820431436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1820431436},{"type":"v1","addr":"172.21.15.134:6837","nonce":1820431436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-09-21T05:42:34.874 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 45 2024-09-21T05:42:34.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0afc078bc0 msgr2=0x7f0afc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0afc078bc0 0x7f0afc07b0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009b30 tx=0x7f0b0c005f90 comp rx=0 tx=0).stop 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 msgr2=0x7f0b1c06eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 0x7f0b1c06eab0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0b04034040 tx=0x7f0b040038a0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 shutdown_connections 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0afc078bc0 0x7f0afc07b0d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b1c160c90 0x7f0b1c06eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b1c0cff70 0x7f0b1c06eab0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 --2- 172.21.15.22:0/1438869150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b1c0b4360 0x7f0b1c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 >> 172.21.15.22:0/1438869150 conn(0x7f0b1c0cee20 msgr2=0x7f0b1c158540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 shutdown_connections 2024-09-21T05:42:34.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:34.873+0000 7f0b22615640 1 -- 172.21.15.22:0/1438869150 wait complete. 2024-09-21T05:42:35.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:42:29.001833+0000', 'modified': '2024-09-21T05:42:32.016336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6156}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6156': {'gid': 6156, 'name': 'b', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6837/1820431436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6835', 'nonce': 1820431436}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 1820431436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6156, 'qdb_cluster': [6156]} 2024-09-21T05:42:35.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:42:35.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T05:42:35.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9aeddb6640 1 -- 172.21.15.22:0/2978034768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad4003c10 msgr2=0x7f9ad4004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/2978034768 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ad4003c10 0x7f9ad4004060 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8001830 tx=0x7f9ad4005830 comp rx=0 tx=0).stop 2024-09-21T05:42:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9ae7fff640 1 -- 172.21.15.22:0/2978034768 reap_dead start 2024-09-21T05:42:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9aeddb6640 1 -- 172.21.15.22:0/2978034768 shutdown_connections 2024-09-21T05:42:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9aeddb6640 1 -- 172.21.15.22:0/2978034768 >> 172.21.15.22:0/2978034768 conn(0x7f9ae8072ae0 msgr2=0x7f9ae8072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9aeddb6640 1 -- 172.21.15.22:0/2978034768 shutdown_connections 2024-09-21T05:42:35.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.416+0000 7f9aeddb6640 1 -- 172.21.15.22:0/2978034768 wait complete. 2024-09-21T05:42:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.418+0000 7f9aeddb6640 1 Processor -- start 2024-09-21T05:42:35.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.418+0000 7f9aeddb6640 1 -- start start 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.419+0000 7f9aeddb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4003c10 0x7f9ae806fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.419+0000 7f9aeddb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 0x7f9ae8164530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9aeddb6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae8164a70 0x7f9ae806e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9aeddb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81656d0 con 0x7f9ae815a950 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9aeddb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae80d1b40 con 0x7f9ad4003c10 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9aeddb6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae8165850 con 0x7f9ae8164a70 2024-09-21T05:42:35.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 0x7f9ae8164530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 0x7f9ae8164530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34564/0 (socket says 172.21.15.22:34564) 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 -- 172.21.15.22:0/3874439580 learned_addr learned my addr 172.21.15.22:0/3874439580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4003c10 0x7f9ae806fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae8164a70 0x7f9ae806e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 -- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae8164a70 msgr2=0x7f9ae806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae8164a70 0x7f9ae806e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 -- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4003c10 msgr2=0x7f9ae806fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4003c10 0x7f9ae806fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 -- 172.21.15.22:0/3874439580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae8164e80 con 0x7f9ae815a950 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9adeffd640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 0x7f9ae8164530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006660 tx=0x7f9ad00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9ae4ff9640 1 -- 172.21.15.22:0/3874439580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0015070 con 0x7f9ae815a950 2024-09-21T05:42:35.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9ae4ff9640 1 -- 172.21.15.22:0/3874439580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad0011900 con 0x7f9ae815a950 2024-09-21T05:42:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.420+0000 7f9ae4ff9640 1 -- 172.21.15.22:0/3874439580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0011d50 con 0x7f9ae815a950 2024-09-21T05:42:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.421+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae8150970 con 0x7f9ae815a950 2024-09-21T05:42:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.421+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae81f9950 con 0x7f9ae815a950 2024-09-21T05:42:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.423+0000 7f9ae4ff9640 1 -- 172.21.15.22:0/3874439580 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ad0018070 con 0x7f9ae815a950 2024-09-21T05:42:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.423+0000 7f9ae4ff9640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9ac8078bc0 0x7f9ac807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.424+0000 7f9ae4ff9640 1 -- 172.21.15.22:0/3874439580 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9ad009ba20 con 0x7f9ae815a950 2024-09-21T05:42:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.424+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9ac8078bc0 0x7f9ac807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.424+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9ac8078bc0 0x7f9ac807b0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8032000 tx=0x7f9ad8001360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.424+0000 7f9af4dff740 1 Processor -- start 2024-09-21T05:42:35.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.424+0000 7f9af4dff740 1 -- start start 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915f9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5555911ff3b0 0x5555915fa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916173e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9af4dff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555591542820 con 0x555591207590 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555591531c50 con 0x555591228270 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555591535a60 con 0x5555911ff3b0 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916173e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916173e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34576/0 (socket says 172.21.15.22:34576) 2024-09-21T05:42:35.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915f9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 -- 172.21.15.22:0/321839546 learned_addr learned my addr 172.21.15.22:0/321839546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9adeffd640 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5555911ff3b0 0x5555915fa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 -- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5555911ff3b0 msgr2=0x5555915fa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5555911ff3b0 0x5555915fa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 -- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 msgr2=0x5555915f9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915f9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.426+0000 7f9ae7fff640 1 -- 172.21.15.22:0/321839546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555591293850 con 0x555591207590 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.427+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916173e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80cd940 tx=0x7f9ae0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e424b08acedb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.427+0000 7f9add7fa640 1 -- 172.21.15.22:0/321839546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0004b20 con 0x555591207590 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.427+0000 7f9add7fa640 1 -- 172.21.15.22:0/321839546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ae0004cc0 con 0x555591207590 2024-09-21T05:42:35.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.427+0000 7f9add7fa640 1 -- 172.21.15.22:0/321839546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0005220 con 0x555591207590 2024-09-21T05:42:35.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 -- 172.21.15.22:0/321839546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 msgr2=0x5555916173e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916173e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80cd940 tx=0x7f9ae0011810 comp rx=0 tx=0).stop 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 -- 172.21.15.22:0/321839546 shutdown_connections 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916173e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5555911ff3b0 0x5555915fa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 --2- 172.21.15.22:0/321839546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915f9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 -- 172.21.15.22:0/321839546 >> 172.21.15.22:0/321839546 conn(0x55559121c3c0 msgr2=0x5555915d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 -- 172.21.15.22:0/321839546 shutdown_connections 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.428+0000 7f9af4dff740 1 -- 172.21.15.22:0/321839546 wait complete. 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.429+0000 7f9af4dff740 1 -- start start 2024-09-21T05:42:35.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.429+0000 7f9af4dff740 1 Processor -- start 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591612650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591207590 0x55559160cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5555912a2620 con 0x5555911ff3b0 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5555914d75f0 con 0x555591228270 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55559160b380 con 0x555591207590 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591612650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9af4dff740 10 client.0 _ll_register_callbacks cb 0x5555914db0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591612650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34584/0 (socket says 172.21.15.22:34584) 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9ae77fe640 1 -- 172.21.15.22:0/584026884 learned_addr learned my addr 172.21.15.22:0/584026884 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.430+0000 7f9adeffd640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591207590 0x55559160cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ae77fe640 1 -- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591207590 msgr2=0x55559160cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591207590 0x55559160cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ae77fe640 1 -- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 msgr2=0x5555915da430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915da430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ae77fe640 1 -- 172.21.15.22:0/584026884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559130a6b0 con 0x5555911ff3b0 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591612650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802aa50 tx=0x7f9ad8006b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6c1b9d0962b7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ac67fc640 1 -- 172.21.15.22:0/584026884 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8055060 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ac67fc640 1 -- 172.21.15.22:0/584026884 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9ad8050b00 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae0004cc0 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ac67fc640 1 -- 172.21.15.22:0/584026884 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9ad8050d00 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9af4dff740 10 client.6258 fetch_fsmap learned FSMap version 45 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x55559160dbc0 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.431+0000 7f9ac67fc640 1 -- 172.21.15.22:0/584026884 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ae0004cc0 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9ac67fc640 1 -- 172.21.15.22:0/584026884 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f9ad8033480 con 0x5555911ff3b0 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9af4dff740 10 client.6258 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9af4dff740 10 client.6258 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9af4dff740 20 client.6258 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9af4dff740 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x5555911e6e40 0x555591626fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9af4dff740 4 client.6258 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 --> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555591544770 con 0x5555911e6e40 2024-09-21T05:42:35.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9adeffd640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x5555911e6e40 0x555591626fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9ac67fc640 10 client.6258 ms_handle_connect on v2:172.21.15.134:6835/1820431436 2024-09-21T05:42:35.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.432+0000 7f9adeffd640 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x5555911e6e40 0x555591626fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6847689faf2f8fd0 server_cookie=c752902bdda74c04 in_seq=0 out_seq=0 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.433+0000 7f9ac67fc640 1 -- 172.21.15.22:0/584026884 <== mds.0 v2:172.21.15.134:6835/1820431436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555591544770 con 0x5555911e6e40 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.433+0000 7f9ac67fc640 10 client.6258 handle_command_reply: tid=0 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.433+0000 7f9af4dff740 1 client.6258 shutdown 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 msgr2=0x555591612650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591612650 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9ad802aa50 tx=0x7f9ad8006b10 comp rx=0 tx=0).stop 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 shutdown_connections 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591228270 0x5555915da430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x5555911e6e40 0x555591626fc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591207590 0x55559160cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 --2- 172.21.15.22:0/584026884 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591612650 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9ac67fc640 0 client.6258 ms_handle_reset on v2:172.21.15.134:6835/1820431436 2024-09-21T05:42:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 >> 172.21.15.22:0/584026884 conn(0x55559121c3c0 msgr2=0x5555915d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 shutdown_connections 2024-09-21T05:42:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 1 -- 172.21.15.22:0/584026884 wait complete. 2024-09-21T05:42:35.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.434+0000 7f9af4dff740 20 client.6258 trim_cache size 0 max 16384 2024-09-21T05:42:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.453+0000 7f9af4dff740 1 Processor -- start 2024-09-21T05:42:35.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.453+0000 7f9af4dff740 1 -- start start 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555911ff3b0 0x5555915cab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916155e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x5555916376b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9af4dff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae8201530 con 0x555591207590 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae81f6800 con 0x5555911ff3b0 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae80cedf0 con 0x555591228270 2024-09-21T05:42:35.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916155e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916155e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34600/0 (socket says 172.21.15.22:34600) 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 -- 172.21.15.22:0/117968576 learned_addr learned my addr 172.21.15.22:0/117968576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555911ff3b0 0x5555915cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x5555916376b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 -- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 msgr2=0x5555916376b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x5555916376b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 -- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555911ff3b0 msgr2=0x5555915cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555911ff3b0 0x5555915cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.454+0000 7f9adeffd640 1 -- 172.21.15.22:0/117968576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555915d1c60 con 0x555591207590 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.455+0000 7f9adeffd640 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916155e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006630 tx=0x7f9ad0087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25d02f9276c11b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.455+0000 7f9adcff9640 1 -- 172.21.15.22:0/117968576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad001f040 con 0x555591207590 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.455+0000 7f9adcff9640 1 -- 172.21.15.22:0/117968576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad00682c0 con 0x555591207590 2024-09-21T05:42:35.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.455+0000 7f9adcff9640 1 -- 172.21.15.22:0/117968576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad006d170 con 0x555591207590 2024-09-21T05:42:35.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 -- 172.21.15.22:0/117968576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 msgr2=0x5555916155e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916155e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006630 tx=0x7f9ad0087040 comp rx=0 tx=0).stop 2024-09-21T05:42:35.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 -- 172.21.15.22:0/117968576 shutdown_connections 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x5555916376b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x555591207590 0x5555916155e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 --2- 172.21.15.22:0/117968576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5555911ff3b0 0x5555915cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 -- 172.21.15.22:0/117968576 >> 172.21.15.22:0/117968576 conn(0x55559121c3c0 msgr2=0x5555915cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 -- 172.21.15.22:0/117968576 shutdown_connections 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.456+0000 7f9af4dff740 1 -- 172.21.15.22:0/117968576 wait complete. 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.457+0000 7f9af4dff740 1 -- start start 2024-09-21T05:42:35.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.457+0000 7f9af4dff740 1 Processor -- start 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591691540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591207590 0x555591694080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x555591637b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5555915e8620 con 0x5555911ff3b0 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55559160df00 con 0x555591207590 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555591614350 con 0x555591228270 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9af4dff740 10 client.0 _ll_register_callbacks cb 0x7f9ad400c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591691540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591691540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34608/0 (socket says 172.21.15.22:34608) 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 -- 172.21.15.22:0/2885653955 learned_addr learned my addr 172.21.15.22:0/2885653955 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9adeffd640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591207590 0x555591694080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x555591637b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 -- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 msgr2=0x555591637b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x555591637b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 -- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591207590 msgr2=0x555591694080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591207590 0x555591694080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae77fe640 1 -- 172.21.15.22:0/2885653955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555915cab00 con 0x5555911ff3b0 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.458+0000 7f9ae7fff640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x555591637b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ae77fe640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591691540 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8001750 tx=0x7f9ad8033c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f19345ee34dc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ac67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ac67fc640 1 -- 172.21.15.22:0/2885653955 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8040680 con 0x5555911ff3b0 2024-09-21T05:42:35.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ac67fc640 1 -- 172.21.15.22:0/2885653955 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f9ad8040820 con 0x5555911ff3b0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad00682c0 con 0x5555911ff3b0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ac67fc640 1 -- 172.21.15.22:0/2885653955 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f9ad8040a20 con 0x5555911ff3b0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 10 client.6264 fetch_fsmap learned FSMap version 45 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x5555916948b0 con 0x5555911ff3b0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ac67fc640 1 -- 172.21.15.22:0/2885653955 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad00682c0 con 0x5555911ff3b0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9ac67fc640 1 -- 172.21.15.22:0/2885653955 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 45) ==== 3502+0+0 (secure 0 0 0) 0x7f9ad8040bc0 con 0x5555911ff3b0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 10 client.6264 fetch_fsmap finished waiting for FSMap version 45 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 10 client.6264 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 20 client.6264 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x55559160c8f0 0x55559162cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 4 client.6264 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.459+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 --> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] -- command(tid 0: {"prefix": "status"}) -- 0x555591540220 con 0x55559160c8f0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9adeffd640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x55559160c8f0 0x55559162cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9adeffd640 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x55559160c8f0 0x55559162cf80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce949653bfe39306 server_cookie=72440749d768827 in_seq=0 out_seq=0 2024-09-21T05:42:35.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9ac67fc640 10 client.6264 ms_handle_connect on v2:172.21.15.134:6835/1820431436 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9ac67fc640 1 -- 172.21.15.22:0/2885653955 <== mds.0 v2:172.21.15.134:6835/1820431436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x555591540220 con 0x55559160c8f0 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9ac67fc640 10 client.6264 handle_command_reply: tid=0 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 client.6264 shutdown 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 msgr2=0x555591691540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591691540 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8001750 tx=0x7f9ad8033c80 comp rx=0 tx=0).stop 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 shutdown_connections 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:6835/1820431436,v1:172.21.15.134:6837/1820431436] conn(0x55559160c8f0 0x55559162cf80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x555591228270 0x555591637b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x555591207590 0x555591694080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9ac67fc640 0 client.6264 ms_handle_reset on v2:172.21.15.134:6835/1820431436 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 --2- 172.21.15.22:0/2885653955 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5555911ff3b0 0x555591691540 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.460+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 >> 172.21.15.22:0/2885653955 conn(0x55559121c3c0 msgr2=0x5555915e7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 shutdown_connections 2024-09-21T05:42:35.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9af4dff740 1 -- 172.21.15.22:0/2885653955 wait complete. 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9af4dff740 20 client.6264 trim_cache size 0 max 16384 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 6156, 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.8520410939999996, 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 45, 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 68, 2024-09-21T05:42:35.468 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 68, 2024-09-21T05:42:35.469 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 10.47604394 2024-09-21T05:42:35.469 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9ac8078bc0 msgr2=0x7f9ac807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9ac8078bc0 0x7f9ac807b0d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8032000 tx=0x7f9ad8001360 comp rx=0 tx=0).stop 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 msgr2=0x7f9ae8164530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 0x7f9ae8164530 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0006660 tx=0x7f9ad00039c0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 shutdown_connections 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9ac8078bc0 0x7f9ac807b0d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9ae8164a70 0x7f9ae806e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9ae815a950 0x7f9ae8164530 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 --2- 172.21.15.22:0/3874439580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9ad4003c10 0x7f9ae806fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 >> 172.21.15.22:0/3874439580 conn(0x7f9ae8072ae0 msgr2=0x7f9ae80b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 shutdown_connections 2024-09-21T05:42:35.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.461+0000 7f9aeddb6640 1 -- 172.21.15.22:0/3874439580 wait complete. 2024-09-21T05:42:35.475 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:42:35.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12be4a640 1 -- 172.21.15.22:0/1096688249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 msgr2=0x7fd110004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12be4a640 1 --2- 172.21.15.22:0/1096688249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd110004800 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd124074550 tx=0x7fd110005830 comp rx=0 tx=0).stop 2024-09-21T05:42:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12a3e7640 1 -- 172.21.15.22:0/1096688249 reap_dead start 2024-09-21T05:42:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12be4a640 1 -- 172.21.15.22:0/1096688249 shutdown_connections 2024-09-21T05:42:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12be4a640 1 -- 172.21.15.22:0/1096688249 >> 172.21.15.22:0/1096688249 conn(0x7fd1240b3f10 msgr2=0x7fd1240b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12be4a640 1 -- 172.21.15.22:0/1096688249 shutdown_connections 2024-09-21T05:42:35.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.589+0000 7fd12be4a640 1 -- 172.21.15.22:0/1096688249 wait complete. 2024-09-21T05:42:35.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.591+0000 7fd12be4a640 1 Processor -- start 2024-09-21T05:42:35.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.591+0000 7fd12be4a640 1 -- start start 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.592+0000 7fd12be4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd12415e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.592+0000 7fd12be4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd12415ed30 0x7fd1241f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.592+0000 7fd12be4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1241f6cf0 0x7fd12415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.592+0000 7fd12be4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240d0fe0 con 0x7fd110003f90 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.592+0000 7fd12be4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240d0e60 con 0x7fd1241f6cf0 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.592+0000 7fd12be4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd1240d1160 con 0x7fd12415ed30 2024-09-21T05:42:35.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd12415e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd12415e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34622/0 (socket says 172.21.15.22:34622) 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 -- 172.21.15.22:0/2099265137 learned_addr learned my addr 172.21.15.22:0/2099265137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd12a3e7640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1241f6cf0 0x7fd12415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd1293e5640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd12415ed30 0x7fd1241f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 -- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd12415ed30 msgr2=0x7fd1241f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd12415ed30 0x7fd1241f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 -- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1241f6cf0 msgr2=0x7fd12415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1241f6cf0 0x7fd12415ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 -- 172.21.15.22:0/2099265137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1241f7100 con 0x7fd110003f90 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd1293e5640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd12415ed30 0x7fd1241f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd129be6640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd12415e600 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd120011b10 tx=0x7fd120011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12001e070 con 0x7fd110003f90 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd120004460 con 0x7fd110003f90 2024-09-21T05:42:35.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd124153710 con 0x7fd110003f90 2024-09-21T05:42:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12000fc40 con 0x7fd110003f90 2024-09-21T05:42:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.593+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd124153cd0 con 0x7fd110003f90 2024-09-21T05:42:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.594+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1240d1690 con 0x7fd110003f90 2024-09-21T05:42:35.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.595+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd120021070 con 0x7fd110003f90 2024-09-21T05:42:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.595+0000 7fd1167fc640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd108078bc0 0x7fd10807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.596+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fd12009b6f0 con 0x7fd110003f90 2024-09-21T05:42:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.596+0000 7fd1293e5640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd108078bc0 0x7fd10807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:35.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.596+0000 7fd1293e5640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd108078bc0 0x7fd10807b0d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd124160260 tx=0x7fd118001270 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:35.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.600+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd1241f7100 con 0x7fd110003f90 2024-09-21T05:42:35.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.825+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd1240b5c10 con 0x7fd110003f90 2024-09-21T05:42:35.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.825+0000 7fd1167fc640 1 -- 172.21.15.22:0/2099265137 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fd120063630 con 0x7fd110003f90 2024-09-21T05:42:35.827 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:42:35.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd108078bc0 msgr2=0x7fd10807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd108078bc0 0x7fd10807b0d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd124160260 tx=0x7fd118001270 comp rx=0 tx=0).stop 2024-09-21T05:42:35.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 msgr2=0x7fd12415e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:35.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd12415e600 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd120011b10 tx=0x7fd120011f10 comp rx=0 tx=0).stop 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 shutdown_connections 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd1241f6cf0 0x7fd12415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd108078bc0 0x7fd10807b0d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd12415ed30 0x7fd1241f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 --2- 172.21.15.22:0/2099265137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd110003f90 0x7fd12415e600 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 >> 172.21.15.22:0/2099265137 conn(0x7fd1240b3f10 msgr2=0x7fd1240c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 shutdown_connections 2024-09-21T05:42:35.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:35.827+0000 7fd12be4a640 1 -- 172.21.15.22:0/2099265137 wait complete. 2024-09-21T05:42:35.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:42:35.841 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:42:35.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:35.841 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:42:35.857 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:42:35.858 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42588sec preferred_lft 42588sec 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:42:35.859 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:35.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:35.860 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:42:35.922 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:42:35.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:35.924 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:42:35.980 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:42:35.980 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:42:35.980 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:42:35.980 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:42:35.981 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:42:35.981 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:42:35.981 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:42:35.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:42:35.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:42:35.981 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:42:35.981 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:42:36.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:42:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:42:36.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.110 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:42:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:42:36.126 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:42:36.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:42:36.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:42:36.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:42:36.193 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:36.250 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:42:36.250 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:42:36.251 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:36.307 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:42:36.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.307 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:36.403 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:36.403 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:42:36.403 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:42:36.414 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:36.414 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:42:36.414 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:42:36.414 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:42:36.415 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:42:36.421 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:36.421 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:42:36.421 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:42:36.421 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:42:36.422 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:42:36.423 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:42:36.423 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:42:36.429 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:42:36.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.429 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:36.455 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:42:36.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:42:36.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:42:36.561 INFO:teuthology.orchestra.run.smithi022.stdout:client6273 2024-09-21T05:42:36.562 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:36.562 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:36.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.578 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-21T05:42:36.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.640 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-21T05:42:36.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:36.699 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-21T05:42:36.756 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:42:36.756 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:42:36.756 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:42:36.756 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:42:36.756 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-21T05:42:36.756 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:42:36.862 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:42:36.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:42:36.871 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:36.871 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:36.928 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:36.928 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:36.985 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 -- 172.21.15.22:0/3119446116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d80d2460 msgr2=0x7f01d815d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3119446116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d80d2460 0x7f01d815d130 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f01d80d2de0 tx=0x7f01cc02a560 comp rx=0 tx=0).stop 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 -- 172.21.15.22:0/3119446116 shutdown_connections 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3119446116 >> v2:172.21.15.134:3300/0 conn(0x7f01d815d670 0x7f01d80b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3119446116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d80d2460 0x7f01d815d130 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3119446116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d81570e0 0x7f01d80d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 -- 172.21.15.22:0/3119446116 >> 172.21.15.22:0/3119446116 conn(0x7f01d806d550 msgr2=0x7f01d8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:37.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 -- 172.21.15.22:0/3119446116 shutdown_connections 2024-09-21T05:42:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.140+0000 7f01dfc38640 1 -- 172.21.15.22:0/3119446116 wait complete. 2024-09-21T05:42:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.142+0000 7f01dfc38640 1 Processor -- start 2024-09-21T05:42:37.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.142+0000 7f01dfc38640 1 -- start start 2024-09-21T05:42:37.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dfc38640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80d2460 0x7f01d806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dfc38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 0x7f01d815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dfc38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815d670 0x7f01d80cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dfc38640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01d80b7b50 con 0x7f01d81570e0 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dfc38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01d80b79d0 con 0x7f01d815d670 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dfc38640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01d80b7cd0 con 0x7f01d80d2460 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dd1d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 0x7f01d815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dd1d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 0x7f01d815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34646/0 (socket says 172.21.15.22:34646) 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.143+0000 7f01dd1d3640 1 -- 172.21.15.22:0/3232784127 learned_addr learned my addr 172.21.15.22:0/3232784127 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01de1d5640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815d670 0x7f01d80cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd9d4640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80d2460 0x7f01d806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd1d3640 1 -- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80d2460 msgr2=0x7f01d806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd1d3640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80d2460 0x7f01d806f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd1d3640 1 -- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815d670 msgr2=0x7f01d80cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd1d3640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815d670 0x7f01d80cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd1d3640 1 -- 172.21.15.22:0/3232784127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d81f9740 con 0x7f01d81570e0 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd9d4640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80d2460 0x7f01d806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dd1d3640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 0x7f01d815bc00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f01cc034040 tx=0x7f01cc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc03e030 con 0x7f01d81570e0 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01cc0047a0 con 0x7f01d81570e0 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d81f9a20 con 0x7f01d81570e0 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc035040 con 0x7f01d81570e0 2024-09-21T05:42:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.144+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d81f9fe0 con 0x7f01d81570e0 2024-09-21T05:42:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.145+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d80d1f20 con 0x7f01d81570e0 2024-09-21T05:42:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.146+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01cc041030 con 0x7f01d81570e0 2024-09-21T05:42:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.147+0000 7f01d67fc640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01c4078b90 0x7f01c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.147+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f01cc0bb690 con 0x7f01d81570e0 2024-09-21T05:42:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.147+0000 7f01dd9d4640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01c4078b90 0x7f01c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.148+0000 7f01dd9d4640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01c4078b90 0x7f01c407b0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f01c8007eb0 tx=0x7f01c80073c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.152+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01cc083ad0 con 0x7f01d81570e0 2024-09-21T05:42:37.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.351+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01d806fe20 con 0x7f01d81570e0 2024-09-21T05:42:37.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.351+0000 7f01d67fc640 1 -- 172.21.15.22:0/3232784127 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f01cc0833d0 con 0x7f01d81570e0 2024-09-21T05:42:37.354 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01c4078b90 msgr2=0x7f01c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01c4078b90 0x7f01c407b0a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f01c8007eb0 tx=0x7f01c80073c0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 msgr2=0x7f01d815bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 0x7f01d815bc00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f01cc034040 tx=0x7f01cc0039e0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 shutdown_connections 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01d815d670 0x7f01d80cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01d81570e0 0x7f01d815bc00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01c4078b90 0x7f01c407b0a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 --2- 172.21.15.22:0/3232784127 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01d80d2460 0x7f01d806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 >> 172.21.15.22:0/3232784127 conn(0x7f01d806d550 msgr2=0x7f01d80b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:37.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 shutdown_connections 2024-09-21T05:42:37.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.354+0000 7f01dfc38640 1 -- 172.21.15.22:0/3232784127 wait complete. 2024-09-21T05:42:37.367 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:42:37.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:37.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:37.425 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:42:37.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:37.426 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:42:37.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:37.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1136536717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcddc005250 msgr2=0x7fcddc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1136536717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcddc005250 0x7fcddc00b9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcde4001dc0 tx=0x7fcddc005830 comp rx=0 tx=0).stop 2024-09-21T05:42:37.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1136536717 shutdown_connections 2024-09-21T05:42:37.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1136536717 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcddc005250 0x7fcddc00b9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1136536717 >> 172.21.15.22:0/1136536717 conn(0x7fcdf4072ae0 msgr2=0x7fcdf4072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:37.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1136536717 shutdown_connections 2024-09-21T05:42:37.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.603+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1136536717 wait complete. 2024-09-21T05:42:37.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.605+0000 7fcdfb84d640 1 Processor -- start 2024-09-21T05:42:37.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.605+0000 7fcdfb84d640 1 -- start start 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdfb84d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcddc005250 0x7fcdf41f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdfb84d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdf406f850 0x7fcdf40ab0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdfb84d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 0x7fcdf40ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdfb84d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf40d1a60 con 0x7fcdf406fc60 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdfb84d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf40d18e0 con 0x7fcddc005250 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdfb84d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf40d1100 con 0x7fcdf406f850 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdf9dea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 0x7fcdf40ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdf9dea640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 0x7fcdf40ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34654/0 (socket says 172.21.15.22:34654) 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.606+0000 7fcdf9dea640 1 -- 172.21.15.22:0/1030072694 learned_addr learned my addr 172.21.15.22:0/1030072694 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf95e9640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcddc005250 0x7fcdf41f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf8de8640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdf406f850 0x7fcdf40ab0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf9dea640 1 -- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdf406f850 msgr2=0x7fcdf40ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf9dea640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdf406f850 0x7fcdf40ab0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf9dea640 1 -- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcddc005250 msgr2=0x7fcdf41f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf9dea640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcddc005250 0x7fcdf41f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf9dea640 1 -- 172.21.15.22:0/1030072694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf41f5e30 con 0x7fcdf406fc60 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf9dea640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 0x7fcdf40ab630 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcde402acc0 tx=0x7fcde4006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4003d80 con 0x7fcdf406fc60 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde4005a10 con 0x7fcdf406fc60 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf40a9680 con 0x7fcdf406fc60 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4005e20 con 0x7fcdf406fc60 2024-09-21T05:42:37.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.607+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf40a9c40 con 0x7fcdf406fc60 2024-09-21T05:42:37.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.608+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf415c9d0 con 0x7fcdf406fc60 2024-09-21T05:42:37.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.609+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcde4005fc0 con 0x7fcdf406fc60 2024-09-21T05:42:37.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.610+0000 7fcdf1ffb640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcde0078b90 0x7fcde007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:37.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.610+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fcde40bae40 con 0x7fcdf406fc60 2024-09-21T05:42:37.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.610+0000 7fcdf95e9640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcde0078b90 0x7fcde007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:37.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.611+0000 7fcdf95e9640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcde0078b90 0x7fcde007b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcdec006b50 tx=0x7fcdec006720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:37.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.614+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcde4043060 con 0x7fcdf406fc60 2024-09-21T05:42:37.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.842+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdf40cbb90 con 0x7fcdf406fc60 2024-09-21T05:42:37.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.842+0000 7fcdf1ffb640 1 -- 172.21.15.22:0/1030072694 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5228 (secure 0 0 0) 0x7fcde40833d0 con 0x7fcdf406fc60 2024-09-21T05:42:37.844 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:37.844 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":45,"btime":"2024-09-21T05:42:32:630314+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6153,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:32.016336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"b","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/1820431436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1820431436},{"type":"v1","addr":"172.21.15.134:6837","nonce":1820431436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6156,"qdb_cluster":[6156]},"id":5}]} 2024-09-21T05:42:37.845 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 45 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcde0078b90 msgr2=0x7fcde007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcde0078b90 0x7fcde007b0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcdec006b50 tx=0x7fcdec006720 comp rx=0 tx=0).stop 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 msgr2=0x7fcdf40ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 0x7fcdf40ab630 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fcde402acc0 tx=0x7fcde4006ca0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 shutdown_connections 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcdf406fc60 0x7fcdf40ab630 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcde0078b90 0x7fcde007b0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcdf406f850 0x7fcdf40ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 --2- 172.21.15.22:0/1030072694 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcddc005250 0x7fcdf41f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 >> 172.21.15.22:0/1030072694 conn(0x7fcdf4072ae0 msgr2=0x7fcdf40b6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:37.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 shutdown_connections 2024-09-21T05:42:37.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:37.844+0000 7fcdfb84d640 1 -- 172.21.15.22:0/1030072694 wait complete. 2024-09-21T05:42:37.858 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-21T05:42:37.993 INFO:teuthology.orchestra.run.smithi134.stdout:{"message":"","return_code":0} 2024-09-21T05:42:37.994 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:42:37.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:42:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 -- 172.21.15.22:0/3569309231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 msgr2=0x7fb5d00735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/3569309231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d00735b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8005fd0 tx=0x7fb5c802aa90 comp rx=0 tx=0).stop 2024-09-21T05:42:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 -- 172.21.15.22:0/3569309231 shutdown_connections 2024-09-21T05:42:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/3569309231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d00735b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/3569309231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5d00706b0 0x7fb5d00cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 -- 172.21.15.22:0/3569309231 >> 172.21.15.22:0/3569309231 conn(0x7fb5d00b1cb0 msgr2=0x7fb5d00b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 -- 172.21.15.22:0/3569309231 shutdown_connections 2024-09-21T05:42:38.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.110+0000 7fb5d60c5640 1 -- 172.21.15.22:0/3569309231 wait complete. 2024-09-21T05:42:38.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.112+0000 7fb5d60c5640 1 Processor -- start 2024-09-21T05:42:38.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.112+0000 7fb5d60c5640 1 -- start start 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5d60c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5d00ca800 0x7fb5d00b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5d60c5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d01f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5d60c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5d015bee0 0x7fb5d01f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5d60c5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0073e80 con 0x7fb5d00cad30 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5d60c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0073d00 con 0x7fb5d00ca800 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5d60c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d0074000 con 0x7fb5d015bee0 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5ceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d01f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5ceffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d01f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34668/0 (socket says 172.21.15.22:34668) 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.113+0000 7fb5ceffd640 1 -- 172.21.15.22:0/2593056981 learned_addr learned my addr 172.21.15.22:0/2593056981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:38.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5cffff640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5d015bee0 0x7fb5d01f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5cf7fe640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5d00ca800 0x7fb5d00b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5ceffd640 1 -- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5d015bee0 msgr2=0x7fb5d01f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5ceffd640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5d015bee0 0x7fb5d01f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5ceffd640 1 -- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5d00ca800 msgr2=0x7fb5d00b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5ceffd640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5d00ca800 0x7fb5d00b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5ceffd640 1 -- 172.21.15.22:0/2593056981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d01f9650 con 0x7fb5d00cad30 2024-09-21T05:42:38.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5ceffd640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d01f27e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400e790 tx=0x7fb5c400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c400eea0 con 0x7fb5d00cad30 2024-09-21T05:42:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5c400cdb0 con 0x7fb5d00cad30 2024-09-21T05:42:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4010690 con 0x7fb5d00cad30 2024-09-21T05:42:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d0159660 con 0x7fb5d00cad30 2024-09-21T05:42:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.114+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d01f5480 con 0x7fb5d00cad30 2024-09-21T05:42:38.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.115+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d0074560 con 0x7fb5d00cad30 2024-09-21T05:42:38.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.116+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb5c4010830 con 0x7fb5d00cad30 2024-09-21T05:42:38.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.117+0000 7fb5b7fff640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5bc078b90 0x7fb5bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.117+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fb5c409b790 con 0x7fb5d00cad30 2024-09-21T05:42:38.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.117+0000 7fb5cf7fe640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5bc078b90 0x7fb5bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.117+0000 7fb5cf7fe640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5bc078b90 0x7fb5bc07b0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0000f60 tx=0x7fb5c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.121+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5c40634e0 con 0x7fb5d00cad30 2024-09-21T05:42:38.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.348+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb5d0159b80 con 0x7fb5d00cad30 2024-09-21T05:42:38.350 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:42:38.349+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:42:38.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.350+0000 7fb5b7fff640 1 -- 172.21.15.22:0/2593056981 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7fb5c4067f10 con 0x7fb5d00cad30 2024-09-21T05:42:38.352 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5bc078b90 msgr2=0x7fb5bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5bc078b90 0x7fb5bc07b0a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0000f60 tx=0x7fb5c0009040 comp rx=0 tx=0).stop 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 msgr2=0x7fb5d01f27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d01f27e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb5c400e790 tx=0x7fb5c400ec50 comp rx=0 tx=0).stop 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 shutdown_connections 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb5bc078b90 0x7fb5bc07b0a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb5d015bee0 0x7fb5d01f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb5d00cad30 0x7fb5d01f27e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 --2- 172.21.15.22:0/2593056981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb5d00ca800 0x7fb5d00b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 >> 172.21.15.22:0/2593056981 conn(0x7fb5d00b1cb0 msgr2=0x7fb5d00acf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:38.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 shutdown_connections 2024-09-21T05:42:38.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.353+0000 7fb5d60c5640 1 -- 172.21.15.22:0/2593056981 wait complete. 2024-09-21T05:42:38.358 INFO:tasks.ceph.mds.b.smithi134.stderr: -5250> 2024-09-21T05:42:31.582+0000 7f6abf3f7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:42:38.366 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T05:42:38.413 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:42:38.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.423+0000 7f01d2a1af40 1 Processor -- start 2024-09-21T05:42:38.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.423+0000 7f01d2a1af40 1 -- start start 2024-09-21T05:42:38.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01d2a1af40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 0x55ce02a6a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01d2a1af40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce02a77300 0x55ce029a4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01d2a1af40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01d2a1af40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ce029eb840 con 0x55ce029d22c0 2024-09-21T05:42:38.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01d2a1af40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ce028f2d60 con 0x55ce02a77300 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01d2a1af40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ce029eb9c0 con 0x55ce02a77c50 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 0x55ce02a6a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 0x55ce02a6a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34686/0 (socket says 172.21.15.22:34686) 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 -- 172.21.15.22:0/2902696820 learned_addr learned my addr 172.21.15.22:0/2902696820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 -- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 msgr2=0x55ce02a728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01c3fff640 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce02a77300 0x55ce029a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cccd5640 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 -- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce02a77300 msgr2=0x55ce029a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce02a77300 0x55ce029a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 -- 172.21.15.22:0/2902696820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce02a69c40 con 0x55ce029d22c0 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cc4d4640 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 0x55ce02a6a690 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b550 tx=0x7f01bc00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657e35976be9b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cb4d2640 1 -- 172.21.15.22:0/2902696820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00a8c0 con 0x55ce029d22c0 2024-09-21T05:42:38.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cb4d2640 1 -- 172.21.15.22:0/2902696820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01bc00aa60 con 0x55ce029d22c0 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.425+0000 7f01cb4d2640 1 -- 172.21.15.22:0/2902696820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc008450 con 0x55ce029d22c0 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 -- 172.21.15.22:0/2902696820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 msgr2=0x55ce02a6a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 0x55ce02a6a690 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b550 tx=0x7f01bc00ba10 comp rx=0 tx=0).stop 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 -- 172.21.15.22:0/2902696820 shutdown_connections 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce02a77300 0x55ce029a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/2902696820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce029d22c0 0x55ce02a6a690 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 -- 172.21.15.22:0/2902696820 >> 172.21.15.22:0/2902696820 conn(0x55ce028c3ac0 msgr2=0x55ce029a51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 -- 172.21.15.22:0/2902696820 shutdown_connections 2024-09-21T05:42:38.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.426+0000 7f01d2a1af40 1 -- 172.21.15.22:0/2902696820 wait complete. 2024-09-21T05:42:38.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.427+0000 7f01d2a1af40 1 Processor -- start 2024-09-21T05:42:38.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.427+0000 7f01d2a1af40 1 -- start start 2024-09-21T05:42:38.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.428+0000 7f01d2a1af40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce029d22c0 0x55ce02a68a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.428+0000 7f01d2a1af40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce02a77300 0x55ce02a760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.428+0000 7f01d2a1af40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a76610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.428+0000 7f01d2a1af40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ce029a3440 con 0x55ce02a77300 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.428+0000 7f01d2a1af40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ce029a3720 con 0x55ce029d22c0 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.428+0000 7f01d2a1af40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ce029e9590 con 0x55ce02a77c50 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce02a77300 0x55ce02a760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce02a77300 0x55ce02a760d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34700/0 (socket says 172.21.15.22:34700) 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 -- 172.21.15.22:0/1334362839 learned_addr learned my addr 172.21.15.22:0/1334362839 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01cc4d4640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce029d22c0 0x55ce02a68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01cccd5640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a76610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 -- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 msgr2=0x55ce02a76610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ce02a77c50 0x55ce02a76610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 -- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce029d22c0 msgr2=0x55ce02a68a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ce029d22c0 0x55ce02a68a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 -- 172.21.15.22:0/1334362839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce02a6cc00 con 0x55ce02a77300 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01c3fff640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce02a77300 0x55ce02a760d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f01b400b770 tx=0x7f01b400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01ca4d0640 1 -- 172.21.15.22:0/1334362839 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b401a070 con 0x55ce02a77300 2024-09-21T05:42:38.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01ca4d0640 1 -- 172.21.15.22:0/1334362839 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01b4008990 con 0x55ce02a77300 2024-09-21T05:42:38.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ce029e6dd0 con 0x55ce02a77300 2024-09-21T05:42:38.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01ca4d0640 1 -- 172.21.15.22:0/1334362839 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4008de0 con 0x55ce02a77300 2024-09-21T05:42:38.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.429+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce029e92d0 con 0x55ce02a77300 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.431+0000 7f01ca4d0640 1 -- 172.21.15.22:0/1334362839 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01b401d070 con 0x55ce02a77300 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01ca4d0640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01ac0775a0 0x7f01ac079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01ca4d0640 1 -- 172.21.15.22:0/1334362839 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f01b40997a0 con 0x55ce02a77300 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01cc4d4640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01ac0775a0 0x7f01ac079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55ce02a695d0 0x55ce029e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x55ce02a6d3c0 con 0x55ce02a695d0 2024-09-21T05:42:38.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01cccd5640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55ce02a695d0 0x55ce029e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.432+0000 7f01cc4d4640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01ac0775a0 0x7f01ac079ab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b4f0 tx=0x7f01bc018040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.433+0000 7f01cccd5640 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55ce02a695d0 0x55ce029e8c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01cccd5640 1 -- 172.21.15.22:0/1334362839 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v69'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f01c400b040 con 0x55ce02a695d0 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55ce02a695d0 msgr2=0x55ce029e8c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55ce02a695d0 0x55ce029e8c60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01ac0775a0 msgr2=0x7f01ac079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01ac0775a0 0x7f01ac079ab0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00b4f0 tx=0x7f01bc018040 comp rx=0 tx=0).stop 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce02a77300 msgr2=0x55ce02a760d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 --2- 172.21.15.22:0/1334362839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ce02a77300 0x55ce02a760d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f01b400b770 tx=0x7f01b400bc30 comp rx=0 tx=0).stop 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01cccd5640 1 -- 172.21.15.22:0/1334362839 reap_dead start 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 shutdown_connections 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 >> 172.21.15.22:0/1334362839 conn(0x55ce028c3ac0 msgr2=0x55ce02a728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 shutdown_connections 2024-09-21T05:42:38.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.434+0000 7f01d2a1af40 1 -- 172.21.15.22:0/1334362839 wait complete. 2024-09-21T05:42:38.439 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T05:42:38.559 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c6455640 1 -- 172.21.15.22:0/2377297246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9a8004000 msgr2=0x7fb9a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c6455640 1 --2- 172.21.15.22:0/2377297246 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9a8004000 0x7fb9a8004800 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4001660 tx=0x7fb9a8005830 comp rx=0 tx=0).stop 2024-09-21T05:42:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c49f2640 1 -- 172.21.15.22:0/2377297246 reap_dead start 2024-09-21T05:42:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c6455640 1 -- 172.21.15.22:0/2377297246 shutdown_connections 2024-09-21T05:42:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c6455640 1 -- 172.21.15.22:0/2377297246 >> 172.21.15.22:0/2377297246 conn(0x7fb9c008bfc0 msgr2=0x7fb9c009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c6455640 1 -- 172.21.15.22:0/2377297246 shutdown_connections 2024-09-21T05:42:38.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.557+0000 7fb9c6455640 1 -- 172.21.15.22:0/2377297246 wait complete. 2024-09-21T05:42:38.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.559+0000 7fb9c6455640 1 Processor -- start 2024-09-21T05:42:38.561 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.559+0000 7fb9c6455640 1 -- start start 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.560+0000 7fb9c6455640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a8004000 0x7fb9c00cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.560+0000 7fb9c6455640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9c014d810 0x7fb9c00b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c6455640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 0x7fb9c01f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c6455640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c00d28c0 con 0x7fb9c00b2810 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c6455640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c00d2740 con 0x7fb9a8004000 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c6455640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb9c00d2a40 con 0x7fb9c014d810 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 0x7fb9c01f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 0x7fb9c01f90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34728/0 (socket says 172.21.15.22:34728) 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9bf7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9c014d810 0x7fb9c00b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 -- 172.21.15.22:0/3566275301 learned_addr learned my addr 172.21.15.22:0/3566275301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9bffff640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a8004000 0x7fb9c00cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 -- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9c014d810 msgr2=0x7fb9c00b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9c014d810 0x7fb9c00b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 -- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a8004000 msgr2=0x7fb9c00cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a8004000 0x7fb9c00cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 -- 172.21.15.22:0/3566275301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c00b2c20 con 0x7fb9c00b2810 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9bf7fe640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9c014d810 0x7fb9c00b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c49f2640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 0x7fb9c01f90e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005fd0 tx=0x7fb9b800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8004b20 con 0x7fb9c00b2810 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9b8004cc0 con 0x7fb9c00b2810 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c00b5ef0 con 0x7fb9c00b2810 2024-09-21T05:42:38.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.561+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8003bd0 con 0x7fb9c00b2810 2024-09-21T05:42:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.562+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c00b5410 con 0x7fb9c00b2810 2024-09-21T05:42:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.562+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c00cdc10 con 0x7fb9c00b2810 2024-09-21T05:42:38.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.563+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb9b8004030 con 0x7fb9c00b2810 2024-09-21T05:42:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.564+0000 7fb9bcff9640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb9ac078b90 0x7fb9ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.564+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fb9b809b540 con 0x7fb9c00b2810 2024-09-21T05:42:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.564+0000 7fb9bffff640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb9ac078b90 0x7fb9ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:38.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.565+0000 7fb9bffff640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb9ac078b90 0x7fb9ac07b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0000ff0 tx=0x7fb9b00067a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:38.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.568+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb9c00b2c20 con 0x7fb9c00b2810 2024-09-21T05:42:38.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:38.798+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fb9c00d2fd0 con 0x7fb9c00b2810 2024-09-21T05:42:39.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.352+0000 7fb9bcff9640 1 -- 172.21.15.22:0/3566275301 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) ==== 162+0+0 (secure 0 0 0) 0x7fb9b80633f0 con 0x7fb9c00b2810 2024-09-21T05:42:39.354 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T05:42:39.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb9ac078b90 msgr2=0x7fb9ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb9ac078b90 0x7fb9ac07b0a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0000ff0 tx=0x7fb9b00067a0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 msgr2=0x7fb9c01f90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 0x7fb9c01f90e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005fd0 tx=0x7fb9b800f3c0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 shutdown_connections 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb9c00b2810 0x7fb9c01f90e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb9ac078b90 0x7fb9ac07b0a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb9c014d810 0x7fb9c00b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 --2- 172.21.15.22:0/3566275301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb9a8004000 0x7fb9c00cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.354+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 >> 172.21.15.22:0/3566275301 conn(0x7fb9c008bfc0 msgr2=0x7fb9c0068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.355+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 shutdown_connections 2024-09-21T05:42:39.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.355+0000 7fb9c6455640 1 -- 172.21.15.22:0/3566275301 wait complete. 2024-09-21T05:42:39.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 -- 172.21.15.22:0/2008330853 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 msgr2=0x7f91d40b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 --2- 172.21.15.22:0/2008330853 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 0x7f91d40b63b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009830 tx=0x7f91c402a510 comp rx=0 tx=0).stop 2024-09-21T05:42:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 -- 172.21.15.22:0/2008330853 shutdown_connections 2024-09-21T05:42:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 --2- 172.21.15.22:0/2008330853 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 0x7f91d40b63b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 --2- 172.21.15.22:0/2008330853 >> v2:172.21.15.134:3300/0 conn(0x7f91d40b9970 0x7f91d40b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 -- 172.21.15.22:0/2008330853 >> 172.21.15.22:0/2008330853 conn(0x7f91d409d7f0 msgr2=0x7f91d40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 -- 172.21.15.22:0/2008330853 shutdown_connections 2024-09-21T05:42:39.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.490+0000 7f91db7ba640 1 -- 172.21.15.22:0/2008330853 wait complete. 2024-09-21T05:42:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.492+0000 7f91db7ba640 1 Processor -- start 2024-09-21T05:42:39.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.492+0000 7f91db7ba640 1 -- start start 2024-09-21T05:42:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91db7ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 0x7f91d41f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91db7ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 0x7f91d41615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:39.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91db7ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91d40ced80 0x7f91d4161af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91db7ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91d406f270 con 0x7f91d40b9970 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91db7ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91d406f0f0 con 0x7f91d40ced80 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91db7ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f91d406f3f0 con 0x7f91d40b5fa0 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91d8d55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 0x7f91d41615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91d8d55640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 0x7f91d41615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34740/0 (socket says 172.21.15.22:34740) 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91d8d55640 1 -- 172.21.15.22:0/3600935810 learned_addr learned my addr 172.21.15.22:0/3600935810 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.493+0000 7f91d9556640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 0x7f91d41f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d9d57640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91d40ced80 0x7f91d4161af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d8d55640 1 -- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 msgr2=0x7f91d41f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d8d55640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 0x7f91d41f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d8d55640 1 -- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91d40ced80 msgr2=0x7f91d4161af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d8d55640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91d40ced80 0x7f91d4161af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d8d55640 1 -- 172.21.15.22:0/3600935810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d415f540 con 0x7f91d40b9970 2024-09-21T05:42:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91d8d55640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 0x7f91d41615b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009950 tx=0x7f91c4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4031d60 con 0x7f91d40b9970 2024-09-21T05:42:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d415f820 con 0x7f91d40b9970 2024-09-21T05:42:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91c4033e10 con 0x7f91d40b9970 2024-09-21T05:42:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4033310 con 0x7f91d40b9970 2024-09-21T05:42:39.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.494+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d415d540 con 0x7f91d40b9970 2024-09-21T05:42:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.496+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d40cfc40 con 0x7f91d40b9970 2024-09-21T05:42:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.496+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91c40334b0 con 0x7f91d40b9970 2024-09-21T05:42:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.497+0000 7f91c9ffb640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91b8078b90 0x7f91b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.497+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f91c40bbe60 con 0x7f91d40b9970 2024-09-21T05:42:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.497+0000 7f91d9556640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91b8078b90 0x7f91b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:39.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.498+0000 7f91d9556640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91b8078b90 0x7f91b807b0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009660 tx=0x7f91d00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:39.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.501+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91c404a030 con 0x7f91d40b9970 2024-09-21T05:42:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.725+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91d40b4de0 con 0x7f91d40b9970 2024-09-21T05:42:39.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.726+0000 7f91c9ffb640 1 -- 172.21.15.22:0/3600935810 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5222 (secure 0 0 0) 0x7f91c403f030 con 0x7f91d40b9970 2024-09-21T05:42:39.728 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:39.728 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":48,"btime":"2024-09-21T05:42:39:355715+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:39.355703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-21T05:42:39.728 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 48 2024-09-21T05:42:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91b8078b90 msgr2=0x7f91b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91b8078b90 0x7f91b807b0a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009660 tx=0x7f91d00091e0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 msgr2=0x7f91d41615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:39.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 0x7f91d41615b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009950 tx=0x7f91c4033c20 comp rx=0 tx=0).stop 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 shutdown_connections 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91d40ced80 0x7f91d4161af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91d40b9970 0x7f91d41615b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91b8078b90 0x7f91b807b0a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 --2- 172.21.15.22:0/3600935810 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91d40b5fa0 0x7f91d41f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 >> 172.21.15.22:0/3600935810 conn(0x7f91d409d7f0 msgr2=0x7f91d40d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 shutdown_connections 2024-09-21T05:42:39.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:39.728+0000 7f91db7ba640 1 -- 172.21.15.22:0/3600935810 wait complete. 2024-09-21T05:42:40.144 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:42:29.001833+0000', 'modified': '2024-09-21T05:42:39.355703+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6153}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6153': {'gid': 6153, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.134:6836/2827334455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2827334455}, {'type': 'v1', 'addr': '172.21.15.134:6836', 'nonce': 2827334455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:42:40.144 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6153:up:replay 2024-09-21T05:42:41.145 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.274+0000 7f93c1b4a640 1 -- 172.21.15.22:0/323129924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0cd220 msgr2=0x7f93bc15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.274+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/323129924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0cd220 0x7f93bc15f410 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f93bc070fe0 tx=0x7f93b402aa30 comp rx=0 tx=0).stop 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.275+0000 7f93c1b4a640 1 -- 172.21.15.22:0/323129924 shutdown_connections 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.275+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/323129924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0cd220 0x7f93bc15f410 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.275+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/323129924 >> v2:172.21.15.134:3300/0 conn(0x7f93bc06ee90 0x7f93bc0b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.275+0000 7f93c1b4a640 1 -- 172.21.15.22:0/323129924 >> 172.21.15.22:0/323129924 conn(0x7f93bc06d7d0 msgr2=0x7f93bc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.275+0000 7f93c1b4a640 1 -- 172.21.15.22:0/323129924 shutdown_connections 2024-09-21T05:42:41.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.275+0000 7f93c1b4a640 1 -- 172.21.15.22:0/323129924 wait complete. 2024-09-21T05:42:41.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.276+0000 7f93c1b4a640 1 Processor -- start 2024-09-21T05:42:41.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.277+0000 7f93c1b4a640 1 -- start start 2024-09-21T05:42:41.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93c1b4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93bc06ee90 0x7f93bc14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:41.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93c1b4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 0x7f93bc14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:41.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93c1b4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93bc0cd220 0x7f93bc0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:41.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93c1b4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc15f950 con 0x7f93bc0b4d30 2024-09-21T05:42:41.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93c1b4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc06f4b0 con 0x7f93bc06ee90 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93c1b4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93bc15fad0 con 0x7f93bc0cd220 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 0x7f93bc14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 0x7f93bc14f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34752/0 (socket says 172.21.15.22:34752) 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93baffd640 1 -- 172.21.15.22:0/1672510470 learned_addr learned my addr 172.21.15.22:0/1672510470 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93bb7fe640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93bc06ee90 0x7f93bc14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93bbfff640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93bc0cd220 0x7f93bc0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93baffd640 1 -- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93bc0cd220 msgr2=0x7f93bc0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93baffd640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93bc0cd220 0x7f93bc0b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.278+0000 7f93baffd640 1 -- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93bc06ee90 msgr2=0x7f93bc14edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93baffd640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93bc06ee90 0x7f93bc14edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93baffd640 1 -- 172.21.15.22:0/1672510470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc0afb20 con 0x7f93bc0b4d30 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93baffd640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 0x7f93bc14f330 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f93b000b420 tx=0x7f93b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0011830 con 0x7f93bc0b4d30 2024-09-21T05:42:41.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b0016070 con 0x7f93bc0b4d30 2024-09-21T05:42:41.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc0cc2b0 con 0x7f93bc0b4d30 2024-09-21T05:42:41.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0011d60 con 0x7f93bc0b4d30 2024-09-21T05:42:41.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.279+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc0b00c0 con 0x7f93bc0b4d30 2024-09-21T05:42:41.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.280+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc0b47f0 con 0x7f93bc0b4d30 2024-09-21T05:42:41.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.281+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93b001b460 con 0x7f93bc0b4d30 2024-09-21T05:42:41.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.281+0000 7f93a3fff640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93a8078b90 0x7f93a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:41.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.281+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f93b009a850 con 0x7f93bc0b4d30 2024-09-21T05:42:41.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.282+0000 7f93bb7fe640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93a8078b90 0x7f93a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:41.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.282+0000 7f93bb7fe640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93a8078b90 0x7f93a807b0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005d70 tx=0x7f93ac005d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:41.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.286+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93b0020020 con 0x7f93bc0b4d30 2024-09-21T05:42:41.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.518+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93bc0cc040 con 0x7f93bc0b4d30 2024-09-21T05:42:41.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.518+0000 7f93a3fff640 1 -- 172.21.15.22:0/1672510470 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5222 (secure 0 0 0) 0x7f93b0062c30 con 0x7f93bc0b4d30 2024-09-21T05:42:41.520 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:41.521 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":50,"btime":"2024-09-21T05:42:41:358457+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:40.361262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-21T05:42:41.521 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 50 2024-09-21T05:42:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93a8078b90 msgr2=0x7f93a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93a8078b90 0x7f93a807b0a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005d70 tx=0x7f93ac005d00 comp rx=0 tx=0).stop 2024-09-21T05:42:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 msgr2=0x7f93bc14f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 0x7f93bc14f330 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f93b000b420 tx=0x7f93b000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 shutdown_connections 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93a8078b90 0x7f93a807b0a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93bc0cd220 0x7f93bc0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93bc0b4d30 0x7f93bc14f330 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 --2- 172.21.15.22:0/1672510470 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93bc06ee90 0x7f93bc14edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 >> 172.21.15.22:0/1672510470 conn(0x7f93bc06d7d0 msgr2=0x7f93bc0ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 shutdown_connections 2024-09-21T05:42:41.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:41.521+0000 7f93c1b4a640 1 -- 172.21.15.22:0/1672510470 wait complete. 2024-09-21T05:42:41.937 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:42:29.001833+0000', 'modified': '2024-09-21T05:42:40.361262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6153}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6153': {'gid': 6153, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.134:6836/2827334455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2827334455}, {'type': 'v1', 'addr': '172.21.15.134:6836', 'nonce': 2827334455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:42:41.937 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6153:up:rejoin 2024-09-21T05:42:42.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:42:43.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/886204012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d0340 msgr2=0x7f3f1415e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/886204012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d0340 0x7f3f1415e940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f10005fd0 tx=0x7f3f1002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/886204012 shutdown_connections 2024-09-21T05:42:43.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/886204012 >> v2:172.21.15.134:3300/0 conn(0x7f3f140d2910 0x7f3f140b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/886204012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d0340 0x7f3f1415e940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/886204012 >> 172.21.15.22:0/886204012 conn(0x7f3f140b3f10 msgr2=0x7f3f140b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/886204012 shutdown_connections 2024-09-21T05:42:43.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.068+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/886204012 wait complete. 2024-09-21T05:42:43.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.070+0000 7f3f1c0a2640 1 Processor -- start 2024-09-21T05:42:43.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.070+0000 7f3f1c0a2640 1 -- start start 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1c0a2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f140d0340 0x7f3f141e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1c0a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 0x7f3f140af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1c0a2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d2910 0x7f3f140b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1c0a2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3f140b5ef0 con 0x7f3f140d08e0 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1c0a2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3f140b5d70 con 0x7f3f140d0340 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1c0a2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3f140b6070 con 0x7f3f140d2910 2024-09-21T05:42:43.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 0x7f3f140af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 0x7f3f140af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34760/0 (socket says 172.21.15.22:34760) 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 -- 172.21.15.22:0/2566075262 learned_addr learned my addr 172.21.15.22:0/2566075262 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f19e3e640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f140d0340 0x7f3f141e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1a63f640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d2910 0x7f3f140b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 -- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d2910 msgr2=0x7f3f140b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d2910 0x7f3f140b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 -- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f140d0340 msgr2=0x7f3f141e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f140d0340 0x7f3f141e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 -- 172.21.15.22:0/2566075262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1415bb70 con 0x7f3f140d08e0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.072+0000 7f3f1963d640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 0x7f3f140af2d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c400 tx=0x7f3f0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.073+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04017070 con 0x7f3f140d08e0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.073+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f040053c0 con 0x7f3f140d08e0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.073+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f141e3f60 con 0x7f3f140d08e0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.073+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04003850 con 0x7f3f140d08e0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.073+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f141e4520 con 0x7f3f140d08e0 2024-09-21T05:42:43.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.074+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f140b6590 con 0x7f3f140d08e0 2024-09-21T05:42:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.075+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f0401a070 con 0x7f3f140d08e0 2024-09-21T05:42:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.075+0000 7f3f0a7fc640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ef8078b90 0x7f3ef807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.075+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f3f0409a170 con 0x7f3f140d08e0 2024-09-21T05:42:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.075+0000 7f3f19e3e640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ef8078b90 0x7f3ef807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.076+0000 7f3f19e3e640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ef8078b90 0x7f3ef807b0a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f10034000 tx=0x7f3f10004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.080+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f04061f00 con 0x7f3f140d08e0 2024-09-21T05:42:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.305+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f140b4b30 con 0x7f3f140d08e0 2024-09-21T05:42:43.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.305+0000 7f3f0a7fc640 1 -- 172.21.15.22:0/2566075262 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5229 (secure 0 0 0) 0x7f3f04066930 con 0x7f3f140d08e0 2024-09-21T05:42:43.307 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:42:43.308 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":51,"btime":"2024-09-21T05:42:42:360219+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:42.360210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":7,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:42:43.308 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 51 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ef8078b90 msgr2=0x7f3ef807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ef8078b90 0x7f3ef807b0a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f10034000 tx=0x7f3f10004ab0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 msgr2=0x7f3f140af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 0x7f3f140af2d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c400 tx=0x7f3f0400c8c0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 shutdown_connections 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ef8078b90 0x7f3ef807b0a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3f140d2910 0x7f3f140b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3f140d08e0 0x7f3f140af2d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 --2- 172.21.15.22:0/2566075262 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3f140d0340 0x7f3f141e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.310 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.307+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 >> 172.21.15.22:0/2566075262 conn(0x7f3f140b3f10 msgr2=0x7f3f140cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.308+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 shutdown_connections 2024-09-21T05:42:43.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.308+0000 7f3f1c0a2640 1 -- 172.21.15.22:0/2566075262 wait complete. 2024-09-21T05:42:43.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:42:29.001833+0000', 'modified': '2024-09-21T05:42:42.360210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6153}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6153': {'gid': 6153, 'name': 'd', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.134:6836/2827334455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6834', 'nonce': 2827334455}, {'type': 'v1', 'addr': '172.21.15.134:6836', 'nonce': 2827334455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6153, 'qdb_cluster': [6153]} 2024-09-21T05:42:43.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:42:43.724 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T05:42:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 -- 172.21.15.22:0/395287520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9740cad30 msgr2=0x7fe9740cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/395287520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9740cad30 0x7fe9740cba00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005fd0 tx=0x7fe96c02a940 comp rx=0 tx=0).stop 2024-09-21T05:42:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 -- 172.21.15.22:0/395287520 shutdown_connections 2024-09-21T05:42:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/395287520 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe9740cad30 0x7fe9740cba00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 -- 172.21.15.22:0/395287520 >> 172.21.15.22:0/395287520 conn(0x7fe9740b1cb0 msgr2=0x7fe9740b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 -- 172.21.15.22:0/395287520 shutdown_connections 2024-09-21T05:42:43.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.849+0000 7fe97a6a4640 1 -- 172.21.15.22:0/395287520 wait complete. 2024-09-21T05:42:43.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.851+0000 7fe97a6a4640 1 Processor -- start 2024-09-21T05:42:43.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.851+0000 7fe97a6a4640 1 -- start start 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.852+0000 7fe97a6a4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9740ca800 0x7fe9741587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.852+0000 7fe97a6a4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9740cad30 0x7fe974155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.852+0000 7fe97a6a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 0x7fe9740b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.852+0000 7fe97a6a4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe974074130 con 0x7fe974158e80 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.852+0000 7fe97a6a4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe974073fb0 con 0x7fe9740cad30 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.852+0000 7fe97a6a4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9740742b0 con 0x7fe9740ca800 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 0x7fe9740b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 0x7fe9740b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34780/0 (socket says 172.21.15.22:34780) 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 -- 172.21.15.22:0/611038407 learned_addr learned my addr 172.21.15.22:0/611038407 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe9737fe640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9740cad30 0x7fe974155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe973fff640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9740ca800 0x7fe9741587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 -- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9740ca800 msgr2=0x7fe9741587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9740ca800 0x7fe9741587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 -- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9740cad30 msgr2=0x7fe974155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9740cad30 0x7fe974155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 -- 172.21.15.22:0/611038407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe974155bc0 con 0x7fe974158e80 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe978c41640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 0x7fe9740b0e10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006010 tx=0x7fe96c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe970ff9640 1 -- 172.21.15.22:0/611038407 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c03e070 con 0x7fe974158e80 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe970ff9640 1 -- 172.21.15.22:0/611038407 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe96c004540 con 0x7fe974158e80 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9741f7390 con 0x7fe974158e80 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.853+0000 7fe970ff9640 1 -- 172.21.15.22:0/611038407 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c043460 con 0x7fe974158e80 2024-09-21T05:42:43.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.854+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97409ae90 con 0x7fe974158e80 2024-09-21T05:42:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.855+0000 7fe970ff9640 1 -- 172.21.15.22:0/611038407 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe96c043600 con 0x7fe974158e80 2024-09-21T05:42:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.856+0000 7fe970ff9640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe960078b90 0x7fe96007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.856+0000 7fe970ff9640 1 -- 172.21.15.22:0/611038407 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fe96c0bc890 con 0x7fe974158e80 2024-09-21T05:42:43.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.856+0000 7fe973fff640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe960078b90 0x7fe96007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.857+0000 7fe98160e740 1 Processor -- start 2024-09-21T05:42:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.857+0000 7fe98160e740 1 -- start start 2024-09-21T05:42:43.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.857+0000 7fe973fff640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe960078b90 0x7fe96007b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe964000f60 tx=0x7fe964009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd074fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe98160e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd0923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe98160e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bebcfbd820 con 0x55bebcc7a3b0 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bebcfacc50 con 0x55bebcca3270 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bebcfb0a60 con 0x55bebcc82590 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0754e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34788/0 (socket says 172.21.15.22:34788) 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 -- 172.21.15.22:0/4281405428 learned_addr learned my addr 172.21.15.22:0/4281405428 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe973fff640 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd074fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe978c41640 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd0923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 -- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 msgr2=0x55bebd0923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd0923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 -- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 msgr2=0x55bebd074fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd074fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.858+0000 7fe9737fe640 1 -- 172.21.15.22:0/4281405428 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebcd0e850 con 0x55bebcc7a3b0 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.859+0000 7fe9737fe640 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0754e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe968001720 tx=0x7fe968001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1970ea0c22a365 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.859+0000 7fe959ffb640 1 -- 172.21.15.22:0/4281405428 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9680052e0 con 0x55bebcc7a3b0 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.859+0000 7fe959ffb640 1 -- 172.21.15.22:0/4281405428 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe968018070 con 0x55bebcc7a3b0 2024-09-21T05:42:43.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.859+0000 7fe959ffb640 1 -- 172.21.15.22:0/4281405428 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96800fa10 con 0x55bebcc7a3b0 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.859+0000 7fe98160e740 1 -- 172.21.15.22:0/4281405428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 msgr2=0x55bebd0754e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.859+0000 7fe98160e740 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0754e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe968001720 tx=0x7fe968001be0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 -- 172.21.15.22:0/4281405428 shutdown_connections 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd0923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0754e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 --2- 172.21.15.22:0/4281405428 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 -- 172.21.15.22:0/4281405428 >> 172.21.15.22:0/4281405428 conn(0x55bebcc973c0 msgr2=0x55bebd04b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 -- 172.21.15.22:0/4281405428 shutdown_connections 2024-09-21T05:42:43.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 -- 172.21.15.22:0/4281405428 wait complete. 2024-09-21T05:42:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.860+0000 7fe98160e740 1 -- start start 2024-09-21T05:42:43.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.861+0000 7fe98160e740 1 Processor -- start 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd05e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd087fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bebcd1d620 con 0x55bebcc7a3b0 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bebcf525f0 con 0x55bebcca3270 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bebd075a20 con 0x55bebcc82590 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe973fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe98160e740 10 client.0 _ll_register_callbacks cb 0x55bebcf560a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe973fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0a2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34798/0 (socket says 172.21.15.22:34798) 2024-09-21T05:42:43.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe973fff640 1 -- 172.21.15.22:0/551015025 learned_addr learned my addr 172.21.15.22:0/551015025 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe978c41640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd087fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.862+0000 7fe9737fe640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd05e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe973fff640 1 -- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 msgr2=0x55bebd05e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe973fff640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd05e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe973fff640 1 -- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 msgr2=0x55bebd087fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe973fff640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd087fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe973fff640 1 -- 172.21.15.22:0/551015025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebcd856b0 con 0x55bebcc7a3b0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe978c41640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd087fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe973fff640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0a2c30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe964014700 tx=0x7fe964001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944710f63a3b6e65 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe952ffd640 1 -- 172.21.15.22:0/551015025 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964025020 con 0x55bebcc7a3b0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe952ffd640 1 -- 172.21.15.22:0/551015025 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fe964013680 con 0x55bebcc7a3b0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe968018070 con 0x55bebcc7a3b0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe952ffd640 1 -- 172.21.15.22:0/551015025 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe96402f050 con 0x55bebcc7a3b0 2024-09-21T05:42:43.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe98160e740 10 client.6342 fetch_fsmap learned FSMap version 51 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55bebd087a80 con 0x55bebcc7a3b0 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe952ffd640 1 -- 172.21.15.22:0/551015025 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe968018070 con 0x55bebcc7a3b0 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.863+0000 7fe952ffd640 1 -- 172.21.15.22:0/551015025 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7fe964032020 con 0x55bebcc7a3b0 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe98160e740 10 client.6342 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe98160e740 10 client.6342 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe98160e740 20 client.6342 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe98160e740 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebcc61e40 0x55bebd09ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe98160e740 4 client.6342 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bebcfbf770 con 0x55bebcc61e40 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe9737fe640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebcc61e40 0x55bebd09ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe9737fe640 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebcc61e40 0x55bebd09ecd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d522fdf18413676c server_cookie=9c84be212bac8647 in_seq=0 out_seq=0 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.864+0000 7fe952ffd640 10 client.6342 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe952ffd640 1 -- 172.21.15.22:0/551015025 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bebcfbf770 con 0x55bebcc61e40 2024-09-21T05:42:43.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe952ffd640 10 client.6342 handle_command_reply: tid=0 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 client.6342 shutdown 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 msgr2=0x55bebd0a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0a2c30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe964014700 tx=0x7fe964001080 comp rx=0 tx=0).stop 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 shutdown_connections 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebcc61e40 0x55bebd09ecd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd087fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd05e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe952ffd640 0 client.6342 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 --2- 172.21.15.22:0/551015025 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd0a2c30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 >> 172.21.15.22:0/551015025 conn(0x55bebcc973c0 msgr2=0x55bebd04b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.865+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 shutdown_connections 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.866+0000 7fe98160e740 1 -- 172.21.15.22:0/551015025 wait complete. 2024-09-21T05:42:43.871 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.866+0000 7fe98160e740 20 client.6342 trim_cache size 0 max 16384 2024-09-21T05:42:43.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.884+0000 7fe98160e740 1 Processor -- start 2024-09-21T05:42:43.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.884+0000 7fe98160e740 1 -- start start 2024-09-21T05:42:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.885+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc7a3b0 0x55bebd0450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.885+0000 7fe98160e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 0x55bebd0a2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.885+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd0a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.885+0000 7fe98160e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe9741fefa0 con 0x55bebcc82590 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.885+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe9740ada40 con 0x55bebcca3270 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.885+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe9741fea30 con 0x55bebcc7a3b0 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 0x55bebd0a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 0x55bebd0a2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34802/0 (socket says 172.21.15.22:34802) 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 -- 172.21.15.22:0/2598867612 learned_addr learned my addr 172.21.15.22:0/2598867612 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe978c41640 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd0a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 -- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc7a3b0 msgr2=0x55bebd0450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc7a3b0 0x55bebd0450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 -- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 msgr2=0x55bebd0a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd0a2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 -- 172.21.15.22:0/2598867612 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebd04cbc0 con 0x55bebcc82590 2024-09-21T05:42:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 0x55bebd0a2200 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe968001720 tx=0x7fe968007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280c63116a4743f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9597fa640 1 -- 172.21.15.22:0/2598867612 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96800fc40 con 0x55bebcc82590 2024-09-21T05:42:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9597fa640 1 -- 172.21.15.22:0/2598867612 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe96802c030 con 0x55bebcc82590 2024-09-21T05:42:43.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.886+0000 7fe9597fa640 1 -- 172.21.15.22:0/2598867612 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968007280 con 0x55bebcc82590 2024-09-21T05:42:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 -- 172.21.15.22:0/2598867612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 msgr2=0x55bebd0a2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 0x55bebd0a2200 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe968001720 tx=0x7fe968007e30 comp rx=0 tx=0).stop 2024-09-21T05:42:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 -- 172.21.15.22:0/2598867612 shutdown_connections 2024-09-21T05:42:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd0a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc82590 0x55bebd0a2200 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 --2- 172.21.15.22:0/2598867612 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc7a3b0 0x55bebd0450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 -- 172.21.15.22:0/2598867612 >> 172.21.15.22:0/2598867612 conn(0x55bebcc973c0 msgr2=0x55bebd04c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 -- 172.21.15.22:0/2598867612 shutdown_connections 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.887+0000 7fe98160e740 1 -- 172.21.15.22:0/2598867612 wait complete. 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.888+0000 7fe98160e740 1 -- start start 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.888+0000 7fe98160e740 1 Processor -- start 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd10d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd114090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd1145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bebd09ffe0 con 0x55bebcc7a3b0 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bebd0885f0 con 0x55bebcca3270 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bebd0a05d0 con 0x55bebcc82590 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe98160e740 10 client.0 _ll_register_callbacks cb 0x7fe964001d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.889+0000 7fe973fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd10d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34816/0 (socket says 172.21.15.22:34816) 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 -- 172.21.15.22:0/2138545954 learned_addr learned my addr 172.21.15.22:0/2138545954 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe978c41640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd1145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 -- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 msgr2=0x55bebd114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 -- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 msgr2=0x55bebd1145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd1145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 -- 172.21.15.22:0/2138545954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebd08e6f0 con 0x55bebcc7a3b0 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe978c41640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd1145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe973fff640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd10d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe964016010 tx=0x7fe964013e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4da3c91542f8cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe952ffd640 1 -- 172.21.15.22:0/2138545954 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964025070 con 0x55bebcc7a3b0 2024-09-21T05:42:43.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe952ffd640 1 -- 172.21.15.22:0/2138545954 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fe964021810 con 0x55bebcc7a3b0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe96802c030 con 0x55bebcc7a3b0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe952ffd640 1 -- 172.21.15.22:0/2138545954 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fe964040050 con 0x55bebcc7a3b0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe98160e740 10 client.6348 fetch_fsmap learned FSMap version 51 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.890+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55bebd062290 con 0x55bebcc7a3b0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe952ffd640 1 -- 172.21.15.22:0/2138545954 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe96802c030 con 0x55bebcc7a3b0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe952ffd640 1 -- 172.21.15.22:0/2138545954 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7fe964043020 con 0x55bebcc7a3b0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe98160e740 10 client.6348 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe98160e740 10 client.6348 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe98160e740 20 client.6348 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe98160e740 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebd1170f0 0x55bebd117500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe98160e740 4 client.6348 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "status"}) -- 0x55bebd09e4f0 con 0x55bebd1170f0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebd1170f0 0x55bebd117500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:43.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe952ffd640 10 client.6348 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.891+0000 7fe9737fe640 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebd1170f0 0x55bebd117500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258caac5d60d1767 server_cookie=1efd311c0552292e in_seq=0 out_seq=0 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe952ffd640 1 -- 172.21.15.22:0/2138545954 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55bebd09e4f0 con 0x55bebd1170f0 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe952ffd640 10 client.6348 handle_command_reply: tid=0 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 client.6348 shutdown 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 msgr2=0x55bebd10d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd10d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe964016010 tx=0x7fe964013e00 comp rx=0 tx=0).stop 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 shutdown_connections 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55bebd1170f0 0x55bebd117500 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bebcca3270 0x55bebd1145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe952ffd640 0 client.6348 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bebcc82590 0x55bebd114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 --2- 172.21.15.22:0/2138545954 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bebcc7a3b0 0x55bebd10d990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 >> 172.21.15.22:0/2138545954 conn(0x55bebcc973c0 msgr2=0x55bebd04b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 shutdown_connections 2024-09-21T05:42:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 1 -- 172.21.15.22:0/2138545954 wait complete. 2024-09-21T05:42:43.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.892+0000 7fe98160e740 20 client.6348 trim_cache size 0 max 16384 2024-09-21T05:42:43.900 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:42:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:42:43.900 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 6153, 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 4.5355776419999998, 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 51, 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 69, 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 69, 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 18.907613250000001 2024-09-21T05:42:43.901 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe960078b90 msgr2=0x7fe96007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe960078b90 0x7fe96007b0a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe964000f60 tx=0x7fe964009040 comp rx=0 tx=0).stop 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 msgr2=0x7fe9740b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 0x7fe9740b0e10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006010 tx=0x7fe96c031e20 comp rx=0 tx=0).stop 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 shutdown_connections 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe974158e80 0x7fe9740b0e10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe9740cad30 0x7fe974155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe960078b90 0x7fe96007b0a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 --2- 172.21.15.22:0/611038407 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe9740ca800 0x7fe9741587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 >> 172.21.15.22:0/611038407 conn(0x7fe9740b1cb0 msgr2=0x7fe9740b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 shutdown_connections 2024-09-21T05:42:43.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:43.893+0000 7fe97a6a4640 1 -- 172.21.15.22:0/611038407 wait complete. 2024-09-21T05:42:43.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:42:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 -- 172.21.15.22:0/1843130836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46640b9b60 msgr2=0x7f46640d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1843130836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46640b9b60 0x7f46640d4d80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4660005fd0 tx=0x7f466002a8a0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 -- 172.21.15.22:0/1843130836 shutdown_connections 2024-09-21T05:42:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1843130836 >> v2:172.21.15.134:3300/0 conn(0x7f46640b66b0 0x7f466406eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1843130836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46640b9b60 0x7f46640d4d80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 -- 172.21.15.22:0/1843130836 >> 172.21.15.22:0/1843130836 conn(0x7f466409d7f0 msgr2=0x7f46640d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 -- 172.21.15.22:0/1843130836 shutdown_connections 2024-09-21T05:42:44.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.021+0000 7f466b3c3640 1 -- 172.21.15.22:0/1843130836 wait complete. 2024-09-21T05:42:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.023+0000 7f466b3c3640 1 Processor -- start 2024-09-21T05:42:44.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.023+0000 7f466b3c3640 1 -- start start 2024-09-21T05:42:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466b3c3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b9b60 0x7f46640cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466b3c3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640cf540 0x7f466415a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466b3c3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 0x7f46641f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:44.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466b3c3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f466406f600 con 0x7f46641f2920 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466b3c3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f466406f480 con 0x7f46640b9b60 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466b3c3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f466406f780 con 0x7f46640cf540 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f4669960640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 0x7f46641f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f4669960640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 0x7f46641f8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34830/0 (socket says 172.21.15.22:34830) 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f4669960640 1 -- 172.21.15.22:0/1872009616 learned_addr learned my addr 172.21.15.22:0/1872009616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466915f640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b9b60 0x7f46640cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.024+0000 7f466895e640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640cf540 0x7f466415a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4669960640 1 -- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640cf540 msgr2=0x7f466415a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4669960640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640cf540 0x7f466415a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4669960640 1 -- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b9b60 msgr2=0x7f46640cab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4669960640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b9b60 0x7f46640cab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4669960640 1 -- 172.21.15.22:0/1872009616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46640cc700 con 0x7f46641f2920 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4669960640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 0x7f46641f8ae0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c940 tx=0x7f465c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c004460 con 0x7f46641f2920 2024-09-21T05:42:44.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f465c0077a0 con 0x7f46641f2920 2024-09-21T05:42:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46640ccec0 con 0x7f46641f2920 2024-09-21T05:42:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c007c10 con 0x7f46641f2920 2024-09-21T05:42:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.025+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46640cd480 con 0x7f46641f2920 2024-09-21T05:42:44.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.026+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466406fd00 con 0x7f46641f2920 2024-09-21T05:42:44.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.027+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f465c0035f0 con 0x7f46641f2920 2024-09-21T05:42:44.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.027+0000 7f4659ffb640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4648078bc0 0x7f464807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:44.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.027+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f465c09b400 con 0x7f46641f2920 2024-09-21T05:42:44.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.028+0000 7f466915f640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4648078bc0 0x7f464807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:44.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.028+0000 7f466915f640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4648078bc0 0x7f464807b0d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4660035000 tx=0x7f4660004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:44.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.032+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f465c013030 con 0x7f46641f2920 2024-09-21T05:42:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.257+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46640d3fb0 con 0x7f46641f2920 2024-09-21T05:42:44.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.257+0000 7f4659ffb640 1 -- 172.21.15.22:0/1872009616 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7f465c063190 con 0x7f46641f2920 2024-09-21T05:42:44.260 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:42:44.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4648078bc0 msgr2=0x7f464807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:44.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4648078bc0 0x7f464807b0d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4660035000 tx=0x7f4660004ab0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 msgr2=0x7f46641f8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:44.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 0x7f46641f8ae0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c940 tx=0x7f465c00ce00 comp rx=0 tx=0).stop 2024-09-21T05:42:44.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 shutdown_connections 2024-09-21T05:42:44.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46641f2920 0x7f46641f8ae0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4648078bc0 0x7f464807b0d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f46640cf540 0x7f466415a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 --2- 172.21.15.22:0/1872009616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f46640b9b60 0x7f46640cab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:44.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 >> 172.21.15.22:0/1872009616 conn(0x7f466409d7f0 msgr2=0x7f4664160ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:44.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 shutdown_connections 2024-09-21T05:42:44.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44.260+0000 7f466b3c3640 1 -- 172.21.15.22:0/1872009616 wait complete. 2024-09-21T05:42:44.273 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:42:44.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:42:44.273 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:42:44.273 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:42:44.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:44.273 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:42:44.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:42:44.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:42:44.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:42:44.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.274 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:42:44.290 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:42:44.290 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:42:44.290 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:42:44.290 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42580sec preferred_lft 42580sec 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:42:44.291 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:42:44.292 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:42:44.293 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:42:44.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.293 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:42:44.355 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:42:44.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.357 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:42:44.414 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:42:44.414 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:42:44.414 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:42:44.414 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:42:44.414 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:42:44.415 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:42:44.415 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:42:44.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:42:44.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:42:44.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:42:44.415 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:42:44.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:42:44.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:42:44.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.543 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:42:44.559 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:42:44.559 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:42:44.559 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:42:44.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:42:44.625 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:42:44.625 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:44.681 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:42:44.681 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:42:44.681 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:44.736 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:42:44.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:44.831 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:44.831 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:42:44.831 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:42:44.842 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:44.842 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:42:44.842 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:42:44.842 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:42:44.842 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:42:44.853 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:42:44.853 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:42:44.853 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:42:44.854 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:42:44.855 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:42:44.855 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:42:44.855 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:42:44.856 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:42:44.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.856 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:44.881 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:42:44.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:42:44.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:42:44.979 INFO:teuthology.orchestra.run.smithi022.stdout:client6357 2024-09-21T05:42:44.980 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:44.980 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:44.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:44.996 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-21T05:42:45.054 INFO:teuthology.orchestra.run.smithi022.stdout:file_undamaged 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> import json 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> import sys 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> try: 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> except OSError as e: 2024-09-21T05:42:45.055 DEBUG:teuthology.orchestra.run.smithi022:> sys.exit(e.errno) 2024-09-21T05:42:45.056 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:42:45.056 DEBUG:teuthology.orchestra.run.smithi022:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-21T05:42:45.056 DEBUG:teuthology.orchestra.run.smithi022:> print(json.dumps( 2024-09-21T05:42:45.056 DEBUG:teuthology.orchestra.run.smithi022:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-21T05:42:45.056 DEBUG:teuthology.orchestra.run.smithi022:> indent=2)) 2024-09-21T05:42:45.056 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:42:45.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-21T05:42:45.172 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T05:42:45.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:45.172 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-21T05:42:45.230 INFO:teuthology.orchestra.run.smithi022.stdout:2 2024-09-21T05:42:45.231 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:45.231 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:45.288 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:42:45.288 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:42:45.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:42:45.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 -- 172.21.15.22:0/4181867879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e415ff30 msgr2=0x7f01e4155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 --2- 172.21.15.22:0/4181867879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e415ff30 0x7f01e4155b30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009830 tx=0x7f01d402a510 comp rx=0 tx=0).stop 2024-09-21T05:42:45.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 -- 172.21.15.22:0/4181867879 shutdown_connections 2024-09-21T05:42:45.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 --2- 172.21.15.22:0/4181867879 >> v2:172.21.15.134:3300/0 conn(0x7f01e4073e70 0x7f01e4074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 --2- 172.21.15.22:0/4181867879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e415ff30 0x7f01e4155b30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 -- 172.21.15.22:0/4181867879 >> 172.21.15.22:0/4181867879 conn(0x7f01e40cdc60 msgr2=0x7f01e40ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 -- 172.21.15.22:0/4181867879 shutdown_connections 2024-09-21T05:42:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.500+0000 7f01e9030640 1 -- 172.21.15.22:0/4181867879 wait complete. 2024-09-21T05:42:45.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.502+0000 7f01e9030640 1 Processor -- start 2024-09-21T05:42:45.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.502+0000 7f01e9030640 1 -- start start 2024-09-21T05:42:45.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e9030640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e4073e70 0x7f01e415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e9030640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e415c9b0 0x7f01e415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e9030640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 0x7f01e41f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e9030640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40d0620 con 0x7f01e415ff30 2024-09-21T05:42:45.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e9030640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f01e40d04a0 con 0x7f01e415c9b0 2024-09-21T05:42:45.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e9030640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f01e40d07a0 con 0x7f01e4073e70 2024-09-21T05:42:45.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 0x7f01e41f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e359e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 0x7f01e41f92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36052/0 (socket says 172.21.15.22:36052) 2024-09-21T05:42:45.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e359e640 1 -- 172.21.15.22:0/4293185047 learned_addr learned my addr 172.21.15.22:0/4293185047 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e2d9d640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e4073e70 0x7f01e415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.503+0000 7f01e259c640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e415c9b0 0x7f01e415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e359e640 1 -- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e4073e70 msgr2=0x7f01e415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e359e640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e4073e70 0x7f01e415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e359e640 1 -- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e415c9b0 msgr2=0x7f01e415ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e359e640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e415c9b0 0x7f01e415ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e359e640 1 -- 172.21.15.22:0/4293185047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e414eb10 con 0x7f01e415ff30 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e359e640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 0x7f01e41f92a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ec50 tx=0x7f01dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc003b70 con 0x7f01e415ff30 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01dc003d10 con 0x7f01e415ff30 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41f67e0 con 0x7f01e415ff30 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc004280 con 0x7f01e415ff30 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.504+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41f5d00 con 0x7f01e415ff30 2024-09-21T05:42:45.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.505+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e40d0cc0 con 0x7f01e415ff30 2024-09-21T05:42:45.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.506+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01dc004420 con 0x7f01e415ff30 2024-09-21T05:42:45.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.506+0000 7f01cb7fe640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01d0078b90 0x7f01d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.507+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f01dc09bbc0 con 0x7f01e415ff30 2024-09-21T05:42:45.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.507+0000 7f01e2d9d640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01d0078b90 0x7f01d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.507+0000 7f01e2d9d640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01d0078b90 0x7f01d007b0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f01d4034000 tx=0x7f01d4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:45.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.511+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f01dc063950 con 0x7f01e415ff30 2024-09-21T05:42:45.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.711+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01e415b9e0 con 0x7f01e415ff30 2024-09-21T05:42:45.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.712+0000 7f01cb7fe640 1 -- 172.21.15.22:0/4293185047 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v69) ==== 80+0+124 (secure 0 0 0) 0x7f01dc068380 con 0x7f01e415ff30 2024-09-21T05:42:45.714 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6837/1820431436 2024-09-22T05:42:38.349537+0000 2024-09-21T05:42:45.714 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6835/1820431436 2024-09-22T05:42:38.349537+0000 2024-09-21T05:42:45.714 INFO:teuthology.orchestra.run.smithi022.stderr:listed 2 entries 2024-09-21T05:42:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01d0078b90 msgr2=0x7f01d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01d0078b90 0x7f01d007b0a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f01d4034000 tx=0x7f01d4004730 comp rx=0 tx=0).stop 2024-09-21T05:42:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 msgr2=0x7f01e41f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 0x7f01e41f92a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00ec50 tx=0x7f01dc00c6a0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 shutdown_connections 2024-09-21T05:42:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f01e415ff30 0x7f01e41f92a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f01e415c9b0 0x7f01e415ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f01d0078b90 0x7f01d007b0a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 --2- 172.21.15.22:0/4293185047 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f01e4073e70 0x7f01e415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 >> 172.21.15.22:0/4293185047 conn(0x7f01e40cdc60 msgr2=0x7f01e415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 shutdown_connections 2024-09-21T05:42:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.714+0000 7f01e9030640 1 -- 172.21.15.22:0/4293185047 wait complete. 2024-09-21T05:42:45.727 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:42:45.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:45.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:42:45.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:42:45.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:45.757 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:42:45.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:45.813 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-09-21T05:42:45.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/2916316335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00cf280 msgr2=0x7f2ba00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/2916316335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00cf280 0x7f2ba00cf690 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2ba00d5530 tx=0x7f2b9802a9e0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/2916316335 shutdown_connections 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/2916316335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00cf280 0x7f2ba00cf690 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/2916316335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba00d24a0 0x7f2ba00d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/2916316335 >> 172.21.15.22:0/2916316335 conn(0x7f2ba00d45c0 msgr2=0x7f2ba00d49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/2916316335 shutdown_connections 2024-09-21T05:42:45.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.972+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/2916316335 wait complete. 2024-09-21T05:42:45.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.974+0000 7f2ba5f4a640 1 Processor -- start 2024-09-21T05:42:45.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.974+0000 7f2ba5f4a640 1 -- start start 2024-09-21T05:42:45.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2ba5f4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba00cf280 0x7f2ba0161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2ba5f4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 0x7f2ba00b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2ba5f4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba015eb70 0x7f2ba00b6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2ba5f4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba00b2aa0 con 0x7f2ba00d24a0 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2ba5f4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba00b2920 con 0x7f2ba00cf280 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2ba5f4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba00b2c20 con 0x7f2ba015eb70 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 0x7f2ba00b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 0x7f2ba00b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36074/0 (socket says 172.21.15.22:36074) 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.975+0000 7f2b9effd640 1 -- 172.21.15.22:0/3572333668 learned_addr learned my addr 172.21.15.22:0/3572333668 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:45.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba00cf280 0x7f2ba0161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9ffff640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba015eb70 0x7f2ba00b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9effd640 1 -- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba015eb70 msgr2=0x7f2ba00b6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9effd640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba015eb70 0x7f2ba00b6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9effd640 1 -- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba00cf280 msgr2=0x7f2ba0161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9effd640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba00cf280 0x7f2ba0161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9effd640 1 -- 172.21.15.22:0/3572333668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba00cc4d0 con 0x7f2ba00d24a0 2024-09-21T05:42:45.981 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba00cf280 0x7f2ba0161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9ffff640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba015eb70 0x7f2ba00b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b9effd640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 0x7f2ba00b5f10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d700 tx=0x7f2b9400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b87fff640 1 -- 172.21.15.22:0/3572333668 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94017070 con 0x7f2ba00d24a0 2024-09-21T05:42:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b87fff640 1 -- 172.21.15.22:0/3572333668 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b94005310 con 0x7f2ba00d24a0 2024-09-21T05:42:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba015f4a0 con 0x7f2ba00d24a0 2024-09-21T05:42:45.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2b87fff640 1 -- 172.21.15.22:0/3572333668 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94009820 con 0x7f2ba00d24a0 2024-09-21T05:42:45.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.976+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba015fa60 con 0x7f2ba00d24a0 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.978+0000 7f2b87fff640 1 -- 172.21.15.22:0/3572333668 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2b940099c0 con 0x7f2ba00d24a0 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.979+0000 7f2b87fff640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b8c078b90 0x7f2b8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.979+0000 7f2b87fff640 1 -- 172.21.15.22:0/3572333668 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f2b9409aed0 con 0x7f2ba00d24a0 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.979+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b8c078b90 0x7f2b8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.979+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b8c078b90 0x7f2b8c07b0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009890 tx=0x7f2b90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.980+0000 7f2baceb3740 1 Processor -- start 2024-09-21T05:42:45.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.980+0000 7f2baceb3740 1 -- start start 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76ebe5b0 0x559f772ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f7728fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e9d8d0 0x559f77290280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2baceb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559f771d8a80 con 0x559f76e956f0 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f771c7eb0 con 0x559f76ebe5b0 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559f771cbcc0 con 0x559f76e9d8d0 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f7728fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f7728fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36088/0 (socket says 172.21.15.22:36088) 2024-09-21T05:42:45.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2b9effd640 1 -- 172.21.15.22:0/3551406193 learned_addr learned my addr 172.21.15.22:0/3551406193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76ebe5b0 0x559f772ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.981+0000 7f2b9ffff640 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e9d8d0 0x559f77290280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b9effd640 1 -- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e9d8d0 msgr2=0x559f77290280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b9effd640 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e9d8d0 0x559f77290280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b9effd640 1 -- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76ebe5b0 msgr2=0x559f772ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b9effd640 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76ebe5b0 0x559f772ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b9effd640 1 -- 172.21.15.22:0/3551406193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f76fa2a20 con 0x559f76e956f0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b9effd640 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f7728fd40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2b94099320 tx=0x7f2b94062770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22b8e380a612c42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b857fa640 1 -- 172.21.15.22:0/3551406193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94067070 con 0x559f76e956f0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b857fa640 1 -- 172.21.15.22:0/3551406193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b94062ba0 con 0x559f76e956f0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.982+0000 7f2b857fa640 1 -- 172.21.15.22:0/3551406193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94020070 con 0x559f76e956f0 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 -- 172.21.15.22:0/3551406193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 msgr2=0x559f7728fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f7728fd40 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2b94099320 tx=0x7f2b94062770 comp rx=0 tx=0).stop 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 -- 172.21.15.22:0/3551406193 shutdown_connections 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e9d8d0 0x559f77290280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f7728fd40 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 --2- 172.21.15.22:0/3551406193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76ebe5b0 0x559f772ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 -- 172.21.15.22:0/3551406193 >> 172.21.15.22:0/3551406193 conn(0x559f76eb2700 msgr2=0x559f77265f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 -- 172.21.15.22:0/3551406193 shutdown_connections 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.983+0000 7f2baceb3740 1 -- 172.21.15.22:0/3551406193 wait complete. 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.984+0000 7f2baceb3740 1 -- start start 2024-09-21T05:42:45.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.984+0000 7f2baceb3740 1 Processor -- start 2024-09-21T05:42:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f772aad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f772bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76ebe5b0 0x559f772bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559f76f38960 con 0x559f76e956f0 2024-09-21T05:42:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559f7714e670 con 0x559f76e9d8d0 2024-09-21T05:42:45.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559f7728cc00 con 0x559f76ebe5b0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2baceb3740 10 client.0 _ll_register_callbacks cb 0x559f771c13e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f772aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f772aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36098/0 (socket says 172.21.15.22:36098) 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.985+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1382353269 learned_addr learned my addr 172.21.15.22:0/1382353269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76ebe5b0 msgr2=0x559f772bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76ebe5b0 0x559f772bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9effd640 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f772bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 msgr2=0x559f772bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f772bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9f7fe640 1 -- 172.21.15.22:0/1382353269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f76f078c0 con 0x559f76e956f0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f772aad70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000da60 tx=0x7f2b900046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41292f8ae93eb29 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:45.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/1382353269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90027060 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b94062ba0 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/1382353269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f2b90006300 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/1382353269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2b9002d050 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2baceb3740 10 client.6372 fetch_fsmap learned FSMap version 51 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x559f770031b0 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.986+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/1382353269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b94062ba0 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/1382353269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7f2b90030020 con 0x559f76e956f0 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2baceb3740 10 client.6372 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2baceb3740 10 client.6372 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2baceb3740 20 client.6372 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:45.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2baceb3740 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e7d180 0x559f772bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2baceb3740 4 client.6372 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f771da9d0 con 0x559f76e7d180 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2b9effd640 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e7d180 0x559f772bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2b7e7fc640 10 client.6372 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.987+0000 7f2b9effd640 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e7d180 0x559f772bdd30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e5ce7737152e2f server_cookie=f4c37a3848100b09 in_seq=0 out_seq=0 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/1382353269 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f771da9d0 con 0x559f76e7d180 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2b7e7fc640 10 client.6372 handle_command_reply: tid=0 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 client.6372 shutdown 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 msgr2=0x559f772aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f772aad70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000da60 tx=0x7f2b900046d0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 shutdown_connections 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e7d180 0x559f772bdd30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76ebe5b0 0x559f772bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f772bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2b7e7fc640 0 client.6372 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 --2- 172.21.15.22:0/1382353269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f772aad70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 >> 172.21.15.22:0/1382353269 conn(0x559f76eb2700 msgr2=0x559f7728ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 shutdown_connections 2024-09-21T05:42:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.988+0000 7f2baceb3740 1 -- 172.21.15.22:0/1382353269 wait complete. 2024-09-21T05:42:45.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:45.989+0000 7f2baceb3740 20 client.6372 trim_cache size 0 max 16384 2024-09-21T05:42:46.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.007+0000 7f2baceb3740 1 Processor -- start 2024-09-21T05:42:46.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.007+0000 7f2baceb3740 1 -- start start 2024-09-21T05:42:46.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e7d180 0x559f772b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f77288040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f77288580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2baceb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba02022b0 con 0x559f76e956f0 2024-09-21T05:42:46.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba015ffd0 con 0x559f76e9d8d0 2024-09-21T05:42:46.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2ba01f9340 con 0x559f76e7d180 2024-09-21T05:42:46.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f77288040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2b9effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f77288040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36112/0 (socket says 172.21.15.22:36112) 2024-09-21T05:42:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.008+0000 7f2b9effd640 1 -- 172.21.15.22:0/2025692280 learned_addr learned my addr 172.21.15.22:0/2025692280 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b9effd640 1 -- 172.21.15.22:0/2025692280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e7d180 msgr2=0x559f772b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b9effd640 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e7d180 0x559f772b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b9effd640 1 -- 172.21.15.22:0/2025692280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 msgr2=0x559f77288580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b9effd640 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f77288580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b9effd640 1 -- 172.21.15.22:0/2025692280 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f76fa0d20 con 0x559f76e956f0 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b9effd640 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f77288040 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2b9405f120 tx=0x7f2b94085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9ff74021c3d929 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b84ff9640 1 -- 172.21.15.22:0/2025692280 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94020040 con 0x559f76e956f0 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b84ff9640 1 -- 172.21.15.22:0/2025692280 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b9404e120 con 0x559f76e956f0 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.009+0000 7f2b84ff9640 1 -- 172.21.15.22:0/2025692280 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94067040 con 0x559f76e956f0 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 -- 172.21.15.22:0/2025692280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 msgr2=0x559f77288040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f77288040 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2b9405f120 tx=0x7f2b94085040 comp rx=0 tx=0).stop 2024-09-21T05:42:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 -- 172.21.15.22:0/2025692280 shutdown_connections 2024-09-21T05:42:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e9d8d0 0x559f77288580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e956f0 0x559f77288040 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 --2- 172.21.15.22:0/2025692280 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e7d180 0x559f772b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 -- 172.21.15.22:0/2025692280 >> 172.21.15.22:0/2025692280 conn(0x559f76eb2700 msgr2=0x559f772a54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 -- 172.21.15.22:0/2025692280 shutdown_connections 2024-09-21T05:42:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.010+0000 7f2baceb3740 1 -- 172.21.15.22:0/2025692280 wait complete. 2024-09-21T05:42:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.011+0000 7f2baceb3740 1 -- start start 2024-09-21T05:42:46.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.011+0000 7f2baceb3740 1 Processor -- start 2024-09-21T05:42:46.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e7d180 0x559f7732e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e956f0 0x559f772aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 0x559f772b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559f7725bf20 con 0x559f76e9d8d0 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba01f9990 con 0x559f76e7d180 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559f77287d00 con 0x559f76e956f0 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2baceb3740 10 client.0 _ll_register_callbacks cb 0x7f2b90005960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:46.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2b9ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 0x559f772b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2b9ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 0x559f772b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36128/0 (socket says 172.21.15.22:36128) 2024-09-21T05:42:46.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2b9ffff640 1 -- 172.21.15.22:0/733018295 learned_addr learned my addr 172.21.15.22:0/733018295 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2b9effd640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e956f0 0x559f772aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.012+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e7d180 0x559f7732e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b9ffff640 1 -- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e956f0 msgr2=0x559f772aaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b9ffff640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e956f0 0x559f772aaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b9ffff640 1 -- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e7d180 msgr2=0x559f7732e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b9ffff640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e7d180 0x559f7732e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b9ffff640 1 -- 172.21.15.22:0/733018295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f772aca50 con 0x559f76e9d8d0 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b9ffff640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 0x559f772b9260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2b9802aa20 tx=0x7f2b9802f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131fb1833effefc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/733018295 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b98047060 con 0x559f76e9d8d0 2024-09-21T05:42:46.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/733018295 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7f2b98004240 con 0x559f76e9d8d0 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9404e120 con 0x559f76e9d8d0 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/733018295 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2b9804d050 con 0x559f76e9d8d0 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 10 client.6378 fetch_fsmap learned FSMap version 51 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x559f7732f6d0 con 0x559f76e9d8d0 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/733018295 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b9404e120 con 0x559f76e9d8d0 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/733018295 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7f2b98004440 con 0x559f76e9d8d0 2024-09-21T05:42:46.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 10 client.6378 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 10 client.6378 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 20 client.6378 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e525c0 0x559f77288040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 4 client.6378 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.013+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x559f771d6480 con 0x559f76e525c0 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.014+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e525c0 0x559f77288040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.014+0000 7f2b9f7fe640 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e525c0 0x559f77288040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3195852fd896a04 server_cookie=ab181c374b6b0996 in_seq=0 out_seq=0 2024-09-21T05:42:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.014+0000 7f2b7e7fc640 10 client.6378 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.015+0000 7f2b7e7fc640 1 -- 172.21.15.22:0/733018295 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f771d6480 con 0x559f76e525c0 2024-09-21T05:42:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.015+0000 7f2b7e7fc640 10 client.6378 handle_command_reply: tid=0 2024-09-21T05:42:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.015+0000 7f2baceb3740 1 client.6378 shutdown 2024-09-21T05:42:46.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 msgr2=0x559f772b9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 0x559f772b9260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2b9802aa20 tx=0x7f2b9802f720 comp rx=0 tx=0).stop 2024-09-21T05:42:46.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 shutdown_connections 2024-09-21T05:42:46.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559f76e525c0 0x559f77288040 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559f76e9d8d0 0x559f772b9260 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559f76e956f0 0x559f772aaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 --2- 172.21.15.22:0/733018295 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559f76e7d180 0x559f7732e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 >> 172.21.15.22:0/733018295 conn(0x559f76eb2700 msgr2=0x559f772c4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2b7e7fc640 0 client.6378 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:46.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 shutdown_connections 2024-09-21T05:42:46.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 1 -- 172.21.15.22:0/733018295 wait complete. 2024-09-21T05:42:46.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2baceb3740 20 client.6378 trim_cache size 0 max 16384 2024-09-21T05:42:46.033 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:46.033 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:42:46.033 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "6e53fcbf-899b-41d9-a24f-0b7e296ee3af", 2024-09-21T05:42:46.033 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:42:46.033 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:46.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b8c078b90 msgr2=0x7f2b8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b8c078b90 0x7f2b8c07b0a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009890 tx=0x7f2b90008040 comp rx=0 tx=0).stop 2024-09-21T05:42:46.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 msgr2=0x7f2ba00b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 0x7f2ba00b5f10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400d700 tx=0x7f2b9400dbc0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 shutdown_connections 2024-09-21T05:42:46.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2b8c078b90 0x7f2b8c07b0a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2ba015eb70 0x7f2ba00b6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2ba00d24a0 0x7f2ba00b5f10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.016+0000 7f2ba5f4a640 1 --2- 172.21.15.22:0/3572333668 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2ba00cf280 0x7f2ba0161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.017+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 >> 172.21.15.22:0/3572333668 conn(0x7f2ba00d45c0 msgr2=0x7f2ba015cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.017+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 shutdown_connections 2024-09-21T05:42:46.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.017+0000 7f2ba5f4a640 1 -- 172.21.15.22:0/3572333668 wait complete. 2024-09-21T05:42:46.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:42:46.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3237950966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 msgr2=0x7fa5f006fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3237950966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f006fe30 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0005fd0 tx=0x7fa5e002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3237950966 shutdown_connections 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3237950966 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5f015c580 0x7fa5f0070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3237950966 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f006fe30 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3237950966 >> 172.21.15.22:0/3237950966 conn(0x7fa5f006def0 msgr2=0x7fa5f006e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3237950966 shutdown_connections 2024-09-21T05:42:46.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.155+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3237950966 wait complete. 2024-09-21T05:42:46.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.156+0000 7fa5f69ad640 1 Processor -- start 2024-09-21T05:42:46.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.157+0000 7fa5f69ad640 1 -- start start 2024-09-21T05:42:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f69ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f013a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f69ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5f0155ce0 0x7fa5f01f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f69ad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5f015b1b0 0x7fa5f006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f69ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f015f850 con 0x7fa5f0074420 2024-09-21T05:42:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f69ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0070ba0 con 0x7fa5f0155ce0 2024-09-21T05:42:46.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f69ad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f015f9d0 con 0x7fa5f015b1b0 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f013a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f013a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36148/0 (socket says 172.21.15.22:36148) 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 -- 172.21.15.22:0/3397467836 learned_addr learned my addr 172.21.15.22:0/3397467836 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5f0155ce0 0x7fa5f01f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5f015b1b0 0x7fa5f006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 -- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5f015b1b0 msgr2=0x7fa5f006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5f015b1b0 0x7fa5f006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 -- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5f0155ce0 msgr2=0x7fa5f01f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5f0155ce0 0x7fa5f01f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.158+0000 7fa5effff640 1 -- 172.21.15.22:0/3397467836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f01f7730 con 0x7fa5f0074420 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.159+0000 7fa5effff640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f013a9b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0000c00 tx=0x7fa5e00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.159+0000 7fa5ecff9640 1 -- 172.21.15.22:0/3397467836 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e003e030 con 0x7fa5f0074420 2024-09-21T05:42:46.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.159+0000 7fa5ecff9640 1 -- 172.21.15.22:0/3397467836 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e0003ca0 con 0x7fa5f0074420 2024-09-21T05:42:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.159+0000 7fa5ecff9640 1 -- 172.21.15.22:0/3397467836 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0035040 con 0x7fa5f0074420 2024-09-21T05:42:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.159+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f01564b0 con 0x7fa5f0074420 2024-09-21T05:42:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.159+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f013d170 con 0x7fa5f0074420 2024-09-21T05:42:46.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.161+0000 7fa5ecff9640 1 -- 172.21.15.22:0/3397467836 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa5e0041030 con 0x7fa5f0074420 2024-09-21T05:42:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.161+0000 7fa5ecff9640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5d0078b90 0x7fa5d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.161+0000 7fa5ecff9640 1 -- 172.21.15.22:0/3397467836 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fa5e00bb2e0 con 0x7fa5f0074420 2024-09-21T05:42:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.162+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5d0078b90 0x7fa5d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.162+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5d0078b90 0x7fa5d007b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa5f006dec0 tx=0x7fa5d800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.162+0000 7fa5fd9fd740 1 Processor -- start 2024-09-21T05:42:46.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.162+0000 7fa5fd9fd740 1 -- start start 2024-09-21T05:42:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.163+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.163+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.163+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.163+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55895b21fa60 con 0x55895af05390 2024-09-21T05:42:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.163+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55895b20ee90 con 0x55895aedc4d0 2024-09-21T05:42:46.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.163+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55895b212ca0 con 0x55895aee46b0 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2f1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36154/0 (socket says 172.21.15.22:36154) 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 -- 172.21.15.22:0/753629364 learned_addr learned my addr 172.21.15.22:0/753629364 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 -- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 msgr2=0x55895b2c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 -- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 msgr2=0x55895b2c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 -- 172.21.15.22:0/753629364 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895afe77d0 con 0x55895af05390 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5effff640 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2f1a10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00b96f0 tx=0x7fa5e0082dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117904877a4cf93e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5e5ffb640 1 -- 172.21.15.22:0/753629364 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0048040 con 0x55895af05390 2024-09-21T05:42:46.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5e5ffb640 1 -- 172.21.15.22:0/753629364 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e00a3e50 con 0x55895af05390 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.164+0000 7fa5e5ffb640 1 -- 172.21.15.22:0/753629364 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0082b00 con 0x55895af05390 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/753629364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 msgr2=0x55895b2f1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2f1a10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa5e00b96f0 tx=0x7fa5e0082dc0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/753629364 shutdown_connections 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/753629364 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2f1a10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/753629364 >> 172.21.15.22:0/753629364 conn(0x55895aef94e0 msgr2=0x55895b2acd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/753629364 shutdown_connections 2024-09-21T05:42:46.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.165+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/753629364 wait complete. 2024-09-21T05:42:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.166+0000 7fa5fd9fd740 1 -- start start 2024-09-21T05:42:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.166+0000 7fa5fd9fd740 1 Processor -- start 2024-09-21T05:42:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.167+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.167+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.167+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.167+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55895af7f740 con 0x55895af05390 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.167+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55895b1b4970 con 0x55895aedc4d0 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.167+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55895b21fa60 con 0x55895aee46b0 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5fd9fd740 10 client.0 _ll_register_callbacks cb 0x55895b1c3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2ed5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36166/0 (socket says 172.21.15.22:36166) 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/2631602967 learned_addr learned my addr 172.21.15.22:0/2631602967 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5effff640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 msgr2=0x55895b2e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 msgr2=0x55895b2e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.174 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/2631602967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895af70970 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2ed5c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005800 tx=0x7fa5e8001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b35a7c7125f1b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5ceffd640 1 -- 172.21.15.22:0/2631602967 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8013540 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5ceffd640 1 -- 172.21.15.22:0/2631602967 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fa5e8010040 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.168+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e00a3e50 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5ceffd640 1 -- 172.21.15.22:0/2631602967 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa5e80136e0 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 10 client.6393 fetch_fsmap learned FSMap version 51 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55895af4e6a0 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5ceffd640 1 -- 172.21.15.22:0/2631602967 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e00a3e50 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5ceffd640 1 -- 172.21.15.22:0/2631602967 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7fa5e8013880 con 0x55895af05390 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 10 client.6393 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 10 client.6393 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 20 client.6393 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:46.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895aec3f60 0x55895b2e8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 4 client.6393 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895b2219b0 con 0x55895aec3f60 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5effff640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895aec3f60 0x55895b2e8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5ceffd640 10 client.6393 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.169+0000 7fa5effff640 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895aec3f60 0x55895b2e8870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36ac2bffe885752 server_cookie=c9dc2aac837dfae9 in_seq=0 out_seq=0 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.170+0000 7fa5ceffd640 1 -- 172.21.15.22:0/2631602967 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55895b2219b0 con 0x55895aec3f60 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.170+0000 7fa5ceffd640 10 client.6393 handle_command_reply: tid=0 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.170+0000 7fa5fd9fd740 1 client.6393 shutdown 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 msgr2=0x55895b2ed5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2ed5c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005800 tx=0x7fa5e8001c10 comp rx=0 tx=0).stop 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 shutdown_connections 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895aec3f60 0x55895b2e8870 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2ed5c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5ceffd640 0 client.6393 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/2631602967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 >> 172.21.15.22:0/2631602967 conn(0x55895aef94e0 msgr2=0x55895b2f1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 shutdown_connections 2024-09-21T05:42:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/2631602967 wait complete. 2024-09-21T05:42:46.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.171+0000 7fa5fd9fd740 20 client.6393 trim_cache size 0 max 16384 2024-09-21T05:42:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.189+0000 7fa5fd9fd740 1 Processor -- start 2024-09-21T05:42:46.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.189+0000 7fa5fd9fd740 1 -- start start 2024-09-21T05:42:46.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2d5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2d5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f0200060 con 0x55895af05390 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01ff060 con 0x55895aedc4d0 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f014e1b0 con 0x55895aee46b0 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2d5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2d5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36172/0 (socket says 172.21.15.22:36172) 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/983079041 learned_addr learned my addr 172.21.15.22:0/983079041 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2d5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5effff640 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 msgr2=0x55895b2d5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2d5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 msgr2=0x55895b2c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5f4f4a640 1 -- 172.21.15.22:0/983079041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895b2a9860 con 0x55895af05390 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.190+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2d5f80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80054d0 tx=0x7fa5e8013d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46800239e2256008 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5e57fa640 1 -- 172.21.15.22:0/983079041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8013880 con 0x55895af05390 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5e57fa640 1 -- 172.21.15.22:0/983079041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e80011c0 con 0x55895af05390 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5e57fa640 1 -- 172.21.15.22:0/983079041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8006d90 con 0x55895af05390 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/983079041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 msgr2=0x55895b2d5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2d5f80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80054d0 tx=0x7fa5e8013d80 comp rx=0 tx=0).stop 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/983079041 shutdown_connections 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895af05390 0x55895b2d5f80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895aee46b0 0x55895b2d5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/983079041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b2c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/983079041 >> 172.21.15.22:0/983079041 conn(0x55895aef94e0 msgr2=0x55895b2c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/983079041 shutdown_connections 2024-09-21T05:42:46.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.191+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/983079041 wait complete. 2024-09-21T05:42:46.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.192+0000 7fa5fd9fd740 1 -- start start 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.193+0000 7fa5fd9fd740 1 Processor -- start 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b366c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 0x55895b307b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895af05390 0x55895b2ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55895b2eb780 con 0x55895aee46b0 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f01feb70 con 0x55895aedc4d0 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55895b21f790 con 0x55895af05390 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5fd9fd740 10 client.0 _ll_register_callbacks cb 0x7fa5e8003a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 0x55895b307b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 0x55895b307b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36186/0 (socket says 172.21.15.22:36186) 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 -- 172.21.15.22:0/4261374959 learned_addr learned my addr 172.21.15.22:0/4261374959 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895af05390 0x55895b2ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5effff640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b366c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 -- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895af05390 msgr2=0x55895b2ebf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895af05390 0x55895b2ebf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 -- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 msgr2=0x55895b366c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b366c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 -- 172.21.15.22:0/4261374959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55895b2d3250 con 0x55895aee46b0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5effff640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b366c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5f4f4a640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895af05390 0x55895b2ebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ef7fe640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 0x55895b307b80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa5f015d000 tx=0x7fa5d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e03b819d452008 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.194+0000 7fa5ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5ceffd640 1 -- 172.21.15.22:0/4261374959 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d802e060 con 0x55895aee46b0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5ceffd640 1 -- 172.21.15.22:0/4261374959 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 5795+0+0 (secure 0 0 0) 0x7fa5d801fac0 con 0x55895aee46b0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5e80011c0 con 0x55895aee46b0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5ceffd640 1 -- 172.21.15.22:0/4261374959 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa5d8034050 con 0x55895aee46b0 2024-09-21T05:42:46.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 10 client.6399 fetch_fsmap learned FSMap version 51 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x55895b2a83f0 con 0x55895aee46b0 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5ceffd640 1 -- 172.21.15.22:0/4261374959 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e80011c0 con 0x55895aee46b0 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5ceffd640 1 -- 172.21.15.22:0/4261374959 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 51) ==== 3502+0+0 (secure 0 0 0) 0x7fa5d8037020 con 0x55895aee46b0 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 10 client.6399 fetch_fsmap finished waiting for FSMap version 51 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 10 client.6399 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 20 client.6399 populate_metadata read hostname 'smithi022' 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895b3080c0 0x55895b2ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 4 client.6399 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.195+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55895b21d460 con 0x55895b3080c0 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.196+0000 7fa5effff640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895b3080c0 0x55895b2ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.196+0000 7fa5effff640 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895b3080c0 0x55895b2ec490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f531bcef5412f960 server_cookie=75b8bbd38f8546a2 in_seq=0 out_seq=0 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.196+0000 7fa5ceffd640 10 client.6399 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:46.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.196+0000 7fa5ceffd640 1 -- 172.21.15.22:0/4261374959 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+576 (crc 0 0 0) 0x55895b21d460 con 0x55895b3080c0 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.196+0000 7fa5ceffd640 10 client.6399 handle_command_reply: tid=0 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.196+0000 7fa5fd9fd740 1 client.6399 shutdown 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 msgr2=0x55895b307b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 0x55895b307b80 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fa5f015d000 tx=0x7fa5d8004730 comp rx=0 tx=0).stop 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 shutdown_connections 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55895b3080c0 0x55895b2ec490 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55895af05390 0x55895b2ebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55895aee46b0 0x55895b307b80 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5ceffd640 0 client.6399 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 --2- 172.21.15.22:0/4261374959 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55895aedc4d0 0x55895b366c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 >> 172.21.15.22:0/4261374959 conn(0x55895aef94e0 msgr2=0x55895b3677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 shutdown_connections 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 1 -- 172.21.15.22:0/4261374959 wait complete. 2024-09-21T05:42:46.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5fd9fd740 20 client.6399 trim_cache size 0 max 16384 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "6e53fcbf-899b-41d9-a24f-0b7e296ee3af": { 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "6e53fcbf-899b-41d9-a24f-0b7e296ee3af", 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive" 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:42:46.204 INFO:teuthology.orchestra.run.smithi022.stdout: "6e53fcbf-899b-41d9-a24f-0b7e296ee3af": { 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:42:46.015156+0000", 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:42:46.205 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5d0078b90 msgr2=0x7fa5d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5d0078b90 0x7fa5d007b0a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa5f006dec0 tx=0x7fa5d800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 msgr2=0x7fa5f013a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f013a9b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0000c00 tx=0x7fa5e00038a0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 shutdown_connections 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5d0078b90 0x7fa5d007b0a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa5f015b1b0 0x7fa5f006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5f0155ce0 0x7fa5f01f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 --2- 172.21.15.22:0/3397467836 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5f0074420 0x7fa5f013a9b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:42:46.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.197+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 >> 172.21.15.22:0/3397467836 conn(0x7fa5f006def0 msgr2=0x7fa5f014ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:42:46.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.198+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 shutdown_connections 2024-09-21T05:42:46.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:42:46.198+0000 7fa5f69ad640 1 -- 172.21.15.22:0/3397467836 wait complete. 2024-09-21T05:42:46.210 INFO:tasks.cephfs.filesystem:scrub status for tag:6e53fcbf-899b-41d9-a24f-0b7e296ee3af - {'path': '/subdir', 'tag': '6e53fcbf-899b-41d9-a24f-0b7e296ee3af', 'options': 'recursive'} 2024-09-21T05:42:46.585 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-09-21T05:42:46.584+0000 7fe06d004640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:42:51.698 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:42:51.701 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:42:53.393 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:42:53.391+0000 7fbf61087640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T05:43:16.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:16.212 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 -- 172.21.15.22:0/3926130882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c004800 msgr2=0x7f250c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 --2- 172.21.15.22:0/3926130882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c004800 0x7f250c004c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2520070050 tx=0x7f250c005830 comp rx=0 tx=0).stop 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 -- 172.21.15.22:0/3926130882 shutdown_connections 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 --2- 172.21.15.22:0/3926130882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f250c004800 0x7f250c004c10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 -- 172.21.15.22:0/3926130882 >> 172.21.15.22:0/3926130882 conn(0x7f252009d7f0 msgr2=0x7f25200d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 -- 172.21.15.22:0/3926130882 shutdown_connections 2024-09-21T05:43:16.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.345+0000 7f2525ab6640 1 -- 172.21.15.22:0/3926130882 wait complete. 2024-09-21T05:43:16.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.347+0000 7f2525ab6640 1 Processor -- start 2024-09-21T05:43:16.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.347+0000 7f2525ab6640 1 -- start start 2024-09-21T05:43:16.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2525ab6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c004800 0x7f25200cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2525ab6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25200ca9d0 0x7f25200ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2525ab6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 0x7f25201f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2525ab6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f25200b6120 con 0x7f25201f5c60 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2525ab6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25200b5fa0 con 0x7f25200ca9d0 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2525ab6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f25200b62a0 con 0x7f250c004800 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 0x7f25201f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 0x7f25201f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58788/0 (socket says 172.21.15.22:58788) 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 -- 172.21.15.22:0/2131104099 learned_addr learned my addr 172.21.15.22:0/2131104099 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251f7fe640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c004800 0x7f25200cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251effd640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25200ca9d0 0x7f25200ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 -- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c004800 msgr2=0x7f25200cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c004800 0x7f25200cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 -- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25200ca9d0 msgr2=0x7f25200ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25200ca9d0 0x7f25200ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 -- 172.21.15.22:0/2131104099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25200ca0f0 con 0x7f25201f5c60 2024-09-21T05:43:16.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251f7fe640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c004800 0x7f25200cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f251ffff640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 0x7f25201f6070 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2518011bb0 tx=0x7f251800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.349+0000 7f2517fff640 1 -- 172.21.15.22:0/2131104099 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518003d80 con 0x7f25201f5c60 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.350+0000 7f2517fff640 1 -- 172.21.15.22:0/2131104099 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2518004b20 con 0x7f25201f5c60 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.350+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25201f9220 con 0x7f25201f5c60 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.350+0000 7f2517fff640 1 -- 172.21.15.22:0/2131104099 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2518004030 con 0x7f25201f5c60 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.350+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25201f97e0 con 0x7f25201f5c60 2024-09-21T05:43:16.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.352+0000 7f2517fff640 1 -- 172.21.15.22:0/2131104099 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f25180041d0 con 0x7f25201f5c60 2024-09-21T05:43:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.353+0000 7f2517fff640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2500078b90 0x7f250007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.353+0000 7f2517fff640 1 -- 172.21.15.22:0/2131104099 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f251809bd30 con 0x7f25201f5c60 2024-09-21T05:43:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.353+0000 7f251f7fe640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2500078b90 0x7f250007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.353+0000 7f252ca20740 1 Processor -- start 2024-09-21T05:43:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.353+0000 7f252ca20740 1 -- start start 2024-09-21T05:43:16.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.354+0000 7f251f7fe640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2500078b90 0x7f250007b0a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2520052910 tx=0x7f25100066f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f252ca20740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 0x55ebf66ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf66acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f252ca20740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ebf65f5a60 con 0x55ebf62db390 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ebf65e4e90 con 0x55ebf62ba6b0 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ebf65e8ca0 con 0x55ebf62b24d0 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 0x55ebf66ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 0x55ebf66ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58796/0 (socket says 172.21.15.22:58796) 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 -- 172.21.15.22:0/45961396 learned_addr learned my addr 172.21.15.22:0/45961396 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251ffff640 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251effd640 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf66acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 -- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 msgr2=0x55ebf66acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf66acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 -- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 msgr2=0x55ebf66c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 -- 172.21.15.22:0/45961396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf63bd7d0 con 0x55ebf62db390 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.355+0000 7f251f7fe640 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 0x55ebf66ac740 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f25100038f0 tx=0x7f2510014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0447181d9fe017 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.356+0000 7f25157fa640 1 -- 172.21.15.22:0/45961396 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510012700 con 0x55ebf62db390 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.356+0000 7f25157fa640 1 -- 172.21.15.22:0/45961396 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2510022030 con 0x55ebf62db390 2024-09-21T05:43:16.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.356+0000 7f25157fa640 1 -- 172.21.15.22:0/45961396 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510012ae0 con 0x55ebf62db390 2024-09-21T05:43:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.356+0000 7f252ca20740 1 -- 172.21.15.22:0/45961396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 msgr2=0x55ebf66ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.356+0000 7f252ca20740 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 0x55ebf66ac740 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f25100038f0 tx=0x7f2510014000 comp rx=0 tx=0).stop 2024-09-21T05:43:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 -- 172.21.15.22:0/45961396 shutdown_connections 2024-09-21T05:43:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf66acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 --2- 172.21.15.22:0/45961396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62db390 0x55ebf66ac740 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 -- 172.21.15.22:0/45961396 >> 172.21.15.22:0/45961396 conn(0x55ebf62cf4e0 msgr2=0x55ebf6682a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 -- 172.21.15.22:0/45961396 shutdown_connections 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.357+0000 7f252ca20740 1 -- 172.21.15.22:0/45961396 wait complete. 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.358+0000 7f252ca20740 1 -- start start 2024-09-21T05:43:16.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.358+0000 7f252ca20740 1 Processor -- start 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66d9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf66c0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6355740 con 0x55ebf62b24d0 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ebf658a970 con 0x55ebf62ba6b0 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ebf66ad1c0 con 0x55ebf62db390 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f252ca20740 10 client.0 _ll_register_callbacks cb 0x55ebf658dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66d9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66d9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58804/0 (socket says 172.21.15.22:58804) 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 -- 172.21.15.22:0/3506122713 learned_addr learned my addr 172.21.15.22:0/3506122713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251ffff640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf66c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251effd640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 -- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 msgr2=0x55ebf66c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf66c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 -- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 msgr2=0x55ebf66da430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.359+0000 7f251f7fe640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66da430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f251f7fe640 1 -- 172.21.15.22:0/3506122713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf6346970 con 0x55ebf62b24d0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f251f7fe640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66d9ef0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f25100128e0 tx=0x7f2510012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521e3a3b348a60e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f24fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f24fe7fc640 1 -- 172.21.15.22:0/3506122713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510012b30 con 0x55ebf62b24d0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f24fe7fc640 1 -- 172.21.15.22:0/3506122713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f2510012cd0 con 0x55ebf62b24d0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2510022030 con 0x55ebf62b24d0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f24fe7fc640 1 -- 172.21.15.22:0/3506122713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2510039050 con 0x55ebf62b24d0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 10 client.6411 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55ebf63246a0 con 0x55ebf62b24d0 2024-09-21T05:43:16.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f24fe7fc640 1 -- 172.21.15.22:0/3506122713 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2510022030 con 0x55ebf62b24d0 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f24fe7fc640 1 -- 172.21.15.22:0/3506122713 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f251003c020 con 0x55ebf62b24d0 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 10 client.6411 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 10 client.6411 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 20 client.6411 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6299f60 0x55ebf66aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 4 client.6411 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.360+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebf65f79b0 con 0x55ebf6299f60 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.361+0000 7f251effd640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6299f60 0x55ebf66aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.361+0000 7f24fe7fc640 10 client.6411 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.361+0000 7f251effd640 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6299f60 0x55ebf66aa660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa09f20b5dbf293 server_cookie=86b87a5757f037e1 in_seq=0 out_seq=0 2024-09-21T05:43:16.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f24fe7fc640 1 -- 172.21.15.22:0/3506122713 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ebf65f79b0 con 0x55ebf6299f60 2024-09-21T05:43:16.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f24fe7fc640 10 client.6411 handle_command_reply: tid=0 2024-09-21T05:43:16.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 client.6411 shutdown 2024-09-21T05:43:16.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 msgr2=0x55ebf66d9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66d9ef0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f25100128e0 tx=0x7f2510012910 comp rx=0 tx=0).stop 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 shutdown_connections 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6299f60 0x55ebf66aa660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf66c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f24fe7fc640 0 client.6411 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf66da430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 --2- 172.21.15.22:0/3506122713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66d9ef0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 >> 172.21.15.22:0/3506122713 conn(0x55ebf62cf4e0 msgr2=0x55ebf66c9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 shutdown_connections 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.362+0000 7f252ca20740 1 -- 172.21.15.22:0/3506122713 wait complete. 2024-09-21T05:43:16.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.363+0000 7f252ca20740 20 client.6411 trim_cache size 0 max 16384 2024-09-21T05:43:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.381+0000 7f252ca20740 1 Processor -- start 2024-09-21T05:43:16.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.381+0000 7f252ca20740 1 -- start start 2024-09-21T05:43:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f252ca20740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66c0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf669b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf669b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f252ca20740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2520202520 con 0x55ebf62b24d0 2024-09-21T05:43:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25200c8710 con 0x55ebf62ba6b0 2024-09-21T05:43:16.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f252015eba0 con 0x55ebf62db390 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58812/0 (socket says 172.21.15.22:58812) 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 -- 172.21.15.22:0/1932800726 learned_addr learned my addr 172.21.15.22:0/1932800726 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251ffff640 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf669b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 -- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 msgr2=0x55ebf669b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251effd640 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf669b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf669b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 -- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 msgr2=0x55ebf669b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf669b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.382+0000 7f251f7fe640 1 -- 172.21.15.22:0/1932800726 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebf66a47a0 con 0x55ebf62b24d0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.383+0000 7f251f7fe640 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66c0340 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f25100038b0 tx=0x7f2510012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409e7429c3401960 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.383+0000 7f2514ff9640 1 -- 172.21.15.22:0/1932800726 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2510012c60 con 0x55ebf62b24d0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.383+0000 7f2514ff9640 1 -- 172.21.15.22:0/1932800726 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2510022040 con 0x55ebf62b24d0 2024-09-21T05:43:16.386 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.383+0000 7f2514ff9640 1 -- 172.21.15.22:0/1932800726 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251001e610 con 0x55ebf62b24d0 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.383+0000 7f252ca20740 1 -- 172.21.15.22:0/1932800726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 msgr2=0x55ebf66c0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.383+0000 7f252ca20740 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66c0340 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f25100038b0 tx=0x7f2510012980 comp rx=0 tx=0).stop 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 -- 172.21.15.22:0/1932800726 shutdown_connections 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62db390 0x55ebf669b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62ba6b0 0x55ebf669b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 --2- 172.21.15.22:0/1932800726 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62b24d0 0x55ebf66c0340 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 -- 172.21.15.22:0/1932800726 >> 172.21.15.22:0/1932800726 conn(0x55ebf62cf4e0 msgr2=0x55ebf66da550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 -- 172.21.15.22:0/1932800726 shutdown_connections 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 -- 172.21.15.22:0/1932800726 wait complete. 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.384+0000 7f252ca20740 1 -- start start 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.385+0000 7f252ca20740 1 Processor -- start 2024-09-21T05:43:16.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf673cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 0x55ebf6746550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62db390 0x55ebf674a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f25200c80a0 con 0x55ebf62ba6b0 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55ebf6683fa0 con 0x55ebf62db390 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55ebf6747000 con 0x55ebf62b24d0 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f252ca20740 10 client.0 _ll_register_callbacks cb 0x7f2518005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 0x55ebf6746550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 0x55ebf6746550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58830/0 (socket says 172.21.15.22:58830) 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 -- 172.21.15.22:0/4287162471 learned_addr learned my addr 172.21.15.22:0/4287162471 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251f7fe640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf673cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251ffff640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62db390 0x55ebf674a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 -- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 msgr2=0x55ebf673cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf673cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 -- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62db390 msgr2=0x55ebf674a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62db390 0x55ebf674a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.386+0000 7f251effd640 1 -- 172.21.15.22:0/4287162471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebf66db150 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f251effd640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 0x55ebf6746550 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f250802a8d0 tx=0x7f2508001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173064d3f795fab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f24fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f24fe7fc640 1 -- 172.21.15.22:0/4287162471 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250804a070 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f24fe7fc640 1 -- 172.21.15.22:0/4287162471 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f2508005770 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2510022040 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f24fe7fc640 1 -- 172.21.15.22:0/4287162471 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2508005970 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 10 client.6417 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55ebf6746d40 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f24fe7fc640 1 -- 172.21.15.22:0/4287162471 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2510022040 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f24fe7fc640 1 -- 172.21.15.22:0/4287162471 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f2508057080 con 0x55ebf62ba6b0 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 10 client.6417 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 10 client.6417 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 20 client.6417 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:16.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6698d90 0x55ebf6697bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 4 client.6417 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebf65f3460 con 0x55ebf6698d90 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.387+0000 7f251f7fe640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6698d90 0x55ebf6697bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f24fe7fc640 10 client.6417 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f251f7fe640 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6698d90 0x55ebf6697bb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33e2ec449e8011d2 server_cookie=b111b220bd47235c in_seq=0 out_seq=0 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f24fe7fc640 1 -- 172.21.15.22:0/4287162471 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+380 (crc 0 0 0) 0x55ebf65f3460 con 0x55ebf6698d90 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f24fe7fc640 10 client.6417 handle_command_reply: tid=0 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 client.6417 shutdown 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 msgr2=0x55ebf6746550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 0x55ebf6746550 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f250802a8d0 tx=0x7f2508001400 comp rx=0 tx=0).stop 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 shutdown_connections 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55ebf6698d90 0x55ebf6697bb0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55ebf62db390 0x55ebf674a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55ebf62ba6b0 0x55ebf6746550 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f24fe7fc640 0 client.6417 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 --2- 172.21.15.22:0/4287162471 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55ebf62b24d0 0x55ebf673cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 >> 172.21.15.22:0/4287162471 conn(0x55ebf62cf4e0 msgr2=0x55ebf669b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 shutdown_connections 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.388+0000 7f252ca20740 1 -- 172.21.15.22:0/4287162471 wait complete. 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f252ca20740 20 client.6417 trim_cache size 0 max 16384 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:43:16.395 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "6e53fcbf-899b-41d9-a24f-0b7e296ee3af": { 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:42:46.015156+0000", 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:16.396 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2500078b90 msgr2=0x7f250007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2500078b90 0x7f250007b0a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2520052910 tx=0x7f25100066f0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 msgr2=0x7f25201f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 0x7f25201f6070 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2518011bb0 tx=0x7f251800f790 comp rx=0 tx=0).stop 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 shutdown_connections 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25201f5c60 0x7f25201f6070 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25200ca9d0 0x7f25200ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2500078b90 0x7f250007b0a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 --2- 172.21.15.22:0/2131104099 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f250c004800 0x7f25200cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 >> 172.21.15.22:0/2131104099 conn(0x7f252009d7f0 msgr2=0x7f25200d18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 shutdown_connections 2024-09-21T05:43:16.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.389+0000 7f2525ab6640 1 -- 172.21.15.22:0/2131104099 wait complete. 2024-09-21T05:43:16.402 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:43:16.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:43:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 -- 172.21.15.22:0/849992062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0d2ed0 msgr2=0x7f622c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 --2- 172.21.15.22:0/849992062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0d2ed0 0x7f622c0d1a80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6228005fd0 tx=0x7f622802aa10 comp rx=0 tx=0).stop 2024-09-21T05:43:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 -- 172.21.15.22:0/849992062 shutdown_connections 2024-09-21T05:43:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 --2- 172.21.15.22:0/849992062 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0d2ed0 0x7f622c0d1a80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 -- 172.21.15.22:0/849992062 >> 172.21.15.22:0/849992062 conn(0x7f622c08bfc0 msgr2=0x7f622c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 -- 172.21.15.22:0/849992062 shutdown_connections 2024-09-21T05:43:16.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.517+0000 7f6234139640 1 -- 172.21.15.22:0/849992062 wait complete. 2024-09-21T05:43:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.519+0000 7f6234139640 1 Processor -- start 2024-09-21T05:43:16.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.519+0000 7f6234139640 1 -- start start 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.520+0000 7f6234139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0b9b70 0x7f622c1f9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.520+0000 7f6234139640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 0x7f622c14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.520+0000 7f6234139640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c0d2ed0 0x7f622c14e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.520+0000 7f6234139640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f622c162f70 con 0x7f622c0cf7a0 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.520+0000 7f6234139640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f622c162df0 con 0x7f622c0b9b70 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.520+0000 7f6234139640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f622c1630f0 con 0x7f622c0d2ed0 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 0x7f622c14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 0x7f622c14e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58850/0 (socket says 172.21.15.22:58850) 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 -- 172.21.15.22:0/1200598238 learned_addr learned my addr 172.21.15.22:0/1200598238 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f6231ed5640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0b9b70 0x7f622c1f9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62326d6640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c0d2ed0 0x7f622c14e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 -- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c0d2ed0 msgr2=0x7f622c14e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c0d2ed0 0x7f622c14e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 -- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0b9b70 msgr2=0x7f622c1f9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0b9b70 0x7f622c1f9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 -- 172.21.15.22:0/1200598238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622c1fa250 con 0x7f622c0cf7a0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62326d6640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c0d2ed0 0x7f622c14e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62316d4640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 0x7f622c14e320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c3f0 tx=0x7f621c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c017070 con 0x7f622c0cf7a0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f621c0035f0 con 0x7f622c0cf7a0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622c1e88b0 con 0x7f622c0cf7a0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c003a40 con 0x7f622c0cf7a0 2024-09-21T05:43:16.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.521+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f622c0c9420 con 0x7f622c0cf7a0 2024-09-21T05:43:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.523+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f622c0d1a80 con 0x7f622c0cf7a0 2024-09-21T05:43:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.523+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f621c01a070 con 0x7f622c0cf7a0 2024-09-21T05:43:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.524+0000 7f62227fc640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6210078bc0 0x7f621007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.524+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f621c09a030 con 0x7f622c0cf7a0 2024-09-21T05:43:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.524+0000 7f6231ed5640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6210078bc0 0x7f621007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.524+0000 7f6231ed5640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6210078bc0 0x7f621007b0d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6228035000 tx=0x7f6228004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.528+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f622c1fa250 con 0x7f622c0cf7a0 2024-09-21T05:43:16.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.755+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f622c1e8640 con 0x7f622c0cf7a0 2024-09-21T05:43:16.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.755+0000 7f62227fc640 1 -- 172.21.15.22:0/1200598238 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5230 (secure 0 0 0) 0x7f621c062470 con 0x7f622c0cf7a0 2024-09-21T05:43:16.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:16.758 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":52,"btime":"2024-09-21T05:42:53:392293+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:42:53.360712+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":10,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:43:16.758 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 52 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6210078bc0 msgr2=0x7f621007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6210078bc0 0x7f621007b0d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6228035000 tx=0x7f6228004ab0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 msgr2=0x7f622c14e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 0x7f622c14e320 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f621c00c3f0 tx=0x7f621c00c8b0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 shutdown_connections 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6210078bc0 0x7f621007b0d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f622c0d2ed0 0x7f622c14e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f622c0cf7a0 0x7f622c14e320 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 --2- 172.21.15.22:0/1200598238 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f622c0b9b70 0x7f622c1f9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 >> 172.21.15.22:0/1200598238 conn(0x7f622c08bfc0 msgr2=0x7f622c0d24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 shutdown_connections 2024-09-21T05:43:16.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.758+0000 7f6234139640 1 -- 172.21.15.22:0/1200598238 wait complete. 2024-09-21T05:43:16.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-21T05:43:16.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.887+0000 7febc4f33640 1 -- 172.21.15.22:0/2689090047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 msgr2=0x7febc00d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.887+0000 7febc4f33640 1 --2- 172.21.15.22:0/2689090047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc00d3330 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7febb4005fd0 tx=0x7febb402a540 comp rx=0 tx=0).stop 2024-09-21T05:43:16.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.887+0000 7febc4f33640 1 -- 172.21.15.22:0/2689090047 shutdown_connections 2024-09-21T05:43:16.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.887+0000 7febc4f33640 1 --2- 172.21.15.22:0/2689090047 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc00d3330 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.887+0000 7febc4f33640 1 -- 172.21.15.22:0/2689090047 >> 172.21.15.22:0/2689090047 conn(0x7febc008bfc0 msgr2=0x7febc009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.887+0000 7febc4f33640 1 -- 172.21.15.22:0/2689090047 shutdown_connections 2024-09-21T05:43:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.888+0000 7febc4f33640 1 -- 172.21.15.22:0/2689090047 wait complete. 2024-09-21T05:43:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.889+0000 7febc4f33640 1 Processor -- start 2024-09-21T05:43:16.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.890+0000 7febc4f33640 1 -- start start 2024-09-21T05:43:16.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febc4f33640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febc4f33640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc00d3870 0x7febc01e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febc4f33640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc01e87a0 0x7febc0150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febc4f33640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7febc00d1070 con 0x7febc00b3540 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febc4f33640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febc00d0ef0 con 0x7febc00d3870 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febc4f33640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7febc00d11f0 con 0x7febc01e87a0 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc014e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58862/0 (socket says 172.21.15.22:58862) 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 -- 172.21.15.22:0/3857777291 learned_addr learned my addr 172.21.15.22:0/3857777291 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbdd9b640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc00d3870 0x7febc01e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbed9d640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc01e87a0 0x7febc0150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 -- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc01e87a0 msgr2=0x7febc0150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc01e87a0 0x7febc0150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 -- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc00d3870 msgr2=0x7febc01e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc00d3870 0x7febc01e8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 -- 172.21.15.22:0/3857777291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc00cb2a0 con 0x7febc00b3540 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbdd9b640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc00d3870 0x7febc01e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:16.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbed9d640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc01e87a0 0x7febc0150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.891+0000 7febbe59c640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc014e220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7febb000ba80 tx=0x7febb000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.892+0000 7febaeffd640 1 -- 172.21.15.22:0/3857777291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb000cc00 con 0x7febc00b3540 2024-09-21T05:43:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.892+0000 7febaeffd640 1 -- 172.21.15.22:0/3857777291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb0003970 con 0x7febc00b3540 2024-09-21T05:43:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.892+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc00cd3f0 con 0x7febc00b3540 2024-09-21T05:43:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.892+0000 7febaeffd640 1 -- 172.21.15.22:0/3857777291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0003e00 con 0x7febc00b3540 2024-09-21T05:43:16.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.892+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc00cd860 con 0x7febc00b3540 2024-09-21T05:43:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.894+0000 7febaeffd640 1 -- 172.21.15.22:0/3857777291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febb0004000 con 0x7febc00b3540 2024-09-21T05:43:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.895+0000 7febaeffd640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feba0078bc0 0x7feba007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.895+0000 7febaeffd640 1 -- 172.21.15.22:0/3857777291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7febb009ae00 con 0x7febc00b3540 2024-09-21T05:43:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.895+0000 7febbdd9b640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feba0078bc0 0x7feba007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.895+0000 7febcbe9d740 1 Processor -- start 2024-09-21T05:43:16.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.896+0000 7febcbe9d740 1 -- start start 2024-09-21T05:43:16.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.896+0000 7febbdd9b640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feba0078bc0 0x7feba007b0d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febb4033000 tx=0x7febb4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecd92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecf61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febcbe9d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55761ec219a0 con 0x55761e8de490 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55761ec10dd0 con 0x55761e8e6670 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55761ec14be0 con 0x55761e907350 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecd92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecd92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58866/0 (socket says 172.21.15.22:58866) 2024-09-21T05:43:16.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 -- 172.21.15.22:0/2332826361 learned_addr learned my addr 172.21.15.22:0/2332826361 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbe59c640 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbed9d640 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecf61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 -- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 msgr2=0x55761ecd8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecd8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 -- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 msgr2=0x55761ecf61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecf61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.897+0000 7febbdd9b640 1 -- 172.21.15.22:0/2332826361 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761eba9700 con 0x55761e8de490 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.898+0000 7febbdd9b640 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecd92c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7febb40099c0 tx=0x7febb403b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29085f83a54c93e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.898+0000 7feb9ffff640 1 -- 172.21.15.22:0/2332826361 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4035050 con 0x55761e8de490 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.898+0000 7feb9ffff640 1 -- 172.21.15.22:0/2332826361 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb404b070 con 0x55761e8de490 2024-09-21T05:43:16.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.898+0000 7feb9ffff640 1 -- 172.21.15.22:0/2332826361 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb40047a0 con 0x55761e8de490 2024-09-21T05:43:16.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 -- 172.21.15.22:0/2332826361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 msgr2=0x55761ecd92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecd92c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7febb40099c0 tx=0x7febb403b000 comp rx=0 tx=0).stop 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 -- 172.21.15.22:0/2332826361 shutdown_connections 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecf61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecd92c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 --2- 172.21.15.22:0/2332826361 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecd8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 -- 172.21.15.22:0/2332826361 >> 172.21.15.22:0/2332826361 conn(0x55761e8fb4a0 msgr2=0x55761ecaf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 -- 172.21.15.22:0/2332826361 shutdown_connections 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.899+0000 7febcbe9d740 1 -- 172.21.15.22:0/2332826361 wait complete. 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.900+0000 7febcbe9d740 1 -- start start 2024-09-21T05:43:16.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.900+0000 7febcbe9d740 1 Processor -- start 2024-09-21T05:43:16.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55761e981700 con 0x55761e8de490 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55761ebb6340 con 0x55761e8e6670 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55761ecd9800 con 0x55761e907350 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febcbe9d740 10 client.0 _ll_register_callbacks cb 0x55761ebb4ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febbe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febbe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58876/0 (socket says 172.21.15.22:58876) 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febbe59c640 1 -- 172.21.15.22:0/1229259748 learned_addr learned my addr 172.21.15.22:0/1229259748 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febbdd9b640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.901+0000 7febbed9d640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbe59c640 1 -- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 msgr2=0x55761ecebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbe59c640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbe59c640 1 -- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 msgr2=0x55761ecea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbe59c640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbe59c640 1 -- 172.21.15.22:0/1229259748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761e9eb7c0 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbed9d640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febbe59c640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecba450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7febb000ba50 tx=0x7febb0062fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fc47a25b98af08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7feb9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7feb9d7fa640 1 -- 172.21.15.22:0/1229259748 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0078dc0 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7feb9d7fa640 1 -- 172.21.15.22:0/1229259748 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7febb0061c70 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb404b070 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7feb9d7fa640 1 -- 172.21.15.22:0/1229259748 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7febb0083050 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 10 client.6441 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55761ea4bf50 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7feb9d7fa640 1 -- 172.21.15.22:0/1229259748 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb404b070 con 0x55761e8de490 2024-09-21T05:43:16.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7feb9d7fa640 1 -- 172.21.15.22:0/1229259748 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7febb005e360 con 0x55761e8de490 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 10 client.6441 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 10 client.6441 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 20 client.6441 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761e8c6130 0x55761ecbc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 4 client.6441 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.902+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55761ec238f0 con 0x55761e8c6130 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.903+0000 7febbdd9b640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761e8c6130 0x55761ecbc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.903+0000 7febbdd9b640 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761e8c6130 0x55761ecbc500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f87beaad3f9714 server_cookie=39ef3960a983f7c4 in_seq=0 out_seq=0 2024-09-21T05:43:16.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.903+0000 7feb9d7fa640 10 client.6441 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7feb9d7fa640 1 -- 172.21.15.22:0/1229259748 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55761ec238f0 con 0x55761e8c6130 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7feb9d7fa640 10 client.6441 handle_command_reply: tid=0 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 client.6441 shutdown 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 msgr2=0x55761ecba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecba450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7febb000ba50 tx=0x7febb0062fb0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 shutdown_connections 2024-09-21T05:43:16.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761e8c6130 0x55761ecbc500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ecebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7feb9d7fa640 0 client.6441 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1229259748 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ecba450 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 >> 172.21.15.22:0/1229259748 conn(0x55761e8fb4a0 msgr2=0x55761ecf5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 shutdown_connections 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 1 -- 172.21.15.22:0/1229259748 wait complete. 2024-09-21T05:43:16.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.904+0000 7febcbe9d740 20 client.6441 trim_cache size 0 max 16384 2024-09-21T05:43:16.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.918+0000 7febcbe9d740 1 Processor -- start 2024-09-21T05:43:16.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.918+0000 7febcbe9d740 1 -- start start 2024-09-21T05:43:16.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e8de490 0x55761ece9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecc70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 0x55761ecd8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febcbe9d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7febc0201170 con 0x55761e907350 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7febc00ce560 con 0x55761e8e6670 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7febc01f9900 con 0x55761e8de490 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febbed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 0x55761ecd8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febbed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 0x55761ecd8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58884/0 (socket says 172.21.15.22:58884) 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.919+0000 7febbed9d640 1 -- 172.21.15.22:0/1416669404 learned_addr learned my addr 172.21.15.22:0/1416669404 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbe59c640 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e8de490 0x55761ece9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbdd9b640 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecc70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbed9d640 1 -- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e8de490 msgr2=0x55761ece9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbed9d640 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e8de490 0x55761ece9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbed9d640 1 -- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 msgr2=0x55761ecc70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbed9d640 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecc70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbed9d640 1 -- 172.21.15.22:0/1416669404 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761e950660 con 0x55761e907350 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7febbed9d640 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 0x55761ecd8a10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7febb80055a0 tx=0x7febb8003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9345e0c55466047 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7feb9f7fe640 1 -- 172.21.15.22:0/1416669404 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8001930 con 0x55761e907350 2024-09-21T05:43:16.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7feb9f7fe640 1 -- 172.21.15.22:0/1416669404 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb8001bc0 con 0x55761e907350 2024-09-21T05:43:16.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.920+0000 7feb9f7fe640 1 -- 172.21.15.22:0/1416669404 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb80156a0 con 0x55761e907350 2024-09-21T05:43:16.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 -- 172.21.15.22:0/1416669404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 msgr2=0x55761ecd8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 0x55761ecd8a10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7febb80055a0 tx=0x7febb8003a80 comp rx=0 tx=0).stop 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 -- 172.21.15.22:0/1416669404 shutdown_connections 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e907350 0x55761ecd8a10 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ecc70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 --2- 172.21.15.22:0/1416669404 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e8de490 0x55761ece9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 -- 172.21.15.22:0/1416669404 >> 172.21.15.22:0/1416669404 conn(0x55761e8fb4a0 msgr2=0x55761ed063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 -- 172.21.15.22:0/1416669404 shutdown_connections 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.921+0000 7febcbe9d740 1 -- 172.21.15.22:0/1416669404 wait complete. 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.922+0000 7febcbe9d740 1 -- start start 2024-09-21T05:43:16.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.922+0000 7febcbe9d740 1 Processor -- start 2024-09-21T05:43:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ed587e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ed07240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ed07780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55761ecec0b0 con 0x55761e8de490 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55761ecbab00 con 0x55761e8e6670 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55761ecd6660 con 0x55761e907350 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febcbe9d740 10 client.0 _ll_register_callbacks cb 0x7febb00623b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febbe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ed587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febbe59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ed587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58896/0 (socket says 172.21.15.22:58896) 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.923+0000 7febbe59c640 1 -- 172.21.15.22:0/383887861 learned_addr learned my addr 172.21.15.22:0/383887861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbed9d640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ed07780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbdd9b640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ed07240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbe59c640 1 -- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 msgr2=0x55761ed07780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbe59c640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ed07780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbe59c640 1 -- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 msgr2=0x55761ed07240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbe59c640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ed07240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbe59c640 1 -- 172.21.15.22:0/383887861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55761ed774b0 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbdd9b640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ed07240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febbe59c640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ed587e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7febb000cfd0 tx=0x7febb0078df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c99e03e2262fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7feb9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7feb9d7fa640 1 -- 172.21.15.22:0/383887861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb005e8b0 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7feb9d7fa640 1 -- 172.21.15.22:0/383887861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7febb00616e0 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb8001bc0 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7feb9d7fa640 1 -- 172.21.15.22:0/383887861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7febb006c430 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 10 client.6447 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55761ed15d20 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7feb9d7fa640 1 -- 172.21.15.22:0/383887861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febb8001bc0 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7feb9d7fa640 1 -- 172.21.15.22:0/383887861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7febb00618e0 con 0x55761e8de490 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 10 client.6447 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 10 client.6447 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:43:16.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 20 client.6447 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761ed08480 0x55761ed6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 4 client.6447 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.924+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55761ecebf00 con 0x55761ed08480 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7febbdd9b640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761ed08480 0x55761ed6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7feb9d7fa640 10 client.6447 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7febbdd9b640 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761ed08480 0x55761ed6a3c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505f68640e00b1f2 server_cookie=872a095714f08e56 in_seq=0 out_seq=0 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7feb9d7fa640 1 -- 172.21.15.22:0/383887861 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55761ecebf00 con 0x55761ed08480 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7feb9d7fa640 10 client.6447 handle_command_reply: tid=0 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7febcbe9d740 1 client.6447 shutdown 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 msgr2=0x55761ed587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.925+0000 7febcbe9d740 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ed587e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7febb000cfd0 tx=0x7febb0078df0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 shutdown_connections 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55761ed08480 0x55761ed6a3c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55761e907350 0x55761ed07780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55761e8e6670 0x55761ed07240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 --2- 172.21.15.22:0/383887861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55761e8de490 0x55761ed587e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7feb9d7fa640 0 client.6447 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 >> 172.21.15.22:0/383887861 conn(0x55761e8fb4a0 msgr2=0x55761ece9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 shutdown_connections 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 1 -- 172.21.15.22:0/383887861 wait complete. 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febcbe9d740 20 client.6447 trim_cache size 0 max 16384 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:16.932 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "damage_type": "dentry", 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 2310217743, 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627776, 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "frag": "*", 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "dname": "file_to_be_damaged", 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "snap_id": "head", 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:16.933 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feba0078bc0 msgr2=0x7feba007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feba0078bc0 0x7feba007b0d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7febb4033000 tx=0x7febb4004730 comp rx=0 tx=0).stop 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 msgr2=0x7febc014e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc014e220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7febb000ba80 tx=0x7febb000bf40 comp rx=0 tx=0).stop 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 shutdown_connections 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feba0078bc0 0x7feba007b0d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7febc01e87a0 0x7febc0150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7febc00d3870 0x7febc01e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 --2- 172.21.15.22:0/3857777291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7febc00b3540 0x7febc014e220 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:16.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 >> 172.21.15.22:0/3857777291 conn(0x7febc008bfc0 msgr2=0x7febc006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:16.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 shutdown_connections 2024-09-21T05:43:16.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:16.926+0000 7febc4f33640 1 -- 172.21.15.22:0/3857777291 wait complete. 2024-09-21T05:43:16.940 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:43:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 -- 172.21.15.22:0/2352949281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c162f60 msgr2=0x7f540c165380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 --2- 172.21.15.22:0/2352949281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c162f60 0x7f540c165380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5404005fd0 tx=0x7f540402a5d0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 -- 172.21.15.22:0/2352949281 shutdown_connections 2024-09-21T05:43:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 --2- 172.21.15.22:0/2352949281 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c162f60 0x7f540c165380 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 --2- 172.21.15.22:0/2352949281 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f540c0b6100 0x7f540c0b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 -- 172.21.15.22:0/2352949281 >> 172.21.15.22:0/2352949281 conn(0x7f540c0d12a0 msgr2=0x7f540c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 -- 172.21.15.22:0/2352949281 shutdown_connections 2024-09-21T05:43:17.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.053+0000 7f541137b640 1 -- 172.21.15.22:0/2352949281 wait complete. 2024-09-21T05:43:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.055+0000 7f541137b640 1 Processor -- start 2024-09-21T05:43:17.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.055+0000 7f541137b640 1 -- start start 2024-09-21T05:43:17.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f541137b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f540c0b6100 0x7f540c0c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f541137b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 0x7f540c163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f541137b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f540c162f60 0x7f540c1f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f541137b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f540c165a40 con 0x7f540c0b6680 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f541137b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f540c1658c0 con 0x7f540c162f60 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f541137b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f540c165bc0 con 0x7f540c0b6100 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f540a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 0x7f540c163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f540a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 0x7f540c163bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58906/0 (socket says 172.21.15.22:58906) 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.056+0000 7f540a7fc640 1 -- 172.21.15.22:0/48368969 learned_addr learned my addr 172.21.15.22:0/48368969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540affd640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f540c0b6100 0x7f540c0c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540b7fe640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f540c162f60 0x7f540c1f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540a7fc640 1 -- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f540c0b6100 msgr2=0x7f540c0c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540a7fc640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f540c0b6100 0x7f540c0c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540a7fc640 1 -- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f540c162f60 msgr2=0x7f540c1f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540a7fc640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f540c162f60 0x7f540c1f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540a7fc640 1 -- 172.21.15.22:0/48368969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540c15c920 con 0x7f540c0b6680 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540b7fe640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f540c162f60 0x7f540c1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f540a7fc640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 0x7f540c163bf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f53f400d700 tx=0x7f53f400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4017070 con 0x7f540c0b6680 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f4005310 con 0x7f540c0b6680 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540c0c80a0 con 0x7f540c0b6680 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4009820 con 0x7f540c0b6680 2024-09-21T05:43:17.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.057+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540c15bff0 con 0x7f540c0b6680 2024-09-21T05:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.058+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f540c0b0de0 con 0x7f540c0b6680 2024-09-21T05:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.059+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53f40099c0 con 0x7f540c0b6680 2024-09-21T05:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.060+0000 7f54037fe640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53ec078b90 0x7f53ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.060+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f53f409afc0 con 0x7f540c0b6680 2024-09-21T05:43:17.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.060+0000 7f540affd640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53ec078b90 0x7f53ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.060+0000 7f540affd640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53ec078b90 0x7f53ec07b0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f540c0ceda0 tx=0x7f53fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.065+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53f40632e0 con 0x7f540c0b6680 2024-09-21T05:43:17.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.292+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f540c0ae200 con 0x7f540c0b6680 2024-09-21T05:43:17.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.292+0000 7f54037fe640 1 -- 172.21.15.22:0/48368969 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f53f4062be0 con 0x7f540c0b6680 2024-09-21T05:43:17.295 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:17.295 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53ec078b90 msgr2=0x7f53ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53ec078b90 0x7f53ec07b0a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f540c0ceda0 tx=0x7f53fc009040 comp rx=0 tx=0).stop 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 msgr2=0x7f540c163bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 0x7f540c163bf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f53f400d700 tx=0x7f53f400dbc0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 shutdown_connections 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f540c162f60 0x7f540c1f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f540c0b6680 0x7f540c163bf0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53ec078b90 0x7f53ec07b0a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 --2- 172.21.15.22:0/48368969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f540c0b6100 0x7f540c0c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 >> 172.21.15.22:0/48368969 conn(0x7f540c0d12a0 msgr2=0x7f540c0cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 shutdown_connections 2024-09-21T05:43:17.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.295+0000 7f541137b640 1 -- 172.21.15.22:0/48368969 wait complete. 2024-09-21T05:43:17.308 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T05:43:17.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:17.308 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-09-21T05:43:17.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 -- 172.21.15.22:0/3476931211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b006ddf0 msgr2=0x7f89b006e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3476931211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b006ddf0 0x7f89b006e200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f89b00731e0 tx=0x7f89a802a5f0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 -- 172.21.15.22:0/3476931211 shutdown_connections 2024-09-21T05:43:17.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3476931211 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b006ddf0 0x7f89b006e200 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3476931211 >> v2:172.21.15.134:3300/0 conn(0x7f89b015b6c0 0x7f89b0071bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 -- 172.21.15.22:0/3476931211 >> 172.21.15.22:0/3476931211 conn(0x7f89b0073700 msgr2=0x7f89b0073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 -- 172.21.15.22:0/3476931211 shutdown_connections 2024-09-21T05:43:17.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.427+0000 7f89b59ba640 1 -- 172.21.15.22:0/3476931211 wait complete. 2024-09-21T05:43:17.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.429+0000 7f89b59ba640 1 Processor -- start 2024-09-21T05:43:17.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.429+0000 7f89b59ba640 1 -- start start 2024-09-21T05:43:17.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89b59ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b006ddf0 0x7f89b01e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89b59ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b00cf990 0x7f89b01e7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89b59ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 0x7f89b01f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89b59ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89b00b5070 con 0x7f89b015b6c0 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89b59ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89b00b4ef0 con 0x7f89b00cf990 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89b59ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89b00b51f0 con 0x7f89b006ddf0 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 0x7f89b01f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 0x7f89b01f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58920/0 (socket says 172.21.15.22:58920) 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89af7fe640 1 -- 172.21.15.22:0/3443580706 learned_addr learned my addr 172.21.15.22:0/3443580706 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b00cf990 0x7f89b01e7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.430+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b006ddf0 0x7f89b01e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89af7fe640 1 -- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b006ddf0 msgr2=0x7f89b01e6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89af7fe640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b006ddf0 0x7f89b01e6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89af7fe640 1 -- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b00cf990 msgr2=0x7f89b01e7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89af7fe640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b00cf990 0x7f89b01e7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89af7fe640 1 -- 172.21.15.22:0/3443580706 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b015c9a0 con 0x7f89b015b6c0 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b00cf990 0x7f89b01e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b006ddf0 0x7f89b01e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89af7fe640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 0x7f89b01f5b60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f89a8034040 tx=0x7f89a8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89a7fff640 1 -- 172.21.15.22:0/3443580706 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a803e030 con 0x7f89b015b6c0 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89a7fff640 1 -- 172.21.15.22:0/3443580706 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a8003d10 con 0x7f89b015b6c0 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89a7fff640 1 -- 172.21.15.22:0/3443580706 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8035040 con 0x7f89b015b6c0 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b015cc80 con 0x7f89b015b6c0 2024-09-21T05:43:17.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.431+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b015d240 con 0x7f89b015b6c0 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.433+0000 7f89a7fff640 1 -- 172.21.15.22:0/3443580706 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f89a8041030 con 0x7f89b015b6c0 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.433+0000 7f89a7fff640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8990078b90 0x7f899007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.433+0000 7f89a7fff640 1 -- 172.21.15.22:0/3443580706 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f89a80bb8a0 con 0x7f89b015b6c0 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.434+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8990078b90 0x7f899007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.434+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8990078b90 0x7f899007b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89a0005fd0 tx=0x7f89a0009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.434+0000 7f89bca00740 1 Processor -- start 2024-09-21T05:43:17.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.434+0000 7f89bca00740 1 -- start start 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89bca00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 0x5591ff50f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff4f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89bca00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591ff43b0b0 con 0x5591ff120680 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591ff42a4e0 con 0x5591ff0f77c0 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591ff42e2f0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 0x5591ff50f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 0x5591ff50f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58922/0 (socket says 172.21.15.22:58922) 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 -- 172.21.15.22:0/3569392542 learned_addr learned my addr 172.21.15.22:0/3569392542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89af7fe640 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff4f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 -- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 msgr2=0x5591ff4f2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4f2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 -- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 msgr2=0x5591ff4f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff4f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.436+0000 7f89aeffd640 1 -- 172.21.15.22:0/3569392542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ff200a50 con 0x5591ff120680 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.437+0000 7f89aeffd640 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 0x5591ff50f540 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f89a00117e0 tx=0x7f89a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d332a76ff8add4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.437+0000 7f89a57fa640 1 -- 172.21.15.22:0/3569392542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a000e940 con 0x5591ff120680 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.437+0000 7f89a57fa640 1 -- 172.21.15.22:0/3569392542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a000ebd0 con 0x5591ff120680 2024-09-21T05:43:17.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.437+0000 7f89a57fa640 1 -- 172.21.15.22:0/3569392542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a001e420 con 0x5591ff120680 2024-09-21T05:43:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 -- 172.21.15.22:0/3569392542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 msgr2=0x5591ff50f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 0x5591ff50f540 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f89a00117e0 tx=0x7f89a0013000 comp rx=0 tx=0).stop 2024-09-21T05:43:17.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 -- 172.21.15.22:0/3569392542 shutdown_connections 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4f2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff4f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 --2- 172.21.15.22:0/3569392542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff120680 0x5591ff50f540 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 -- 172.21.15.22:0/3569392542 >> 172.21.15.22:0/3569392542 conn(0x5591ff1147d0 msgr2=0x5591ff4c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 -- 172.21.15.22:0/3569392542 shutdown_connections 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.438+0000 7f89bca00740 1 -- 172.21.15.22:0/3569392542 wait complete. 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.439+0000 7f89bca00740 1 -- start start 2024-09-21T05:43:17.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.439+0000 7f89bca00740 1 Processor -- start 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff50dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff503c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff120680 0x5591ff504180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591ff19aa30 con 0x5591ff0ff9a0 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591ff3b09d0 con 0x5591ff0f77c0 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591ff504f40 con 0x5591ff120680 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89bca00740 10 client.0 _ll_register_callbacks cb 0x5591ff424ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff503c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff503c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58926/0 (socket says 172.21.15.22:58926) 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89ae7fc640 1 -- 172.21.15.22:0/916546305 learned_addr learned my addr 172.21.15.22:0/916546305 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89af7fe640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff120680 0x5591ff504180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.440+0000 7f89aeffd640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff50dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89ae7fc640 1 -- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff120680 msgr2=0x5591ff504180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff120680 0x5591ff504180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89ae7fc640 1 -- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 msgr2=0x5591ff50dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff50dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89ae7fc640 1 -- 172.21.15.22:0/916546305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ff169990 con 0x5591ff0ff9a0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff503c40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8998001df0 tx=0x7f8998011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdaf57b98e9c812 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f898e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f898e7fc640 1 -- 172.21.15.22:0/916546305 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89980050d0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f898e7fc640 1 -- 172.21.15.22:0/916546305 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f8998005270 con 0x5591ff0ff9a0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a000ebd0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f898e7fc640 1 -- 172.21.15.22:0/916546305 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8998022050 con 0x5591ff0ff9a0 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89bca00740 10 client.6465 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:17.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5591ff2611e0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f898e7fc640 1 -- 172.21.15.22:0/916546305 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a000ebd0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f898e7fc640 1 -- 172.21.15.22:0/916546305 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f8998025020 con 0x5591ff0ff9a0 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89bca00740 10 client.6465 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89bca00740 10 client.6465 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.441+0000 7f89bca00740 20 client.6465 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.442+0000 7f89bca00740 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff0df250 0x5591ff51c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.442+0000 7f89bca00740 4 client.6465 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.442+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591ff43d000 con 0x5591ff0df250 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.442+0000 7f89aeffd640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff0df250 0x5591ff51c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.442+0000 7f898e7fc640 10 client.6465 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.442+0000 7f89aeffd640 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff0df250 0x5591ff51c500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bdd5b0f7395f146 server_cookie=56bdb75727aa255e in_seq=0 out_seq=0 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f898e7fc640 1 -- 172.21.15.22:0/916546305 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591ff43d000 con 0x5591ff0df250 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f898e7fc640 10 client.6465 handle_command_reply: tid=0 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 client.6465 shutdown 2024-09-21T05:43:17.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 msgr2=0x5591ff503c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff503c40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8998001df0 tx=0x7f8998011620 comp rx=0 tx=0).stop 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 shutdown_connections 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff0df250 0x5591ff51c500 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff120680 0x5591ff504180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff503c40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f898e7fc640 0 client.6465 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 --2- 172.21.15.22:0/916546305 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff0f77c0 0x5591ff50dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.443+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 >> 172.21.15.22:0/916546305 conn(0x5591ff1147d0 msgr2=0x5591ff4c1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.444+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 shutdown_connections 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.444+0000 7f89bca00740 1 -- 172.21.15.22:0/916546305 wait complete. 2024-09-21T05:43:17.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.444+0000 7f89bca00740 20 client.6465 trim_cache size 0 max 16384 2024-09-21T05:43:17.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.462+0000 7f89bca00740 1 Processor -- start 2024-09-21T05:43:17.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.462+0000 7f89bca00740 1 -- start start 2024-09-21T05:43:17.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0f77c0 0x5591ff4ed930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89bca00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff4ede70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff525c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89bca00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01f3dc0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f89b013f0f0 con 0x5591ff120680 2024-09-21T05:43:17.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f89b01f7590 con 0x5591ff0f77c0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff4ede70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff4ede70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58934/0 (socket says 172.21.15.22:58934) 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 -- 172.21.15.22:0/586956690 learned_addr learned my addr 172.21.15.22:0/586956690 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89aeffd640 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0f77c0 0x5591ff4ed930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89af7fe640 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff525c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 -- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0f77c0 msgr2=0x5591ff4ed930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0f77c0 0x5591ff4ed930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 -- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 msgr2=0x5591ff525c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff525c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.463+0000 7f89ae7fc640 1 -- 172.21.15.22:0/586956690 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ff1fed50 con 0x5591ff0ff9a0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.464+0000 7f89aeffd640 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0f77c0 0x5591ff4ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.464+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff4ede70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8998007ae0 tx=0x7f8998011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2031d664cf8b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.464+0000 7f89a4ff9640 1 -- 172.21.15.22:0/586956690 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899802a070 con 0x5591ff0ff9a0 2024-09-21T05:43:17.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.464+0000 7f89a4ff9640 1 -- 172.21.15.22:0/586956690 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899801a070 con 0x5591ff0ff9a0 2024-09-21T05:43:17.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.464+0000 7f89a4ff9640 1 -- 172.21.15.22:0/586956690 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998005330 con 0x5591ff0ff9a0 2024-09-21T05:43:17.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 -- 172.21.15.22:0/586956690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 msgr2=0x5591ff4ede70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff4ede70 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8998007ae0 tx=0x7f8998011b20 comp rx=0 tx=0).stop 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 -- 172.21.15.22:0/586956690 shutdown_connections 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff525c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0ff9a0 0x5591ff4ede70 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 --2- 172.21.15.22:0/586956690 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0f77c0 0x5591ff4ed930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 -- 172.21.15.22:0/586956690 >> 172.21.15.22:0/586956690 conn(0x5591ff1147d0 msgr2=0x5591ff4bde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 -- 172.21.15.22:0/586956690 shutdown_connections 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.465+0000 7f89bca00740 1 -- 172.21.15.22:0/586956690 wait complete. 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.466+0000 7f89bca00740 1 -- start start 2024-09-21T05:43:17.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.466+0000 7f89bca00740 1 Processor -- start 2024-09-21T05:43:17.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 0x5591ff50ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4ed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff4edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f89b013eaa0 con 0x5591ff0f77c0 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591ff42a6d0 con 0x5591ff120680 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591ff52efa0 con 0x5591ff0ff9a0 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89bca00740 10 client.0 _ll_register_callbacks cb 0x7f8998001a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 0x5591ff50ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 0x5591ff50ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58946/0 (socket says 172.21.15.22:58946) 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89aeffd640 1 -- 172.21.15.22:0/705122627 learned_addr learned my addr 172.21.15.22:0/705122627 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89af7fe640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff4edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89aeffd640 1 -- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 msgr2=0x5591ff4ed850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.467+0000 7f89aeffd640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4ed850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89aeffd640 1 -- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 msgr2=0x5591ff4edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89aeffd640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff4edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89aeffd640 1 -- 172.21.15.22:0/705122627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ff4c2620 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89aeffd640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 0x5591ff50ce80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f89a00117b0 tx=0x7f89a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe78f721f571acfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f898e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f898e7fc640 1 -- 172.21.15.22:0/705122627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0011e40 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f898e7fc640 1 -- 172.21.15.22:0/705122627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f89a001e420 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f899801a070 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f898e7fc640 1 -- 172.21.15.22:0/705122627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f89a0046050 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 10 client.6471 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5591ff58da40 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f898e7fc640 1 -- 172.21.15.22:0/705122627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f899801a070 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f898e7fc640 1 -- 172.21.15.22:0/705122627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f89a0049020 con 0x5591ff0f77c0 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 10 client.6471 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 10 client.6471 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 20 client.6471 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:17.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff58cc60 0x5591ff58d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 4 client.6471 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.468+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x5591ff438ab0 con 0x5591ff58cc60 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.469+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff58cc60 0x5591ff58d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.469+0000 7f898e7fc640 10 client.6471 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.469+0000 7f89ae7fc640 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff58cc60 0x5591ff58d070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2102a028e2b7aa5d server_cookie=2085cebe5f79a194 in_seq=0 out_seq=0 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f898e7fc640 1 -- 172.21.15.22:0/705122627 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591ff438ab0 con 0x5591ff58cc60 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f898e7fc640 10 client.6471 handle_command_reply: tid=0 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 client.6471 shutdown 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 msgr2=0x5591ff50ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 0x5591ff50ce80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f89a00117b0 tx=0x7f89a0009840 comp rx=0 tx=0).stop 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 shutdown_connections 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x5591ff58cc60 0x5591ff58d070 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ff120680 0x5591ff4edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ff0ff9a0 0x5591ff4ed850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f898e7fc640 0 client.6471 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 --2- 172.21.15.22:0/705122627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ff0f77c0 0x5591ff50ce80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 >> 172.21.15.22:0/705122627 conn(0x5591ff1147d0 msgr2=0x5591ff4bde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 shutdown_connections 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 1 -- 172.21.15.22:0/705122627 wait complete. 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.470+0000 7f89bca00740 20 client.6471 trim_cache size 0 max 16384 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "06868d00-359e-40fa-9579-d150aaf1de2d", 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:43:17.477 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:43:17.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8990078b90 msgr2=0x7f899007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8990078b90 0x7f899007b0a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89a0005fd0 tx=0x7f89a0009500 comp rx=0 tx=0).stop 2024-09-21T05:43:17.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 msgr2=0x7f89b01f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 0x7f89b01f5b60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f89a8034040 tx=0x7f89a8003820 comp rx=0 tx=0).stop 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 shutdown_connections 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f89b015b6c0 0x7f89b01f5b60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f89b00cf990 0x7f89b01e7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8990078b90 0x7f899007b0a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 --2- 172.21.15.22:0/3443580706 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f89b006ddf0 0x7f89b01e6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 >> 172.21.15.22:0/3443580706 conn(0x7f89b0073700 msgr2=0x7f89b00b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 shutdown_connections 2024-09-21T05:43:17.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.471+0000 7f89b59ba640 1 -- 172.21.15.22:0/3443580706 wait complete. 2024-09-21T05:43:17.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:17.484 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-21T05:43:17.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.600+0000 7fc2c205c640 1 -- 172.21.15.22:0/458919262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 msgr2=0x7fc2bc0d47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.600+0000 7fc2c205c640 1 --2- 172.21.15.22:0/458919262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0d47d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009830 tx=0x7fc2b002a510 comp rx=0 tx=0).stop 2024-09-21T05:43:17.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.600+0000 7fc2c205c640 1 -- 172.21.15.22:0/458919262 shutdown_connections 2024-09-21T05:43:17.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.600+0000 7fc2c205c640 1 --2- 172.21.15.22:0/458919262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0d47d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.600+0000 7fc2c205c640 1 --2- 172.21.15.22:0/458919262 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2bc0b39a0 0x7fc2bc0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.600+0000 7fc2c205c640 1 -- 172.21.15.22:0/458919262 >> 172.21.15.22:0/458919262 conn(0x7fc2bc08bfc0 msgr2=0x7fc2bc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.601+0000 7fc2c205c640 1 -- 172.21.15.22:0/458919262 shutdown_connections 2024-09-21T05:43:17.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.601+0000 7fc2c205c640 1 -- 172.21.15.22:0/458919262 wait complete. 2024-09-21T05:43:17.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.602+0000 7fc2c205c640 1 Processor -- start 2024-09-21T05:43:17.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.603+0000 7fc2c205c640 1 -- start start 2024-09-21T05:43:17.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2c205c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2bc0b39a0 0x7fc2bc1e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2c205c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2bc0b4850 0x7fc2bc0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2c205c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2c205c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0b5370 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2c205c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0b9720 con 0x7fc2bc0b4850 2024-09-21T05:43:17.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2c205c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc0b54f0 con 0x7fc2bc0b39a0 2024-09-21T05:43:17.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58960/0 (socket says 172.21.15.22:58960) 2024-09-21T05:43:17.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 -- 172.21.15.22:0/2676929333 learned_addr learned my addr 172.21.15.22:0/2676929333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2bc0b39a0 0x7fc2bc1e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2baffd640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2bc0b4850 0x7fc2bc0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 -- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2bc0b39a0 msgr2=0x7fc2bc1e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2bc0b39a0 0x7fc2bc1e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 -- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2bc0b4850 msgr2=0x7fc2bc0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2bc0b4850 0x7fc2bc0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.604+0000 7fc2bbfff640 1 -- 172.21.15.22:0/2676929333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2bc1e66b0 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.605+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0caed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400c7b0 tx=0x7fc2b400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.605+0000 7fc2a3fff640 1 -- 172.21.15.22:0/2676929333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4017070 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.605+0000 7fc2a3fff640 1 -- 172.21.15.22:0/2676929333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2b4003c60 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.605+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2bc0cc930 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.605+0000 7fc2a3fff640 1 -- 172.21.15.22:0/2676929333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b40052d0 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.605+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2bc0b2700 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.607+0000 7fc2a3fff640 1 -- 172.21.15.22:0/2676929333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc2b401a070 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.608+0000 7fc2a3fff640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc2a8078b90 0x7fc2a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.608+0000 7fc2a3fff640 1 -- 172.21.15.22:0/2676929333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fc2b409a3b0 con 0x7fc2bc0d43c0 2024-09-21T05:43:17.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.608+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc2a8078b90 0x7fc2a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.608+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc2a8078b90 0x7fc2a807b0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005fd0 tx=0x7fc2ac00b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.608+0000 7fc2c8fc9740 1 Processor -- start 2024-09-21T05:43:17.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.609+0000 7fc2c8fc9740 1 -- start start 2024-09-21T05:43:17.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac67a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac3c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56296ab95a60 con 0x56296a8524d0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56296ab84e90 con 0x56296a85a6b0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56296ab88ca0 con 0x56296a87b390 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac3c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac3c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58964/0 (socket says 172.21.15.22:58964) 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 -- 172.21.15.22:0/453583536 learned_addr learned my addr 172.21.15.22:0/453583536 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac67a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac3cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 -- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 msgr2=0x56296ac67a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac67a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 -- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 msgr2=0x56296ac3cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac3cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.610+0000 7fc2baffd640 1 -- 172.21.15.22:0/453583536 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296a95d7d0 con 0x56296a8524d0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.611+0000 7fc2baffd640 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac3c860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc2b002ab00 tx=0x7fc2b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18232d86fa77986f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.611+0000 7fc2a17fa640 1 -- 172.21.15.22:0/453583536 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00037f0 con 0x56296a8524d0 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.611+0000 7fc2a17fa640 1 -- 172.21.15.22:0/453583536 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2b0040030 con 0x56296a8524d0 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.611+0000 7fc2a17fa640 1 -- 172.21.15.22:0/453583536 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0003d20 con 0x56296a8524d0 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/453583536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 msgr2=0x56296ac3c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac3c860 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc2b002ab00 tx=0x7fc2b00046c0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/453583536 shutdown_connections 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac3c860 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/453583536 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac67a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/453583536 >> 172.21.15.22:0/453583536 conn(0x56296a86f4e0 msgr2=0x56296ac22d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/453583536 shutdown_connections 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.612+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/453583536 wait complete. 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.613+0000 7fc2c8fc9740 1 -- start start 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.613+0000 7fc2c8fc9740 1 Processor -- start 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac24a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac24f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56296a8f5740 con 0x56296a8524d0 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56296ab2a970 con 0x56296a85a6b0 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56296ac61b40 con 0x56296a87b390 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2c8fc9740 10 client.0 _ll_register_callbacks cb 0x56296ab39780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac649b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58980/0 (socket says 172.21.15.22:58980) 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/656247886 learned_addr learned my addr 172.21.15.22:0/656247886 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.614+0000 7fc2baffd640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac24a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac24f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 msgr2=0x56296ac24f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac24f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 msgr2=0x56296ac24a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac24a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/656247886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296a8e6970 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2baffd640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac24a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac24f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac649b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac011720 tx=0x7fc2ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24609ac2f31f2dfb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc29a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc29a7fc640 1 -- 172.21.15.22:0/656247886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01e700 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc29a7fc640 1 -- 172.21.15.22:0/656247886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fc2ac029b70 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b0040030 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc29a7fc640 1 -- 172.21.15.22:0/656247886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac029d70 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2c8fc9740 10 client.6486 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56296a8c46a0 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.615+0000 7fc29a7fc640 1 -- 172.21.15.22:0/656247886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2b0040030 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc29a7fc640 1 -- 172.21.15.22:0/656247886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7fc2ac030050 con 0x56296a8524d0 2024-09-21T05:43:17.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2c8fc9740 10 client.6486 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2c8fc9740 10 client.6486 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2c8fc9740 20 client.6486 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a839f60 0x56296ac27f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2c8fc9740 4 client.6486 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56296ab979b0 con 0x56296a839f60 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2baffd640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a839f60 0x56296ac27f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc2baffd640 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a839f60 0x56296ac27f90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29348b904c9bb530 server_cookie=8aa03da891e165f2 in_seq=0 out_seq=0 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.616+0000 7fc29a7fc640 10 client.6486 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc29a7fc640 1 -- 172.21.15.22:0/656247886 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56296ab979b0 con 0x56296a839f60 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc29a7fc640 10 client.6486 handle_command_reply: tid=0 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 client.6486 shutdown 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 msgr2=0x56296ac649b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac649b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac011720 tx=0x7fc2ac004730 comp rx=0 tx=0).stop 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 shutdown_connections 2024-09-21T05:43:17.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a839f60 0x56296ac27f90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a87b390 0x56296ac24f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a85a6b0 0x56296ac24a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc29a7fc640 0 client.6486 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/656247886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a8524d0 0x56296ac649b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 >> 172.21.15.22:0/656247886 conn(0x56296a86f4e0 msgr2=0x56296ac1da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.617+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 shutdown_connections 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.618+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/656247886 wait complete. 2024-09-21T05:43:17.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.618+0000 7fc2c8fc9740 20 client.6486 trim_cache size 0 max 16384 2024-09-21T05:43:17.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.638+0000 7fc2c8fc9740 1 Processor -- start 2024-09-21T05:43:17.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.638+0000 7fc2c8fc9740 1 -- start start 2024-09-21T05:43:17.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.639+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ac49b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.639+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac88280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.639+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.639+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc2023a0 con 0x56296a839f60 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.639+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0ce350 con 0x56296a8524d0 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2bc0cc3c0 con 0x56296a85a6b0 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ac49b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ac49b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58990/0 (socket says 172.21.15.22:58990) 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/3832799217 learned_addr learned my addr 172.21.15.22:0/3832799217 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2baffd640 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac88280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 msgr2=0x56296ac887c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac887c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 msgr2=0x56296ac88280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac88280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/3832799217 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56296ac5ef00 con 0x56296a839f60 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ac49b60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac010f70 tx=0x7fc2ac010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2e5af2e7f3b43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.640+0000 7fc2a0ff9640 1 -- 172.21.15.22:0/3832799217 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01e8f0 con 0x56296a839f60 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2a0ff9640 1 -- 172.21.15.22:0/3832799217 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2ac037020 con 0x56296a839f60 2024-09-21T05:43:17.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2a0ff9640 1 -- 172.21.15.22:0/3832799217 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01e700 con 0x56296a839f60 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/3832799217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 msgr2=0x56296ac49b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ac49b60 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac010f70 tx=0x7fc2ac010fa0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/3832799217 shutdown_connections 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac887c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac88280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/3832799217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ac49b60 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/3832799217 >> 172.21.15.22:0/3832799217 conn(0x56296a86f4e0 msgr2=0x56296ac4cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/3832799217 shutdown_connections 2024-09-21T05:43:17.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.641+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/3832799217 wait complete. 2024-09-21T05:43:17.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.642+0000 7fc2c8fc9740 1 -- start start 2024-09-21T05:43:17.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.642+0000 7fc2c8fc9740 1 Processor -- start 2024-09-21T05:43:17.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ace62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac6d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac6db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2bc0cdd00 con 0x56296a839f60 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56296ac3b860 con 0x56296a8524d0 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56296ac6e3a0 con 0x56296a85a6b0 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2c8fc9740 10 client.0 _ll_register_callbacks cb 0x7fc2ac010b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ace62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ace62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59002/0 (socket says 172.21.15.22:59002) 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/4205841431 learned_addr learned my addr 172.21.15.22:0/4205841431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2baffd640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac6d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bbfff640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac6db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 msgr2=0x56296ac6db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac6db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.650 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 msgr2=0x56296ac6d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac6d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 -- 172.21.15.22:0/4205841431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56296ac6f270 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc2bb7fe640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ace62b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac010f10 tx=0x7fc2ac01e790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2e719a358ff734 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.644+0000 7fc29a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc29a7fc640 1 -- 172.21.15.22:0/4205841431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01eb80 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc29a7fc640 1 -- 172.21.15.22:0/4205841431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fc2ac03db70 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac037020 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc29a7fc640 1 -- 172.21.15.22:0/4205841431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac03dd70 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 10 client.6492 fetch_fsmap learned FSMap version 52 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x56296ac1da60 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc29a7fc640 1 -- 172.21.15.22:0/4205841431 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2ac037020 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc29a7fc640 1 -- 172.21.15.22:0/4205841431 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7fc2ac044050 con 0x56296a839f60 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 10 client.6492 fetch_fsmap finished waiting for FSMap version 52 2024-09-21T05:43:17.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 10 client.6492 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 20 client.6492 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a80f3a0 0x56296ac6b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 4 client.6492 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.645+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56296ab93460 con 0x56296a80f3a0 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2baffd640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a80f3a0 0x56296ac6b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc29a7fc640 10 client.6492 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2baffd640 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a80f3a0 0x56296ac6b3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb00bcaaeec4d770 server_cookie=e9f6f150fee1f68a in_seq=0 out_seq=0 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc29a7fc640 1 -- 172.21.15.22:0/4205841431 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+873 (crc 0 0 0) 0x56296ab93460 con 0x56296a80f3a0 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc29a7fc640 10 client.6492 handle_command_reply: tid=0 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 client.6492 shutdown 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 msgr2=0x56296ace62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ace62b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac010f10 tx=0x7fc2ac01e790 comp rx=0 tx=0).stop 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 shutdown_connections 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x56296a80f3a0 0x56296ac6b3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56296a85a6b0 0x56296ac6db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56296a8524d0 0x56296ac6d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc29a7fc640 0 client.6492 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 --2- 172.21.15.22:0/4205841431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56296a839f60 0x56296ace62b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 >> 172.21.15.22:0/4205841431 conn(0x56296a86f4e0 msgr2=0x56296ac3c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 shutdown_connections 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.646+0000 7fc2c8fc9740 1 -- 172.21.15.22:0/4205841431 wait complete. 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c8fc9740 20 client.6492 trim_cache size 0 max 16384 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:43:17.653 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "06868d00-359e-40fa-9579-d150aaf1de2d": { 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "06868d00-359e-40fa-9579-d150aaf1de2d", 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,repair" 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "06868d00-359e-40fa-9579-d150aaf1de2d": { 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:43:17.469430+0000", 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:43:17.654 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "6e53fcbf-899b-41d9-a24f-0b7e296ee3af": { 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:42:46.015156+0000", 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:17.655 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc2a8078b90 msgr2=0x7fc2a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc2a8078b90 0x7fc2a807b0a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005fd0 tx=0x7fc2ac00b390 comp rx=0 tx=0).stop 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 msgr2=0x7fc2bc0caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0caed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400c7b0 tx=0x7fc2b400cc70 comp rx=0 tx=0).stop 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 shutdown_connections 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2bc0d43c0 0x7fc2bc0caed0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2bc0b4850 0x7fc2bc0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.656 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc2a8078b90 0x7fc2a807b0a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 --2- 172.21.15.22:0/2676929333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2bc0b39a0 0x7fc2bc1e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:17.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 >> 172.21.15.22:0/2676929333 conn(0x7fc2bc08bfc0 msgr2=0x7fc2bc06f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:17.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 shutdown_connections 2024-09-21T05:43:17.657 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:17.647+0000 7fc2c205c640 1 -- 172.21.15.22:0/2676929333 wait complete. 2024-09-21T05:43:17.660 INFO:tasks.cephfs.filesystem:scrub status for tag:06868d00-359e-40fa-9579-d150aaf1de2d - {'path': '/subdir', 'tag': '06868d00-359e-40fa-9579-d150aaf1de2d', 'options': 'recursive,repair'} 2024-09-21T05:43:21.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:43:21.767 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:43:47.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:47.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.794+0000 7f21de60a640 1 -- 172.21.15.22:0/1047457924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21c0004800 msgr2=0x7f21c0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.794+0000 7f21de60a640 1 --2- 172.21.15.22:0/1047457924 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21c0004800 0x7f21c0004c10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f21d80ced00 tx=0x7f21c0005830 comp rx=0 tx=0).stop 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.794+0000 7f21dcba7640 1 -- 172.21.15.22:0/1047457924 reap_dead start 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.795+0000 7f21de60a640 1 -- 172.21.15.22:0/1047457924 shutdown_connections 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.795+0000 7f21de60a640 1 -- 172.21.15.22:0/1047457924 >> 172.21.15.22:0/1047457924 conn(0x7f21d80cdcc0 msgr2=0x7f21d80ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.795+0000 7f21de60a640 1 -- 172.21.15.22:0/1047457924 shutdown_connections 2024-09-21T05:43:47.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.795+0000 7f21de60a640 1 -- 172.21.15.22:0/1047457924 wait complete. 2024-09-21T05:43:47.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.796+0000 7f21de60a640 1 Processor -- start 2024-09-21T05:43:47.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.797+0000 7f21de60a640 1 -- start start 2024-09-21T05:43:47.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21de60a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21c0004800 0x7f21d81f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21de60a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21d80ccc00 0x7f21d809c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21de60a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 0x7f21d809e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21de60a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21d80b46a0 con 0x7f21d809ca30 2024-09-21T05:43:47.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21de60a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21d80b39e0 con 0x7f21d80ccc00 2024-09-21T05:43:47.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21de60a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21d80b4820 con 0x7f21c0004800 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 0x7f21d809e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 0x7f21d809e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59078/0 (socket says 172.21.15.22:59078) 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 -- 172.21.15.22:0/1896109230 learned_addr learned my addr 172.21.15.22:0/1896109230 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21d77fe640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21d80ccc00 0x7f21d809c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21d7fff640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21c0004800 0x7f21d81f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 -- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21c0004800 msgr2=0x7f21d81f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21c0004800 0x7f21d81f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 -- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21d80ccc00 msgr2=0x7f21d809c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21d80ccc00 0x7f21d809c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.798+0000 7f21dcba7640 1 -- 172.21.15.22:0/1896109230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d806cf80 con 0x7f21d809ca30 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.799+0000 7f21dcba7640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 0x7f21d809e4f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f21d815f2e0 tx=0x7f21d000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.799+0000 7f21d4ff9640 1 -- 172.21.15.22:0/1896109230 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0004b20 con 0x7f21d809ca30 2024-09-21T05:43:47.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.799+0000 7f21d4ff9640 1 -- 172.21.15.22:0/1896109230 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d0004cc0 con 0x7f21d809ca30 2024-09-21T05:43:47.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.799+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d806d260 con 0x7f21d809ca30 2024-09-21T05:43:47.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.799+0000 7f21d4ff9640 1 -- 172.21.15.22:0/1896109230 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0003bd0 con 0x7f21d809ca30 2024-09-21T05:43:47.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.799+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d806d820 con 0x7f21d809ca30 2024-09-21T05:43:47.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.801+0000 7f21d4ff9640 1 -- 172.21.15.22:0/1896109230 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21d0004030 con 0x7f21d809ca30 2024-09-21T05:43:47.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.801+0000 7f21d4ff9640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21c4078b90 0x7f21c407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.802+0000 7f21d4ff9640 1 -- 172.21.15.22:0/1896109230 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f21d009b2d0 con 0x7f21d809ca30 2024-09-21T05:43:47.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.802+0000 7f21d7fff640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21c4078b90 0x7f21c407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.802+0000 7f21e5651740 1 Processor -- start 2024-09-21T05:43:47.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.802+0000 7f21d7fff640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21c4078b90 0x7f21c407b0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21c8000c00 tx=0x7f21c8006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.802+0000 7f21e5651740 1 -- start start 2024-09-21T05:43:47.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63493f390 0x55e634d2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6349164d0 0x55e634d00860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21e5651740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d00da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21e5651740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e634c59a60 con 0x55e63491e6b0 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e634c48e90 con 0x55e6349164d0 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e634c4cca0 con 0x55e63493f390 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d00da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d00da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59092/0 (socket says 172.21.15.22:59092) 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 -- 172.21.15.22:0/2341433223 learned_addr learned my addr 172.21.15.22:0/2341433223 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21d7fff640 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63493f390 0x55e634d2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21d77fe640 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6349164d0 0x55e634d00860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 -- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63493f390 msgr2=0x55e634d2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63493f390 0x55e634d2ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 -- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6349164d0 msgr2=0x55e634d00860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6349164d0 0x55e634d00860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 -- 172.21.15.22:0/2341433223 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e634a217d0 con 0x55e63491e6b0 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21dcba7640 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d00da0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f21d00996e0 tx=0x7f21d0062f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617ceccc9b4ac4db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21bdffb640 1 -- 172.21.15.22:0/2341433223 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d001e040 con 0x55e63491e6b0 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21bdffb640 1 -- 172.21.15.22:0/2341433223 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d0067af0 con 0x55e63491e6b0 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.804+0000 7f21bdffb640 1 -- 172.21.15.22:0/2341433223 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d006c9a0 con 0x55e63491e6b0 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 -- 172.21.15.22:0/2341433223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 msgr2=0x55e634d00da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d00da0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f21d00996e0 tx=0x7f21d0062f10 comp rx=0 tx=0).stop 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 -- 172.21.15.22:0/2341433223 shutdown_connections 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d00da0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e6349164d0 0x55e634d00860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 --2- 172.21.15.22:0/2341433223 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63493f390 0x55e634d2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 -- 172.21.15.22:0/2341433223 >> 172.21.15.22:0/2341433223 conn(0x55e6349334e0 msgr2=0x55e634ce6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 -- 172.21.15.22:0/2341433223 shutdown_connections 2024-09-21T05:43:47.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.805+0000 7f21e5651740 1 -- 172.21.15.22:0/2341433223 wait complete. 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.806+0000 7f21e5651740 1 -- start start 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.806+0000 7f21e5651740 1 Processor -- start 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634d21330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d21870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d1ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55e6349b9740 con 0x55e63491e6b0 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e634bee970 con 0x55e63493f390 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e634d1ff60 con 0x55e6349164d0 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21e5651740 10 client.0 _ll_register_callbacks cb 0x55e634bfd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d21870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d21870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59104/0 (socket says 172.21.15.22:59104) 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 -- 172.21.15.22:0/707463051 learned_addr learned my addr 172.21.15.22:0/707463051 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:47.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d7fff640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634d21330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21dcba7640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d1ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 -- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 msgr2=0x55e634d21330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634d21330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 -- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 msgr2=0x55e634d1ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d1ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 -- 172.21.15.22:0/707463051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6349aa970 con 0x55e63491e6b0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d7fff640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634d21330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21dcba7640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d1ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21d77fe640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d21870 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c350 tx=0x7f21cc0011f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67c9f2a857c90c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.808+0000 7f21b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21b6ffd640 1 -- 172.21.15.22:0/707463051 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc032070 con 0x55e63491e6b0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21b6ffd640 1 -- 172.21.15.22:0/707463051 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f21cc006460 con 0x55e63491e6b0 2024-09-21T05:43:47.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21d0067af0 con 0x55e63491e6b0 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21b6ffd640 1 -- 172.21.15.22:0/707463051 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f21cc001cb0 con 0x55e63491e6b0 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 10 client.6504 fetch_fsmap learned FSMap version 53 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55e6349886a0 con 0x55e63491e6b0 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21b6ffd640 1 -- 172.21.15.22:0/707463051 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d0067af0 con 0x55e63491e6b0 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21b6ffd640 1 -- 172.21.15.22:0/707463051 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f21cc04d080 con 0x55e63491e6b0 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 10 client.6504 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 10 client.6504 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 20 client.6504 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e6348fdf60 0x55e634d282d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 4 client.6504 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e634c5b9b0 con 0x55e6348fdf60 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.809+0000 7f21d7fff640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e6348fdf60 0x55e634d282d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.810+0000 7f21b6ffd640 10 client.6504 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:47.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.810+0000 7f21d7fff640 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e6348fdf60 0x55e634d282d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d7adfd6c625518 server_cookie=be1ff3934ee3b359 in_seq=0 out_seq=0 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21b6ffd640 1 -- 172.21.15.22:0/707463051 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e634c5b9b0 con 0x55e6348fdf60 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21b6ffd640 10 client.6504 handle_command_reply: tid=0 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 client.6504 shutdown 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 msgr2=0x55e634d21870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d21870 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c350 tx=0x7f21cc0011f0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 shutdown_connections 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e6348fdf60 0x55e634d282d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d1ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21b6ffd640 0 client.6504 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634d21870 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 --2- 172.21.15.22:0/707463051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634d21330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 >> 172.21.15.22:0/707463051 conn(0x55e6349334e0 msgr2=0x55e634ce09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 shutdown_connections 2024-09-21T05:43:47.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 1 -- 172.21.15.22:0/707463051 wait complete. 2024-09-21T05:43:47.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.811+0000 7f21e5651740 20 client.6504 trim_cache size 0 max 16384 2024-09-21T05:43:47.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.829+0000 7f21e5651740 1 Processor -- start 2024-09-21T05:43:47.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.829+0000 7f21e5651740 1 -- start start 2024-09-21T05:43:47.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.830+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634cfe830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.830+0000 7f21e5651740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634cfed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.830+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d0d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.830+0000 7f21e5651740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81fe030 con 0x55e63491e6b0 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.830+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8159060 con 0x55e63493f390 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.830+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21d809b6b0 con 0x55e6349164d0 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634cfed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634cfed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59106/0 (socket says 172.21.15.22:59106) 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 -- 172.21.15.22:0/2224541818 learned_addr learned my addr 172.21.15.22:0/2224541818 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d7fff640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634cfe830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21dcba7640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d0d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 -- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 msgr2=0x55e634cfe830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634cfe830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 -- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 msgr2=0x55e634d0d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d0d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 -- 172.21.15.22:0/2224541818 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e634d559b0 con 0x55e63491e6b0 2024-09-21T05:43:47.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d7fff640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634cfe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21dcba7640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d0d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21d77fe640 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634cfed70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f21cc003670 tx=0x7f21cc0036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e836ac311a15527 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21bd7fa640 1 -- 172.21.15.22:0/2224541818 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc032070 con 0x55e63491e6b0 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21bd7fa640 1 -- 172.21.15.22:0/2224541818 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21cc003940 con 0x55e63491e6b0 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.831+0000 7f21bd7fa640 1 -- 172.21.15.22:0/2224541818 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc003d90 con 0x55e63491e6b0 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 -- 172.21.15.22:0/2224541818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 msgr2=0x55e634cfed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634cfed70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f21cc003670 tx=0x7f21cc0036a0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 -- 172.21.15.22:0/2224541818 shutdown_connections 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d0d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e63491e6b0 0x55e634cfed70 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 --2- 172.21.15.22:0/2224541818 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e6349164d0 0x55e634cfe830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 -- 172.21.15.22:0/2224541818 >> 172.21.15.22:0/2224541818 conn(0x55e6349334e0 msgr2=0x55e634d00620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 -- 172.21.15.22:0/2224541818 shutdown_connections 2024-09-21T05:43:47.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.832+0000 7f21e5651740 1 -- 172.21.15.22:0/2224541818 wait complete. 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.833+0000 7f21e5651740 1 -- start start 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.833+0000 7f21e5651740 1 Processor -- start 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 0x55e634db3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63491e6b0 0x55e634da3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8158a10 con 0x55e6349164d0 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55e634ce3680 con 0x55e63493f390 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55e634d4e840 con 0x55e63491e6b0 2024-09-21T05:43:47.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 10 client.0 _ll_register_callbacks cb 0x7f21c00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 0x55e634db3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 0x55e634db3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59120/0 (socket says 172.21.15.22:59120) 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 -- 172.21.15.22:0/3896730254 learned_addr learned my addr 172.21.15.22:0/3896730254 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d77fe640 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63491e6b0 0x55e634da3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 -- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63491e6b0 msgr2=0x55e634da3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63491e6b0 0x55e634da3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 -- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 msgr2=0x55e634d2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 -- 172.21.15.22:0/3896730254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e634db1110 con 0x55e6349164d0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21d7fff640 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 0x55e634db3dd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55e634d22290 tx=0x7f21c8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47db2e646e1a26db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21b6ffd640 1 -- 172.21.15.22:0/3896730254 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c801f680 con 0x55e6349164d0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21b6ffd640 1 -- 172.21.15.22:0/3896730254 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f21c8027040 con 0x55e6349164d0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.835+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc003940 con 0x55e6349164d0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21b6ffd640 1 -- 172.21.15.22:0/3896730254 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f21c801f820 con 0x55e6349164d0 2024-09-21T05:43:47.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 10 client.6510 fetch_fsmap learned FSMap version 53 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55e634db2f50 con 0x55e6349164d0 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21b6ffd640 1 -- 172.21.15.22:0/3896730254 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21cc003940 con 0x55e6349164d0 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21b6ffd640 1 -- 172.21.15.22:0/3896730254 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f21c801f9c0 con 0x55e6349164d0 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 10 client.6510 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 10 client.6510 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 20 client.6510 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e634d2e080 0x55e634ce1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 4 client.6510 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e634c57460 con 0x55e634d2e080 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21d77fe640 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e634d2e080 0x55e634ce1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21b6ffd640 10 client.6510 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:47.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.836+0000 7f21d77fe640 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e634d2e080 0x55e634ce1a20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152a3cb43b46410a server_cookie=e1b48e82a321b2e0 in_seq=0 out_seq=0 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21b6ffd640 1 -- 172.21.15.22:0/3896730254 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+670 (crc 0 0 0) 0x55e634c57460 con 0x55e634d2e080 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21b6ffd640 10 client.6510 handle_command_reply: tid=0 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 client.6510 shutdown 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 msgr2=0x55e634db3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 0x55e634db3dd0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55e634d22290 tx=0x7f21c8001190 comp rx=0 tx=0).stop 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 shutdown_connections 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55e634d2e080 0x55e634ce1a20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55e63493f390 0x55e634d2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55e63491e6b0 0x55e634da3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21b6ffd640 0 client.6510 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 --2- 172.21.15.22:0/3896730254 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55e6349164d0 0x55e634db3dd0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 >> 172.21.15.22:0/3896730254 conn(0x55e6349334e0 msgr2=0x55e634d54f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 shutdown_connections 2024-09-21T05:43:47.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 1 -- 172.21.15.22:0/3896730254 wait complete. 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.837+0000 7f21e5651740 20 client.6510 trim_cache size 0 max 16384 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "06868d00-359e-40fa-9579-d150aaf1de2d": { 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:43:17.469430+0000", 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:43:47.844 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "6e53fcbf-899b-41d9-a24f-0b7e296ee3af": { 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:42:46.015156+0000", 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/subdir", 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:43:47.845 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:43:47.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21c4078b90 msgr2=0x7f21c407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21c4078b90 0x7f21c407b0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f21c8000c00 tx=0x7f21c8006b70 comp rx=0 tx=0).stop 2024-09-21T05:43:47.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 msgr2=0x7f21d809e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 0x7f21d809e4f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f21d815f2e0 tx=0x7f21d000f3c0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 shutdown_connections 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21d809ca30 0x7f21d809e4f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21d80ccc00 0x7f21d809c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21c4078b90 0x7f21c407b0a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 --2- 172.21.15.22:0/1896109230 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21c0004800 0x7f21d81f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 >> 172.21.15.22:0/1896109230 conn(0x7f21d80cdcc0 msgr2=0x7f21d80adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 shutdown_connections 2024-09-21T05:43:47.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.838+0000 7f21de60a640 1 -- 172.21.15.22:0/1896109230 wait complete. 2024-09-21T05:43:47.851 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:43:47.851 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.965+0000 7f0a9f38b640 1 -- 172.21.15.22:0/1860585859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d1a80 msgr2=0x7f0a980cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.965+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/1860585859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d1a80 0x7f0a980cf090 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0a980d32e0 tx=0x7f0a8802a5f0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.965+0000 7f0a9f38b640 1 -- 172.21.15.22:0/1860585859 shutdown_connections 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.965+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/1860585859 >> v2:172.21.15.134:3300/0 conn(0x7f0a980cf5d0 0x7f0a980cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.965+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/1860585859 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d1a80 0x7f0a980cf090 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.965+0000 7f0a9f38b640 1 -- 172.21.15.22:0/1860585859 >> 172.21.15.22:0/1860585859 conn(0x7f0a9808bfc0 msgr2=0x7f0a9809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.966+0000 7f0a9f38b640 1 -- 172.21.15.22:0/1860585859 shutdown_connections 2024-09-21T05:43:47.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.966+0000 7f0a9f38b640 1 -- 172.21.15.22:0/1860585859 wait complete. 2024-09-21T05:43:47.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.967+0000 7f0a9f38b640 1 Processor -- start 2024-09-21T05:43:47.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.967+0000 7f0a9f38b640 1 -- start start 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a980cf5d0 0x7f0a98151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980d1a80 0x7f0a98151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 0x7f0a981522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98162bb0 con 0x7f0a980d2ed0 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98162a30 con 0x7f0a980cf5d0 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0a98162d30 con 0x7f0a980d1a80 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 0x7f0a981522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 0x7f0a981522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59148/0 (socket says 172.21.15.22:59148) 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 -- 172.21.15.22:0/3662774620 learned_addr learned my addr 172.21.15.22:0/3662774620 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9c926640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980d1a80 0x7f0a98151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d127640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a980cf5d0 0x7f0a98151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 -- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980d1a80 msgr2=0x7f0a98151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980d1a80 0x7f0a98151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 -- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a980cf5d0 msgr2=0x7f0a98151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a980cf5d0 0x7f0a98151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 -- 172.21.15.22:0/3662774620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980c9b50 con 0x7f0a980d2ed0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9d928640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 0x7f0a981522c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000cab0 tx=0x7f0a9000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90007d30 con 0x7f0a980d2ed0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a9000f450 con 0x7f0a980d2ed0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a98151a70 con 0x7f0a980d2ed0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.969+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9000f8a0 con 0x7f0a980d2ed0 2024-09-21T05:43:47.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.970+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a98154d20 con 0x7f0a980d2ed0 2024-09-21T05:43:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.971+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a980ce270 con 0x7f0a980d2ed0 2024-09-21T05:43:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.971+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a9000fa40 con 0x7f0a980d2ed0 2024-09-21T05:43:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.972+0000 7f0a8dffb640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a7c078bc0 0x7f0a7c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:47.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.972+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f0a9009b7b0 con 0x7f0a980d2ed0 2024-09-21T05:43:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.972+0000 7f0a9d127640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a7c078bc0 0x7f0a7c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:47.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.973+0000 7f0a9d127640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a7c078bc0 0x7f0a7c07b0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a94000fd0 tx=0x7f0a94009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:47.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:47.977+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a90014070 con 0x7f0a980d2ed0 2024-09-21T05:43:48.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.202+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a981632c0 con 0x7f0a980d2ed0 2024-09-21T05:43:48.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.203+0000 7f0a8dffb640 1 -- 172.21.15.22:0/3662774620 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5230 (secure 0 0 0) 0x7f0a90063be0 con 0x7f0a980d2ed0 2024-09-21T05:43:48.205 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:48.205 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":53,"btime":"2024-09-21T05:43:21:876894+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:43:21.365281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:43:48.205 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 53 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a7c078bc0 msgr2=0x7f0a7c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a7c078bc0 0x7f0a7c07b0d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a94000fd0 tx=0x7f0a94009040 comp rx=0 tx=0).stop 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 msgr2=0x7f0a981522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 0x7f0a981522c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000cab0 tx=0x7f0a9000cf70 comp rx=0 tx=0).stop 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 shutdown_connections 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0a980d2ed0 0x7f0a981522c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0a7c078bc0 0x7f0a7c07b0d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0a980d1a80 0x7f0a98151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 --2- 172.21.15.22:0/3662774620 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0a980cf5d0 0x7f0a98151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 >> 172.21.15.22:0/3662774620 conn(0x7f0a9808bfc0 msgr2=0x7f0a98161be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 shutdown_connections 2024-09-21T05:43:48.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.205+0000 7f0a9f38b640 1 -- 172.21.15.22:0/3662774620 wait complete. 2024-09-21T05:43:48.219 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-21T05:43:48.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde07a94640 1 -- 172.21.15.22:0/3194303319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 msgr2=0x7fdde800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde07a94640 1 --2- 172.21.15.22:0/3194303319 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fdde800b9c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fddf8001690 tx=0x7fdde8005830 comp rx=0 tx=0).stop 2024-09-21T05:43:48.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde06031640 1 -- 172.21.15.22:0/3194303319 reap_dead start 2024-09-21T05:43:48.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde07a94640 1 -- 172.21.15.22:0/3194303319 shutdown_connections 2024-09-21T05:43:48.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde07a94640 1 -- 172.21.15.22:0/3194303319 >> 172.21.15.22:0/3194303319 conn(0x7fde0009d7f0 msgr2=0x7fde000d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde07a94640 1 -- 172.21.15.22:0/3194303319 shutdown_connections 2024-09-21T05:43:48.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.336+0000 7fde07a94640 1 -- 172.21.15.22:0/3194303319 wait complete. 2024-09-21T05:43:48.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.338+0000 7fde07a94640 1 Processor -- start 2024-09-21T05:43:48.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.338+0000 7fde07a94640 1 -- start start 2024-09-21T05:43:48.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde07a94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fde0014ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde07a94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde001607b0 0x7fde00150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde07a94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde00150c90 0x7fde000cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde07a94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde000d5110 con 0x7fdde8005270 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde07a94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde000d4f90 con 0x7fde001607b0 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde07a94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fde0006f1c0 con 0x7fde00150c90 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fde0014ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fde0014ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59162/0 (socket says 172.21.15.22:59162) 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 -- 172.21.15.22:0/2689555224 learned_addr learned my addr 172.21.15.22:0/2689555224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde0502f640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde001607b0 0x7fde00150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 -- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde00150c90 msgr2=0x7fde000cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde00150c90 0x7fde000cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 -- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde001607b0 msgr2=0x7fde00150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde001607b0 0x7fde00150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.339+0000 7fde05830640 1 -- 172.21.15.22:0/2689555224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde000caca0 con 0x7fdde8005270 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.340+0000 7fde05830640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fde0014ddf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006340 tx=0x7fddf0006800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.340+0000 7fddfe7fc640 1 -- 172.21.15.22:0/2689555224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0015070 con 0x7fdde8005270 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.340+0000 7fddfe7fc640 1 -- 172.21.15.22:0/2689555224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf000fce0 con 0x7fdde8005270 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.340+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde000c81d0 con 0x7fdde8005270 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.340+0000 7fddfe7fc640 1 -- 172.21.15.22:0/2689555224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0011690 con 0x7fdde8005270 2024-09-21T05:43:48.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.340+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0014e330 con 0x7fdde8005270 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.342+0000 7fddfe7fc640 1 -- 172.21.15.22:0/2689555224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fddf0011830 con 0x7fdde8005270 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.342+0000 7fddfe7fc640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddec078bc0 0x7fddec07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.342+0000 7fddfe7fc640 1 -- 172.21.15.22:0/2689555224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fddf009bd70 con 0x7fdde8005270 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.342+0000 7fde0502f640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddec078bc0 0x7fddec07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.343+0000 7fde0502f640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddec078bc0 0x7fddec07b0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde000d4c20 tx=0x7fddf4005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.343+0000 7fde0ea05740 1 Processor -- start 2024-09-21T05:43:48.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.343+0000 7fde0ea05740 1 -- start start 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.344+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 0x55934118ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x5593411ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde0ea05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5593410d79a0 con 0x559340dbd350 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5593410c6dd0 con 0x559340d94490 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5593410cabe0 con 0x559340d9c670 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 0x55934118ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 0x55934118ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59164/0 (socket says 172.21.15.22:59164) 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 -- 172.21.15.22:0/85317226 learned_addr learned my addr 172.21.15.22:0/85317226 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde0502f640 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde06031640 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x5593411ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 -- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 msgr2=0x5593411ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x5593411ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 -- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 msgr2=0x55934118f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 -- 172.21.15.22:0/85317226 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934105f700 con 0x559340dbd350 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde06031640 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x5593411ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fde05830640 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 0x55934118ed80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006990 tx=0x7fddf0063220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2242dc901e321811 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fdde77fe640 1 -- 172.21.15.22:0/85317226 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0020070 con 0x559340dbd350 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fdde77fe640 1 -- 172.21.15.22:0/85317226 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf0063ab0 con 0x559340dbd350 2024-09-21T05:43:48.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.345+0000 7fdde77fe640 1 -- 172.21.15.22:0/85317226 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0068590 con 0x559340dbd350 2024-09-21T05:43:48.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 -- 172.21.15.22:0/85317226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 msgr2=0x55934118ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 0x55934118ed80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006990 tx=0x7fddf0063220 comp rx=0 tx=0).stop 2024-09-21T05:43:48.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 -- 172.21.15.22:0/85317226 shutdown_connections 2024-09-21T05:43:48.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x5593411ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 --2- 172.21.15.22:0/85317226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340dbd350 0x55934118ed80 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 -- 172.21.15.22:0/85317226 >> 172.21.15.22:0/85317226 conn(0x559340db14a0 msgr2=0x559341165040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 -- 172.21.15.22:0/85317226 shutdown_connections 2024-09-21T05:43:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.346+0000 7fde0ea05740 1 -- 172.21.15.22:0/85317226 wait complete. 2024-09-21T05:43:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.347+0000 7fde0ea05740 1 -- start start 2024-09-21T05:43:48.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.347+0000 7fde0ea05740 1 Processor -- start 2024-09-21T05:43:48.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d9c670 0x5593411abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559340e37700 con 0x559340d94490 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55934106c340 con 0x559340d9c670 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55934118f800 con 0x559340dbd350 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 10 client.0 _ll_register_callbacks cb 0x55934106aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59166/0 (socket says 172.21.15.22:59166) 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 -- 172.21.15.22:0/2663026981 learned_addr learned my addr 172.21.15.22:0/2663026981 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0502f640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d9c670 0x5593411abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 -- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 msgr2=0x5593411ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde06031640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 -- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d9c670 msgr2=0x5593411abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d9c670 0x5593411abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 -- 172.21.15.22:0/2663026981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559340ea17c0 con 0x559340d94490 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0502f640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d9c670 0x5593411abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde05830640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c8b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fddf009a1e0 tx=0x7fddf0062fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed78ab75e4a2b7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fdde4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fdde4ff9640 1 -- 172.21.15.22:0/2663026981 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0020070 con 0x559340d94490 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fdde4ff9640 1 -- 172.21.15.22:0/2663026981 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fddf0011d30 con 0x559340d94490 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.349+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf0063ab0 con 0x559340d94490 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fdde4ff9640 1 -- 172.21.15.22:0/2663026981 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fddf0068590 con 0x559340d94490 2024-09-21T05:43:48.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 10 client.6531 fetch_fsmap learned FSMap version 53 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x559340f01f50 con 0x559340d94490 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fdde4ff9640 1 -- 172.21.15.22:0/2663026981 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf0063ab0 con 0x559340d94490 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fdde4ff9640 1 -- 172.21.15.22:0/2663026981 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fddf0079cc0 con 0x559340d94490 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 10 client.6531 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 10 client.6531 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 20 client.6531 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559340d7c130 0x5593411ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 4 client.6531 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593410d98f0 con 0x559340d7c130 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.350+0000 7fde0502f640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559340d7c130 0x5593411ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.351+0000 7fdde4ff9640 10 client.6531 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.351+0000 7fde0502f640 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559340d7c130 0x5593411ad910 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e982d23b716d971 server_cookie=ef937d1539a7aef3 in_seq=0 out_seq=0 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.351+0000 7fdde4ff9640 1 -- 172.21.15.22:0/2663026981 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593410d98f0 con 0x559340d7c130 2024-09-21T05:43:48.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.351+0000 7fdde4ff9640 10 client.6531 handle_command_reply: tid=0 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.351+0000 7fde0ea05740 1 client.6531 shutdown 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 msgr2=0x55934117c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c8b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fddf009a1e0 tx=0x7fddf0062fc0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 shutdown_connections 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x559340d7c130 0x5593411ad910 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d9c670 0x5593411abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fdde4ff9640 0 client.6531 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 --2- 172.21.15.22:0/2663026981 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c8b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 >> 172.21.15.22:0/2663026981 conn(0x559340db14a0 msgr2=0x5593411ab790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 shutdown_connections 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 1 -- 172.21.15.22:0/2663026981 wait complete. 2024-09-21T05:43:48.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.352+0000 7fde0ea05740 20 client.6531 trim_cache size 0 max 16384 2024-09-21T05:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.366+0000 7fde0ea05740 1 Processor -- start 2024-09-21T05:43:48.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.366+0000 7fde0ea05740 1 -- start start 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x55934117c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340dbd350 0x5593411a2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0ea05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde00201430 con 0x559340d94490 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde001e8f40 con 0x559340dbd350 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fde001e9590 con 0x559340d9c670 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59180/0 (socket says 172.21.15.22:59180) 2024-09-21T05:43:48.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 -- 172.21.15.22:0/3036750450 learned_addr learned my addr 172.21.15.22:0/3036750450 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0502f640 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x55934117c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde06031640 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340dbd350 0x5593411a2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 -- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 msgr2=0x55934117c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x55934117c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 -- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340dbd350 msgr2=0x5593411a2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340dbd350 0x5593411a2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde05830640 1 -- 172.21.15.22:0/3036750450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559340e06660 con 0x559340d94490 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.367+0000 7fde0502f640 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x55934117c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde05830640 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c490 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fddf009a260 tx=0x7fddf005f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f70fbaae772a8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fdde6ffd640 1 -- 172.21.15.22:0/3036750450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0020070 con 0x559340d94490 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fdde6ffd640 1 -- 172.21.15.22:0/3036750450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf0011d30 con 0x559340d94490 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fdde6ffd640 1 -- 172.21.15.22:0/3036750450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf0079bd0 con 0x559340d94490 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 -- 172.21.15.22:0/3036750450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 msgr2=0x55934117c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c490 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fddf009a260 tx=0x7fddf005f4d0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 -- 172.21.15.22:0/3036750450 shutdown_connections 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340dbd350 0x5593411a2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340d9c670 0x55934117c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 --2- 172.21.15.22:0/3036750450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d94490 0x55934117c490 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.368+0000 7fde0ea05740 1 -- 172.21.15.22:0/3036750450 >> 172.21.15.22:0/3036750450 conn(0x559340db14a0 msgr2=0x55934115ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.369+0000 7fde0ea05740 1 -- 172.21.15.22:0/3036750450 shutdown_connections 2024-09-21T05:43:48.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.369+0000 7fde0ea05740 1 -- 172.21.15.22:0/3036750450 wait complete. 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.369+0000 7fde0ea05740 1 -- start start 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.369+0000 7fde0ea05740 1 Processor -- start 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 0x55934121f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559340e37700 con 0x559340d9c670 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55934118f800 con 0x559340d94490 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55934106c340 con 0x559340dbd350 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0ea05740 10 client.0 _ll_register_callbacks cb 0x7fddf0063910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 0x55934121f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 0x55934121f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59184/0 (socket says 172.21.15.22:59184) 2024-09-21T05:43:48.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 -- 172.21.15.22:0/4003086276 learned_addr learned my addr 172.21.15.22:0/4003086276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde05830640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde06031640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 -- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 msgr2=0x5593411cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411cab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 -- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 msgr2=0x55934118eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde0502f640 1 -- 172.21.15.22:0/4003086276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593411a0970 con 0x559340d9c670 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.371+0000 7fde06031640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0502f640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 0x55934121f3f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5593411602b0 tx=0x7fddf4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f4d3ef0d3af365 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fdde4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fdde4ff9640 1 -- 172.21.15.22:0/4003086276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf4020430 con 0x559340d9c670 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fdde4ff9640 1 -- 172.21.15.22:0/4003086276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fddf40205d0 con 0x559340d9c670 2024-09-21T05:43:48.378 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf0011d30 con 0x559340d9c670 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fdde4ff9640 1 -- 172.21.15.22:0/4003086276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fddf4036050 con 0x559340d9c670 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 10 client.6537 fetch_fsmap learned FSMap version 53 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55934115fd80 con 0x559340d9c670 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fdde4ff9640 1 -- 172.21.15.22:0/4003086276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddf0011d30 con 0x559340d9c670 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fdde4ff9640 1 -- 172.21.15.22:0/4003086276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fddf4039020 con 0x559340d9c670 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 10 client.6537 fetch_fsmap finished waiting for FSMap version 53 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 10 client.6537 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 20 client.6537 populate_metadata read hostname 'smithi022' 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55934117ca90 0x55934122c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 4 client.6537 mds_command: new command op to 6153 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 --> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5593410d53a0 con 0x55934117ca90 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.372+0000 7fde05830640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55934117ca90 0x55934122c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fdde4ff9640 10 client.6537 ms_handle_connect on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde05830640 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55934117ca90 0x55934122c0a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a642c984402351 server_cookie=7f0abfa93930ee0e in_seq=0 out_seq=0 2024-09-21T05:43:48.379 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fdde4ff9640 1 -- 172.21.15.22:0/4003086276 <== mds.0 v2:172.21.15.134:6834/2827334455 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593410d53a0 con 0x55934117ca90 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fdde4ff9640 10 client.6537 handle_command_reply: tid=0 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 client.6537 shutdown 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 msgr2=0x55934121f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 0x55934121f3f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5593411602b0 tx=0x7fddf4015040 comp rx=0 tx=0).stop 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 shutdown_connections 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:6834/2827334455,v1:172.21.15.134:6836/2827334455] conn(0x55934117ca90 0x55934122c0a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559340dbd350 0x5593411cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559340d9c670 0x55934121f3f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 --2- 172.21.15.22:0/4003086276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559340d94490 0x55934118eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 >> 172.21.15.22:0/4003086276 conn(0x559340db14a0 msgr2=0x55934115ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fdde4ff9640 0 client.6537 ms_handle_reset on v2:172.21.15.134:6834/2827334455 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 shutdown_connections 2024-09-21T05:43:48.380 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.373+0000 7fde0ea05740 1 -- 172.21.15.22:0/4003086276 wait complete. 2024-09-21T05:43:48.381 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde0ea05740 20 client.6537 trim_cache size 0 max 16384 2024-09-21T05:43:48.381 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:48.381 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddec078bc0 msgr2=0x7fddec07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddec078bc0 0x7fddec07b0d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde000d4c20 tx=0x7fddf4005f90 comp rx=0 tx=0).stop 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 msgr2=0x7fde0014ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fde0014ddf0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fddf0006340 tx=0x7fddf0006800 comp rx=0 tx=0).stop 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 shutdown_connections 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fddec078bc0 0x7fddec07b0d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde00150c90 0x7fde000cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde001607b0 0x7fde00150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 --2- 172.21.15.22:0/2689555224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdde8005270 0x7fde0014ddf0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 >> 172.21.15.22:0/2689555224 conn(0x7fde0009d7f0 msgr2=0x7fde000b8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 shutdown_connections 2024-09-21T05:43:48.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.374+0000 7fde07a94640 1 -- 172.21.15.22:0/2689555224 wait complete. 2024-09-21T05:43:48.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:43:48.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 -- 172.21.15.22:0/2190947177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc156c50 msgr2=0x7f03fc0731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 --2- 172.21.15.22:0/2190947177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc156c50 0x7f03fc0731a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009830 tx=0x7f03f002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 -- 172.21.15.22:0/2190947177 shutdown_connections 2024-09-21T05:43:48.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 --2- 172.21.15.22:0/2190947177 >> v2:172.21.15.134:3300/0 conn(0x7f03fc0736e0 0x7f03fc0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 --2- 172.21.15.22:0/2190947177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc156c50 0x7f03fc0731a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 -- 172.21.15.22:0/2190947177 >> 172.21.15.22:0/2190947177 conn(0x7f03fc06cde0 msgr2=0x7f03fc06d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 -- 172.21.15.22:0/2190947177 shutdown_connections 2024-09-21T05:43:48.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.500+0000 7f0401c8b640 1 -- 172.21.15.22:0/2190947177 wait complete. 2024-09-21T05:43:48.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.502+0000 7f0401c8b640 1 Processor -- start 2024-09-21T05:43:48.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.502+0000 7f0401c8b640 1 -- start start 2024-09-21T05:43:48.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f0401c8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc06fad0 0x7f03fc14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f0401c8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 0x7f03fc15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f0401c8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc156c50 0x7f03fc15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f0401c8b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0cb3e0 con 0x7f03fc0736e0 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f0401c8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0cb260 con 0x7f03fc156c50 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f0401c8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0cb560 con 0x7f03fc06fad0 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f03faffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 0x7f03fc15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f03faffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 0x7f03fc15d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59192/0 (socket says 172.21.15.22:59192) 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.503+0000 7f03faffd640 1 -- 172.21.15.22:0/167874915 learned_addr learned my addr 172.21.15.22:0/167874915 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03fb7fe640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc06fad0 0x7f03fc14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03fbfff640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc156c50 0x7f03fc15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03faffd640 1 -- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc06fad0 msgr2=0x7f03fc14f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03faffd640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc06fad0 0x7f03fc14f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03faffd640 1 -- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc156c50 msgr2=0x7f03fc15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03faffd640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc156c50 0x7f03fc15d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03faffd640 1 -- 172.21.15.22:0/167874915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc1f7020 con 0x7f03fc0736e0 2024-09-21T05:43:48.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03faffd640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 0x7f03fc15d3e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f03f002aa90 tx=0x7f03f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0005c90 con 0x7f03fc0736e0 2024-09-21T05:43:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03f0005020 con 0x7f03fc0736e0 2024-09-21T05:43:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc14fe10 con 0x7f03fc0736e0 2024-09-21T05:43:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0005450 con 0x7f03fc0736e0 2024-09-21T05:43:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.504+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1f75a0 con 0x7f03fc0736e0 2024-09-21T05:43:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.505+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc073ff0 con 0x7f03fc0736e0 2024-09-21T05:43:48.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.506+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03f00317c0 con 0x7f03fc0736e0 2024-09-21T05:43:48.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.507+0000 7f03e3fff640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f03e8078b90 0x7f03e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.507+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f03f00bbd60 con 0x7f03fc0736e0 2024-09-21T05:43:48.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.507+0000 7f03fb7fe640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f03e8078b90 0x7f03e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.507+0000 7f03fb7fe640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f03e8078b90 0x7f03e807b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f03ec001250 tx=0x7f03ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.511+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03f003f030 con 0x7f03fc0736e0 2024-09-21T05:43:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.740+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f03fc0c6580 con 0x7f03fc0736e0 2024-09-21T05:43:48.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.741+0000 7f03e3fff640 1 -- 172.21.15.22:0/167874915 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f03f0084230 con 0x7f03fc0736e0 2024-09-21T05:43:48.743 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:48.743 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f03e8078b90 msgr2=0x7f03e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f03e8078b90 0x7f03e807b0a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f03ec001250 tx=0x7f03ec009040 comp rx=0 tx=0).stop 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 msgr2=0x7f03fc15d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 0x7f03fc15d3e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f03f002aa90 tx=0x7f03f0003c10 comp rx=0 tx=0).stop 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 shutdown_connections 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03fc156c50 0x7f03fc15d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03fc0736e0 0x7f03fc15d3e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f03e8078b90 0x7f03e807b0a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 --2- 172.21.15.22:0/167874915 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f03fc06fad0 0x7f03fc14f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 >> 172.21.15.22:0/167874915 conn(0x7f03fc06cde0 msgr2=0x7f03fc0c6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 shutdown_connections 2024-09-21T05:43:48.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.743+0000 7f0401c8b640 1 -- 172.21.15.22:0/167874915 wait complete. 2024-09-21T05:43:48.756 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T05:43:48.756 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:43:48.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 -- 172.21.15.22:0/3396572627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 msgr2=0x7f62040b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 --2- 172.21.15.22:0/3396572627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040b5fa0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f620408cab0 tx=0x7f61f802a440 comp rx=0 tx=0).stop 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 -- 172.21.15.22:0/3396572627 shutdown_connections 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 --2- 172.21.15.22:0/3396572627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 0x7f62040d25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 --2- 172.21.15.22:0/3396572627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040b5fa0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 --2- 172.21.15.22:0/3396572627 >> v2:172.21.15.134:3300/0 conn(0x7f62040b9b60 0x7f62040d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 -- 172.21.15.22:0/3396572627 >> 172.21.15.22:0/3396572627 conn(0x7f620409d7f0 msgr2=0x7f62040d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 -- 172.21.15.22:0/3396572627 shutdown_connections 2024-09-21T05:43:48.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.871+0000 7f620b20a640 1 -- 172.21.15.22:0/3396572627 wait complete. 2024-09-21T05:43:48.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.873+0000 7f620b20a640 1 Processor -- start 2024-09-21T05:43:48.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.873+0000 7f620b20a640 1 -- start start 2024-09-21T05:43:48.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f620b20a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 0x7f620415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f620b20a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040b9b60 0x7f62040ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f620b20a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f620b20a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f62040d3020 con 0x7f62040cf540 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f620b20a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f62040d2ea0 con 0x7f62040b9b60 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f620b20a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f62040d31a0 con 0x7f62040b64e0 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f62097a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f62097a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59206/0 (socket says 172.21.15.22:59206) 2024-09-21T05:43:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f62097a7640 1 -- 172.21.15.22:0/4115329117 learned_addr learned my addr 172.21.15.22:0/4115329117 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f6208fa6640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 0x7f620415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.874+0000 7f6203fff640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040b9b60 0x7f62040ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62097a7640 1 -- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 msgr2=0x7f620415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62097a7640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 0x7f620415f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62097a7640 1 -- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040b9b60 msgr2=0x7f62040ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62097a7640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040b9b60 0x7f62040ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62097a7640 1 -- 172.21.15.22:0/4115329117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620415fad0 con 0x7f62040cf540 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f6208fa6640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 0x7f620415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62097a7640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040ca710 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ea80 tx=0x7f61fc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc00ce40 con 0x7f62040cf540 2024-09-21T05:43:48.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61fc004c10 con 0x7f62040cf540 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62040a1aa0 con 0x7f62040cf540 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc005020 con 0x7f62040cf540 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.875+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62040c96f0 con 0x7f62040cf540 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.876+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62040d4d80 con 0x7f62040cf540 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.877+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f61fc0051c0 con 0x7f62040cf540 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.877+0000 7f62017fa640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078b90 0x7f61f007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:48.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.878+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f61fc09bb20 con 0x7f62040cf540 2024-09-21T05:43:48.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.878+0000 7f6208fa6640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078b90 0x7f61f007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:48.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.878+0000 7f6208fa6640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078b90 0x7f61f007b0a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005fd0 tx=0x7f61f4005ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:48.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:48.882+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61fc063ec0 con 0x7f62040cf540 2024-09-21T05:43:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.108+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f62040a1830 con 0x7f62040cf540 2024-09-21T05:43:49.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.108+0000 7f62017fa640 1 -- 172.21.15.22:0/4115329117 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f61fc0637c0 con 0x7f62040cf540 2024-09-21T05:43:49.111 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078b90 msgr2=0x7f61f007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078b90 0x7f61f007b0a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005fd0 tx=0x7f61f4005ee0 comp rx=0 tx=0).stop 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 msgr2=0x7f62040ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040ca710 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ea80 tx=0x7f61fc00ef40 comp rx=0 tx=0).stop 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 shutdown_connections 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f62040cf540 0x7f62040ca710 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:49.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f62040b9b60 0x7f62040ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f61f0078b90 0x7f61f007b0a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 --2- 172.21.15.22:0/4115329117 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f62040b64e0 0x7f620415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 >> 172.21.15.22:0/4115329117 conn(0x7f620409d7f0 msgr2=0x7f62040b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 shutdown_connections 2024-09-21T05:43:49.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49.111+0000 7f620b20a640 1 -- 172.21.15.22:0/4115329117 wait complete. 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:43:49.124 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:43:49.125 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:43:49.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.125 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:43:49.142 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:43:49.142 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:43:49.142 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:43:49.142 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:43:49.143 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42515sec preferred_lft 42515sec 2024-09-21T05:43:49.144 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:43:49.145 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.145 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:43:49.145 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:43:49.145 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:43:49.145 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:43:49.145 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:43:49.146 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:43:49.146 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.146 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:43:49.146 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.146 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:43:49.146 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:43:49.147 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:43:49.147 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.147 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:43:49.147 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:43:49.147 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:43:49.147 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:43:49.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.148 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:43:49.206 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:43:49.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.208 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:43:49.265 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:43:49.265 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:43:49.265 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:43:49.265 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:43:49.265 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:43:49.265 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:43:49.265 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:43:49.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:43:49.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:43:49.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:43:49.266 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:43:49.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:43:49.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:43:49.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.394 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:43:49.410 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:43:49.410 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:43:49.411 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:43:49.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.411 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:43:49.476 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:43:49.477 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:43:49.533 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:43:49.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:43:49.534 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:43:49.589 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:43:49.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:43:49.686 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:43:49.686 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:43:49.686 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:43:49.697 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:43:49.697 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:43:49.697 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:43:49.697 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:43:49.697 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:43:49.703 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:43:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:43:49.705 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:43:49.705 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:43:49.706 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:43:49.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.706 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:43:49.732 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:43:49.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:43:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:43:49.830 INFO:teuthology.orchestra.run.smithi022.stdout:client5750 2024-09-21T05:43:49.831 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:43:49.831 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:43:49.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:49.847 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-21T05:43:49.906 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:43:49.906 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-21T05:43:50.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.003+0000 7f1a9fbc8f40 1 Processor -- start 2024-09-21T05:43:50.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.003+0000 7f1a9fbc8f40 1 -- start start 2024-09-21T05:43:50.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.004+0000 7f1a9fbc8f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a5489d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.004+0000 7f1a9fbc8f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5489120 0x55a6a5520510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.004+0000 7f1a9fbc8f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 0x55a6a548e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.004+0000 7f1a9fbc8f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a6a536a430 con 0x55a6a5520a50 2024-09-21T05:43:50.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.004+0000 7f1a9fbc8f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a6a53771f0 con 0x7f1a840022d0 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.004+0000 7f1a9fbc8f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55a6a530bd60 con 0x55a6a5489120 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 0x55a6a548e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 0x55a6a548e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59224/0 (socket says 172.21.15.22:59224) 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 -- 172.21.15.22:0/433657006 learned_addr learned my addr 172.21.15.22:0/433657006 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99682640 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a5489d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a98e81640 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5489120 0x55a6a5520510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 -- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5489120 msgr2=0x55a6a5520510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5489120 0x55a6a5520510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 -- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 msgr2=0x55a6a5489d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a5489d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 -- 172.21.15.22:0/433657006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a548d5e0 con 0x55a6a5520a50 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a99e83640 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 0x55a6a548e8f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55a6a5404080 tx=0x7f1a940098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f7cc6fdf417e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a8b7fe640 1 -- 172.21.15.22:0/433657006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94009280 con 0x55a6a5520a50 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a8b7fe640 1 -- 172.21.15.22:0/433657006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a940074c0 con 0x55a6a5520a50 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.005+0000 7f1a8b7fe640 1 -- 172.21.15.22:0/433657006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94007910 con 0x55a6a5520a50 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/433657006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 msgr2=0x55a6a548e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 0x55a6a548e8f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x55a6a5404080 tx=0x7f1a940098b0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/433657006 shutdown_connections 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5520a50 0x55a6a548e8f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5489120 0x55a6a5520510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/433657006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a5489d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/433657006 >> 172.21.15.22:0/433657006 conn(0x55a6a52dcac0 msgr2=0x55a6a53be0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/433657006 shutdown_connections 2024-09-21T05:43:50.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.006+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/433657006 wait complete. 2024-09-21T05:43:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.007+0000 7f1a9fbc8f40 1 Processor -- start 2024-09-21T05:43:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.007+0000 7f1a9fbc8f40 1 -- start start 2024-09-21T05:43:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a9fbc8f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5489120 0x55a6a5482320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a9fbc8f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5520a50 0x55a6a552bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a9fbc8f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a552c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a9fbc8f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a6a548a260 con 0x55a6a5489120 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a9fbc8f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a6a53bd490 con 0x7f1a840022d0 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a9fbc8f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55a6a5402820 con 0x55a6a5520a50 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a99682640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5489120 0x55a6a5482320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a99682640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5489120 0x55a6a5482320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59238/0 (socket says 172.21.15.22:59238) 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a99682640 1 -- 172.21.15.22:0/874490357 learned_addr learned my addr 172.21.15.22:0/874490357 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a98e81640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5520a50 0x55a6a552bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.008+0000 7f1a99e83640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a552c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a99682640 1 -- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5520a50 msgr2=0x55a6a552bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a99682640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a6a5520a50 0x55a6a552bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a99682640 1 -- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 msgr2=0x55a6a552c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a99682640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a840022d0 0x55a6a552c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a99682640 1 -- 172.21.15.22:0/874490357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6a5483d90 con 0x55a6a5489120 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a99682640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5489120 0x55a6a5482320 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c046880 tx=0x7f1a8c009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.013 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a8a7fc640 1 -- 172.21.15.22:0/874490357 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c009790 con 0x55a6a5489120 2024-09-21T05:43:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a8a7fc640 1 -- 172.21.15.22:0/874490357 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a8c04dcd0 con 0x55a6a5489120 2024-09-21T05:43:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a6a53bc370 con 0x55a6a5489120 2024-09-21T05:43:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a8a7fc640 1 -- 172.21.15.22:0/874490357 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c060630 con 0x55a6a5489120 2024-09-21T05:43:50.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.009+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6a5482860 con 0x55a6a5489120 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.011+0000 7f1a8a7fc640 1 -- 172.21.15.22:0/874490357 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a8c053030 con 0x55a6a5489120 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.011+0000 7f1a8a7fc640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1a7c0775a0 0x7f1a7c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a8a7fc640 1 -- 172.21.15.22:0/874490357 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f1a8c0de080 con 0x55a6a5489120 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a98e81640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1a7c0775a0 0x7f1a7c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55a6a5401ea0 0x55a6a54022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) -- 0x55a6a5535170 con 0x55a6a5401ea0 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a99e83640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55a6a5401ea0 0x55a6a54022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a98e81640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1a7c0775a0 0x7f1a7c079ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a6a53babb0 tx=0x7f1a90004a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.012+0000 7f1a99e83640 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55a6a5401ea0 0x55a6a54022b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a99e83640 1 -- 172.21.15.22:0/874490357 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v71'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1a940025b0 con 0x55a6a5401ea0 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55a6a5401ea0 msgr2=0x55a6a54022b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55a6a5401ea0 0x55a6a54022b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1a7c0775a0 msgr2=0x7f1a7c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1a7c0775a0 0x7f1a7c079ab0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55a6a53babb0 tx=0x7f1a90004a00 comp rx=0 tx=0).stop 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5489120 msgr2=0x55a6a5482320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 --2- 172.21.15.22:0/874490357 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a6a5489120 0x55a6a5482320 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c046880 tx=0x7f1a8c009ac0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a99e83640 1 -- 172.21.15.22:0/874490357 reap_dead start 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 shutdown_connections 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 >> 172.21.15.22:0/874490357 conn(0x55a6a52dcac0 msgr2=0x55a6a548e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 shutdown_connections 2024-09-21T05:43:50.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.014+0000 7f1a9fbc8f40 1 -- 172.21.15.22:0/874490357 wait complete. 2024-09-21T05:43:50.019 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:43:50.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:43:50.056 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:43:50.105 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:43:50.105 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:43:50.105 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:43:50.105 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:43:50.121 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:43:50.121 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:43:50.178 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:43:50.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1060732452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 msgr2=0x7fcbb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1060732452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbb000b9c0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc009ed0 tx=0x7fcbb0005830 comp rx=0 tx=0).stop 2024-09-21T05:43:50.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1060732452 shutdown_connections 2024-09-21T05:43:50.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1060732452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbb000b9c0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1060732452 >> 172.21.15.22:0/1060732452 conn(0x7fcbc4072ae0 msgr2=0x7fcbc4072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:50.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1060732452 shutdown_connections 2024-09-21T05:43:50.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.334+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1060732452 wait complete. 2024-09-21T05:43:50.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.336+0000 7fcbcbf5f640 1 Processor -- start 2024-09-21T05:43:50.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.336+0000 7fcbcbf5f640 1 -- start start 2024-09-21T05:43:50.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbcbf5f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbc40a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbcbf5f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbc40c8010 0x7fcbc41f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbcbf5f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbc41f3f80 0x7fcbc40cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbcbf5f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc415ca70 con 0x7fcbb0005250 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbcbf5f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc415c8f0 con 0x7fcbc41f3f80 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbcbf5f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc415cbf0 con 0x7fcbc40c8010 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbc9cfb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbc40a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbc9cfb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbc40a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59256/0 (socket says 172.21.15.22:59256) 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.337+0000 7fcbc9cfb640 1 -- 172.21.15.22:0/1813612389 learned_addr learned my addr 172.21.15.22:0/1813612389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc94fa640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbc40c8010 0x7fcbc41f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbca4fc640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbc41f3f80 0x7fcbc40cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc9cfb640 1 -- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbc40c8010 msgr2=0x7fcbc41f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc9cfb640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbc40c8010 0x7fcbc41f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc9cfb640 1 -- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbc41f3f80 msgr2=0x7fcbc40cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc9cfb640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbc41f3f80 0x7fcbc40cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc9cfb640 1 -- 172.21.15.22:0/1813612389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc40cabf0 con 0x7fcbb0005250 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc94fa640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbc40c8010 0x7fcbc41f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbca4fc640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbc41f3f80 0x7fcbc40cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:50.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbc9cfb640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbc40a94a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003f00 tx=0x7fcbb8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8011640 con 0x7fcbb0005250 2024-09-21T05:43:50.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcbb8016020 con 0x7fcbb0005250 2024-09-21T05:43:50.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc40caed0 con 0x7fcbb0005250 2024-09-21T05:43:50.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8011a80 con 0x7fcbb0005250 2024-09-21T05:43:50.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.338+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc40cb490 con 0x7fcbb0005250 2024-09-21T05:43:50.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.339+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc40d1000 con 0x7fcbb0005250 2024-09-21T05:43:50.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.340+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcbb8011c20 con 0x7fcbb0005250 2024-09-21T05:43:50.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.340+0000 7fcbb67fc640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcba8078b90 0x7fcba807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.340+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fcbb809b990 con 0x7fcbb0005250 2024-09-21T05:43:50.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.341+0000 7fcbc94fa640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcba8078b90 0x7fcba807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.341+0000 7fcbc94fa640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcba8078b90 0x7fcba807b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0001240 tx=0x7fcbc0001110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.345+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcbb8020020 con 0x7fcbb0005250 2024-09-21T05:43:50.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.547+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcbc40b3940 con 0x7fcbb0005250 2024-09-21T05:43:50.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.548+0000 7fcbb67fc640 1 -- 172.21.15.22:0/1813612389 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) ==== 80+0+124 (secure 0 0 0) 0x7fcbb8063dd0 con 0x7fcbb0005250 2024-09-21T05:43:50.550 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6837/1820431436 2024-09-22T05:42:38.349537+0000 2024-09-21T05:43:50.550 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6835/1820431436 2024-09-22T05:42:38.349537+0000 2024-09-21T05:43:50.550 INFO:teuthology.orchestra.run.smithi022.stderr:listed 2 entries 2024-09-21T05:43:50.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcba8078b90 msgr2=0x7fcba807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcba8078b90 0x7fcba807b0a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fcbc0001240 tx=0x7fcbc0001110 comp rx=0 tx=0).stop 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 msgr2=0x7fcbc40a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbc40a94a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003f00 tx=0x7fcbb8001630 comp rx=0 tx=0).stop 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 shutdown_connections 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcbc41f3f80 0x7fcbc40cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcba8078b90 0x7fcba807b0a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcbc40c8010 0x7fcbc41f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 --2- 172.21.15.22:0/1813612389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcbb0005250 0x7fcbc40a94a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 >> 172.21.15.22:0/1813612389 conn(0x7fcbc4072ae0 msgr2=0x7fcbc40b6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 shutdown_connections 2024-09-21T05:43:50.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.550+0000 7fcbcbf5f640 1 -- 172.21.15.22:0/1813612389 wait complete. 2024-09-21T05:43:50.563 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:43:50.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:50.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:43:50.592 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:43:50.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:43:50.593 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:43:50.649 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:43:50.649 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:43:50.649 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:43:50.668 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:43:50.668 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:43:50.668 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:43:50.668 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:43:50.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:43:50.705 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:43:50.705 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:43:50.705 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:43:50.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:43:50.727 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:43:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a9a430640 1 -- 172.21.15.22:0/3764754226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a7c003a00 msgr2=0x7f8a7c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a9a430640 1 --2- 172.21.15.22:0/3764754226 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a7c003a00 0x7f8a7c003e70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a940d11c0 tx=0x7f8a7c005830 comp rx=0 tx=0).stop 2024-09-21T05:43:50.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a989cd640 1 -- 172.21.15.22:0/3764754226 reap_dead start 2024-09-21T05:43:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a9a430640 1 -- 172.21.15.22:0/3764754226 shutdown_connections 2024-09-21T05:43:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a9a430640 1 -- 172.21.15.22:0/3764754226 >> 172.21.15.22:0/3764754226 conn(0x7f8a940b1cb0 msgr2=0x7f8a940b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a9a430640 1 -- 172.21.15.22:0/3764754226 shutdown_connections 2024-09-21T05:43:50.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.860+0000 7f8a9a430640 1 -- 172.21.15.22:0/3764754226 wait complete. 2024-09-21T05:43:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.862+0000 7f8a9a430640 1 Processor -- start 2024-09-21T05:43:50.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.862+0000 7f8a9a430640 1 -- start start 2024-09-21T05:43:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a9a430640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a7c003a00 0x7f8a94159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a9a430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 0x7f8a9406f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a9a430640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a9414cc30 0x7f8a9414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a9a430640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a94073940 con 0x7f8a9406f070 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a9a430640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a940737c0 con 0x7f8a7c003a00 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a9a430640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8a94073ac0 con 0x7f8a9414cc30 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 0x7f8a9406f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a937fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 0x7f8a9406f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59276/0 (socket says 172.21.15.22:59276) 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a937fe640 1 -- 172.21.15.22:0/1883343944 learned_addr learned my addr 172.21.15.22:0/1883343944 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a93fff640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a7c003a00 0x7f8a94159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.863+0000 7f8a989cd640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a9414cc30 0x7f8a9414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a937fe640 1 -- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a9414cc30 msgr2=0x7f8a9414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a937fe640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a9414cc30 0x7f8a9414d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a937fe640 1 -- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a7c003a00 msgr2=0x7f8a94159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a937fe640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a7c003a00 0x7f8a94159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a937fe640 1 -- 172.21.15.22:0/1883343944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9414d580 con 0x7f8a9406f070 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a989cd640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a9414cc30 0x7f8a9414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:50.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a937fe640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 0x7f8a9406f480 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8a88011a50 tx=0x7f8a88011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8801f070 con 0x7f8a9406f070 2024-09-21T05:43:50.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a88012450 con 0x7f8a9406f070 2024-09-21T05:43:50.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a941f68b0 con 0x7f8a9406f070 2024-09-21T05:43:50.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88012880 con 0x7f8a9406f070 2024-09-21T05:43:50.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.864+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a941f6e70 con 0x7f8a9406f070 2024-09-21T05:43:50.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.865+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a940cba00 con 0x7f8a9406f070 2024-09-21T05:43:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.866+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a88022070 con 0x7f8a9406f070 2024-09-21T05:43:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.866+0000 7f8a90ff9640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a80078b90 0x7f8a8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.866+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f8a8809b700 con 0x7f8a9406f070 2024-09-21T05:43:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.867+0000 7f8a93fff640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a80078b90 0x7f8a8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:50.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.867+0000 7f8a93fff640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a80078b90 0x7f8a8007b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a940d11c0 tx=0x7f8a8402d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:50.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:50.871+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a88063580 con 0x7f8a9406f070 2024-09-21T05:43:51.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.096+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a940703e0 con 0x7f8a9406f070 2024-09-21T05:43:51.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.097+0000 7f8a90ff9640 1 -- 172.21.15.22:0/1883343944 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5230 (secure 0 0 0) 0x7f8a88067fb0 con 0x7f8a9406f070 2024-09-21T05:43:51.099 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:51.099 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":53,"btime":"2024-09-21T05:43:21:876894+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:43:21.365281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:43:51.099 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 53 2024-09-21T05:43:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a80078b90 msgr2=0x7f8a8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a80078b90 0x7f8a8007b0a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8a940d11c0 tx=0x7f8a8402d000 comp rx=0 tx=0).stop 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 msgr2=0x7f8a9406f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 0x7f8a9406f480 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8a88011a50 tx=0x7f8a88011f10 comp rx=0 tx=0).stop 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 shutdown_connections 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a80078b90 0x7f8a8007b0a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a9414cc30 0x7f8a9414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a9406f070 0x7f8a9406f480 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 --2- 172.21.15.22:0/1883343944 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a7c003a00 0x7f8a94159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 >> 172.21.15.22:0/1883343944 conn(0x7f8a940b1cb0 msgr2=0x7f8a9415b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 shutdown_connections 2024-09-21T05:43:51.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.099+0000 7f8a9a430640 1 -- 172.21.15.22:0/1883343944 wait complete. 2024-09-21T05:43:51.113 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:43:51.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 -- 172.21.15.22:0/1564905216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0d1560 msgr2=0x7f21cc06f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 --2- 172.21.15.22:0/1564905216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0d1560 0x7f21cc06f6e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f21c4005fd0 tx=0x7f21c402a980 comp rx=0 tx=0).stop 2024-09-21T05:43:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 -- 172.21.15.22:0/1564905216 shutdown_connections 2024-09-21T05:43:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 --2- 172.21.15.22:0/1564905216 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0d1560 0x7f21cc06f6e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 -- 172.21.15.22:0/1564905216 >> 172.21.15.22:0/1564905216 conn(0x7f21cc09d840 msgr2=0x7f21cc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 -- 172.21.15.22:0/1564905216 shutdown_connections 2024-09-21T05:43:51.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.227+0000 7f21d344a640 1 -- 172.21.15.22:0/1564905216 wait complete. 2024-09-21T05:43:51.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.229+0000 7f21d344a640 1 Processor -- start 2024-09-21T05:43:51.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.229+0000 7f21d344a640 1 -- start start 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d344a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc06e0a0 0x7f21cc1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d344a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 0x7f21cc0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d344a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21cc157f50 0x7f21cc0cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d344a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0cff00 con 0x7f21cc0b97e0 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d344a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc0cfd80 con 0x7f21cc06e0a0 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d344a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0d0080 con 0x7f21cc157f50 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.230+0000 7f21d09e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 0x7f21cc0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 0x7f21cc0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59290/0 (socket says 172.21.15.22:59290) 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 -- 172.21.15.22:0/3545776419 learned_addr learned my addr 172.21.15.22:0/3545776419 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d11e6640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc06e0a0 0x7f21cc1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d19e7640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21cc157f50 0x7f21cc0cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 -- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21cc157f50 msgr2=0x7f21cc0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21cc157f50 0x7f21cc0cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 -- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc06e0a0 msgr2=0x7f21cc1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc06e0a0 0x7f21cc1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 -- 172.21.15.22:0/3545776419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc157ce0 con 0x7f21cc0b97e0 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d09e5640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 0x7f21cc0ccc80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00b3e0 tx=0x7f21bc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc015020 con 0x7f21cc0b97e0 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21bc011920 con 0x7f21cc0b97e0 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc157a00 con 0x7f21cc0b97e0 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc011d70 con 0x7f21cc0b97e0 2024-09-21T05:43:51.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.231+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc1ea520 con 0x7f21cc0b97e0 2024-09-21T05:43:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.232+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc0d0c50 con 0x7f21cc0b97e0 2024-09-21T05:43:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.233+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21bc018020 con 0x7f21cc0b97e0 2024-09-21T05:43:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.234+0000 7f21c1ffb640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b0078bc0 0x7f21b007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.234+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f21bc09ac50 con 0x7f21cc0b97e0 2024-09-21T05:43:51.236 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.234+0000 7f21d11e6640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b0078bc0 0x7f21b007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.237 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.235+0000 7f21d11e6640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b0078bc0 0x7f21b007b0d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f21c8000fd0 tx=0x7f21c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:51.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.238+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21bc01f020 con 0x7f21cc0b97e0 2024-09-21T05:43:51.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.462+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21cc1e7cc0 con 0x7f21cc0b97e0 2024-09-21T05:43:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.463+0000 7f21c1ffb640 1 -- 172.21.15.22:0/3545776419 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5230 (secure 0 0 0) 0x7f21bc062910 con 0x7f21cc0b97e0 2024-09-21T05:43:51.465 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:51.465 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":53,"btime":"2024-09-21T05:43:21:876894+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:43:21.365281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:43:51.465 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 53 2024-09-21T05:43:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b0078bc0 msgr2=0x7f21b007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b0078bc0 0x7f21b007b0d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f21c8000fd0 tx=0x7f21c8009040 comp rx=0 tx=0).stop 2024-09-21T05:43:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 msgr2=0x7f21cc0ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 0x7f21cc0ccc80 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00b3e0 tx=0x7f21bc00b8a0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 shutdown_connections 2024-09-21T05:43:51.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f21b0078bc0 0x7f21b007b0d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f21cc157f50 0x7f21cc0cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f21cc0b97e0 0x7f21cc0ccc80 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 --2- 172.21.15.22:0/3545776419 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f21cc06e0a0 0x7f21cc1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 >> 172.21.15.22:0/3545776419 conn(0x7f21cc09d840 msgr2=0x7f21cc158880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 shutdown_connections 2024-09-21T05:43:51.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.465+0000 7f21d344a640 1 -- 172.21.15.22:0/3545776419 wait complete. 2024-09-21T05:43:51.478 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:43:51.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb695e27640 1 -- 172.21.15.22:0/256497716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb678004d40 msgr2=0x7fb678005170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb695e27640 1 --2- 172.21.15.22:0/256497716 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb678004d40 0x7fb678005170 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fb688005fd0 tx=0x7fb678005830 comp rx=0 tx=0).stop 2024-09-21T05:43:51.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb68ffff640 1 -- 172.21.15.22:0/256497716 reap_dead start 2024-09-21T05:43:51.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb695e27640 1 -- 172.21.15.22:0/256497716 shutdown_connections 2024-09-21T05:43:51.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb695e27640 1 -- 172.21.15.22:0/256497716 >> 172.21.15.22:0/256497716 conn(0x7fb69006d4f0 msgr2=0x7fb690071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:51.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb695e27640 1 -- 172.21.15.22:0/256497716 shutdown_connections 2024-09-21T05:43:51.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.594+0000 7fb695e27640 1 -- 172.21.15.22:0/256497716 wait complete. 2024-09-21T05:43:51.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.596+0000 7fb695e27640 1 Processor -- start 2024-09-21T05:43:51.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.596+0000 7fb695e27640 1 -- start start 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb695e27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb678004d40 0x7fb6900cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb695e27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6901f4200 0x7fb6900c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb695e27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 0x7fb69015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb695e27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb6900d1090 con 0x7fb690159d10 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb695e27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb6900cd0d0 con 0x7fb678004d40 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb695e27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb6900d1210 con 0x7fb6901f4200 2024-09-21T05:43:51.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 0x7fb69015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 0x7fb69015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59302/0 (socket says 172.21.15.22:59302) 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 -- 172.21.15.22:0/1684522482 learned_addr learned my addr 172.21.15.22:0/1684522482 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68f7fe640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb678004d40 0x7fb6900cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68effd640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6901f4200 0x7fb6900c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 -- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6901f4200 msgr2=0x7fb6900c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6901f4200 0x7fb6900c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 -- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb678004d40 msgr2=0x7fb6900cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb678004d40 0x7fb6900cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.597+0000 7fb68ffff640 1 -- 172.21.15.22:0/1684522482 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69015a660 con 0x7fb690159d10 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.598+0000 7fb68ffff640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 0x7fb69015a120 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb688036040 tx=0x7fb688034bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:51.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.598+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688043070 con 0x7fb690159d10 2024-09-21T05:43:51.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.598+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb688034ca0 con 0x7fb690159d10 2024-09-21T05:43:51.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.598+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb688038040 con 0x7fb690159d10 2024-09-21T05:43:51.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.598+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6901f4720 con 0x7fb690159d10 2024-09-21T05:43:51.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.598+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb690149300 con 0x7fb690159d10 2024-09-21T05:43:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.599+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6900d1770 con 0x7fb690159d10 2024-09-21T05:43:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.600+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb688046070 con 0x7fb690159d10 2024-09-21T05:43:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.601+0000 7fb677fff640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb67c078b90 0x7fb67c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.601+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7fb6880baf80 con 0x7fb690159d10 2024-09-21T05:43:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.601+0000 7fb68f7fe640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb67c078b90 0x7fb67c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.601+0000 7fb68f7fe640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb67c078b90 0x7fb67c07b0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb6800036b0 tx=0x7fb680006110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:51.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.605+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb69015a660 con 0x7fb690159d10 2024-09-21T05:43:51.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.793+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb690155c80 con 0x7fb690159d10 2024-09-21T05:43:51.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.795+0000 7fb677fff640 1 -- 172.21.15.22:0/1684522482 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16945 (secure 0 0 0) 0x7fb688082ce0 con 0x7fb690159d10 2024-09-21T05:43:51.797 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:51.797 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":71,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:43:03.421963+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:42:26.665513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T05:42:27.958639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.1c","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.134:6837/1820431436":"2024-09-22T05:42:38.349537+0000","172.21.15.134:6835/1820431436":"2024-09-22T05:42:38.349537+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb67c078b90 msgr2=0x7fb67c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb67c078b90 0x7fb67c07b0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb6800036b0 tx=0x7fb680006110 comp rx=0 tx=0).stop 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 msgr2=0x7fb69015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 0x7fb69015a120 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb688036040 tx=0x7fb688034bb0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 shutdown_connections 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb690159d10 0x7fb69015a120 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb67c078b90 0x7fb67c07b0a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb6901f4200 0x7fb6900c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 --2- 172.21.15.22:0/1684522482 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb678004d40 0x7fb6900cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 >> 172.21.15.22:0/1684522482 conn(0x7fb69006d4f0 msgr2=0x7fb6900b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 shutdown_connections 2024-09-21T05:43:51.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.797+0000 7fb695e27640 1 -- 172.21.15.22:0/1684522482 wait complete. 2024-09-21T05:43:51.811 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:43:51.811 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:43:51.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:43:51.856 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 -- 172.21.15.22:0/634173724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 msgr2=0x7f3d2c0b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 --2- 172.21.15.22:0/634173724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c0b97e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0098e0 tx=0x7f3d1c02a400 comp rx=0 tx=0).stop 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 -- 172.21.15.22:0/634173724 shutdown_connections 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 --2- 172.21.15.22:0/634173724 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d2c0d0e50 0x7f3d2c0d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 --2- 172.21.15.22:0/634173724 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c0b97e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 -- 172.21.15.22:0/634173724 >> 172.21.15.22:0/634173724 conn(0x7f3d2c09d840 msgr2=0x7f3d2c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 -- 172.21.15.22:0/634173724 shutdown_connections 2024-09-21T05:43:51.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.925+0000 7f3d3224d640 1 -- 172.21.15.22:0/634173724 wait complete. 2024-09-21T05:43:51.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.927+0000 7f3d3224d640 1 Processor -- start 2024-09-21T05:43:51.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.927+0000 7f3d3224d640 1 -- start start 2024-09-21T05:43:51.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d3224d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d3224d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d2c06f6e0 0x7f3d2c14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d3224d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2c0d0e50 0x7f3d2c0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d3224d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c0cfea0 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d3224d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c0cfd20 con 0x7f3d2c0d0e50 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d3224d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2c0d0020 con 0x7f3d2c06f6e0 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c14ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59320/0 (socket says 172.21.15.22:59320) 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 -- 172.21.15.22:0/4169126201 learned_addr learned my addr 172.21.15.22:0/4169126201 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2bfff640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2c0d0e50 0x7f3d2c0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2affd640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d2c06f6e0 0x7f3d2c14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 -- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d2c06f6e0 msgr2=0x7f3d2c14a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d2c06f6e0 0x7f3d2c14a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 -- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2c0d0e50 msgr2=0x7f3d2c0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2c0d0e50 0x7f3d2c0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:51.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 -- 172.21.15.22:0/4169126201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c14b270 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.929+0000 7f3d2b7fe640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c14ad30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009a00 tx=0x7f3d1c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.930+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c03f070 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.930+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d1c004360 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.930+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c0c8400 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.930+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c031bb0 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.930+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c0c89c0 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.931+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2c0d0490 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.932+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d1c042070 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.932+0000 7f3d23fff640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:51.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.932+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f3d1c0bbd00 con 0x7f3d2c06e0a0 2024-09-21T05:43:51.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.933+0000 7f3d2affd640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:51.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.933+0000 7f3d2affd640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c151440 tx=0x7f3d14009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:51.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:51.938+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d1c031770 con 0x7f3d2c06e0a0 2024-09-21T05:43:52.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.163+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d2c0690b0 con 0x7f3d2c06e0a0 2024-09-21T05:43:52.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.163+0000 7f3d23fff640 1 -- 172.21.15.22:0/4169126201 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5230 (secure 0 0 0) 0x7f3d1c083af0 con 0x7f3d2c06e0a0 2024-09-21T05:43:52.165 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:52.166 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":53,"btime":"2024-09-21T05:43:21:876894+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:43:21.365281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:43:52.166 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 53 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d0c078b90 msgr2=0x7f3d0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c151440 tx=0x7f3d14009060 comp rx=0 tx=0).stop 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 msgr2=0x7f3d2c14ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c14ad30 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009a00 tx=0x7f3d1c005270 comp rx=0 tx=0).stop 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 shutdown_connections 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3d2c0d0e50 0x7f3d2c0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d0c078b90 0x7f3d0c07b0a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3d2c06f6e0 0x7f3d2c14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 --2- 172.21.15.22:0/4169126201 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3d2c06e0a0 0x7f3d2c14ad30 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 >> 172.21.15.22:0/4169126201 conn(0x7f3d2c09d840 msgr2=0x7f3d2c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:52.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 shutdown_connections 2024-09-21T05:43:52.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.166+0000 7f3d3224d640 1 -- 172.21.15.22:0/4169126201 wait complete. 2024-09-21T05:43:52.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 -- 172.21.15.22:0/1854960465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580b7fe0 msgr2=0x7f0d580d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 --2- 172.21.15.22:0/1854960465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580b7fe0 0x7f0d580d0190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0d48005fd0 tx=0x7f0d4802aa10 comp rx=0 tx=0).stop 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 -- 172.21.15.22:0/1854960465 shutdown_connections 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 --2- 172.21.15.22:0/1854960465 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580b7fe0 0x7f0d580d0190 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 --2- 172.21.15.22:0/1854960465 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d580cf440 0x7f0d5815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 -- 172.21.15.22:0/1854960465 >> 172.21.15.22:0/1854960465 conn(0x7f0d580710b0 msgr2=0x7f0d580714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 -- 172.21.15.22:0/1854960465 shutdown_connections 2024-09-21T05:43:52.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.333+0000 7f0d60332640 1 -- 172.21.15.22:0/1854960465 wait complete. 2024-09-21T05:43:52.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.335+0000 7f0d60332640 1 Processor -- start 2024-09-21T05:43:52.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.335+0000 7f0d60332640 1 -- start start 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.336+0000 7f0d60332640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d580b7fe0 0x7f0d5815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.336+0000 7f0d60332640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 0x7f0d5814a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.336+0000 7f0d60332640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581593f0 0x7f0d5814a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.336+0000 7f0d60332640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580d0b70 con 0x7f0d580cd360 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.336+0000 7f0d60332640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580d09f0 con 0x7f0d580b7fe0 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.336+0000 7f0d60332640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580d0cf0 con 0x7f0d581593f0 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 0x7f0d5814a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 0x7f0d5814a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59336/0 (socket says 172.21.15.22:59336) 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 -- 172.21.15.22:0/556886397 learned_addr learned my addr 172.21.15.22:0/556886397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5e0ce640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d580b7fe0 0x7f0d5815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5e8cf640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581593f0 0x7f0d5814a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 -- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581593f0 msgr2=0x7f0d5814a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581593f0 0x7f0d5814a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 -- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d580b7fe0 msgr2=0x7f0d5815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d580b7fe0 0x7f0d5815f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 -- 172.21.15.22:0/556886397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d580cba80 con 0x7f0d580cd360 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5e0ce640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d580b7fe0 0x7f0d5815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d5d8cd640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 0x7f0d5814a3a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c9a0 tx=0x7f0d4c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c007bb0 con 0x7f0d580cd360 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d4c007e40 con 0x7f0d580cd360 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5814af70 con 0x7f0d580cd360 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c005280 con 0x7f0d580cd360 2024-09-21T05:43:52.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.337+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5814b250 con 0x7f0d580cd360 2024-09-21T05:43:52.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.339+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d58148030 con 0x7f0d580cd360 2024-09-21T05:43:52.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.339+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d4c005420 con 0x7f0d580cd360 2024-09-21T05:43:52.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.340+0000 7f0d56ffd640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d44078bc0 0x7f0d4407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.340+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f0d4c09b200 con 0x7f0d580cd360 2024-09-21T05:43:52.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.340+0000 7f0d5e0ce640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d44078bc0 0x7f0d4407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.340+0000 7f0d5e0ce640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d44078bc0 0x7f0d4407b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d50000f60 tx=0x7f0d50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:52.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.344+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d58148030 con 0x7f0d580cd360 2024-09-21T05:43:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.570+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d5815fc50 con 0x7f0d580cd360 2024-09-21T05:43:52.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.570+0000 7f0d56ffd640 1 -- 172.21.15.22:0/556886397 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5230 (secure 0 0 0) 0x7f0d4c014030 con 0x7f0d580cd360 2024-09-21T05:43:52.573 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:52.573 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":53,"btime":"2024-09-21T05:43:21:876894+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:42:29.001833+0000","modified":"2024-09-21T05:43:21.365281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6153},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6153":{"gid":6153,"name":"d","rank":0,"incarnation":48,"state":"up:active","state_seq":17,"addr":"172.21.15.134:6836/2827334455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2827334455},{"type":"v1","addr":"172.21.15.134:6836","nonce":2827334455}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6153,"qdb_cluster":[6153]},"id":5}]} 2024-09-21T05:43:52.573 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 53 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.572+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d44078bc0 msgr2=0x7f0d4407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.572+0000 7f0d60332640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d44078bc0 0x7f0d4407b0d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0d50000f60 tx=0x7f0d50009040 comp rx=0 tx=0).stop 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 msgr2=0x7f0d5814a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 0x7f0d5814a3a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00c9a0 tx=0x7f0d4c00ce60 comp rx=0 tx=0).stop 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 shutdown_connections 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d44078bc0 0x7f0d4407b0d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d581593f0 0x7f0d5814a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d580cd360 0x7f0d5814a3a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 --2- 172.21.15.22:0/556886397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d580b7fe0 0x7f0d5815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 >> 172.21.15.22:0/556886397 conn(0x7f0d580710b0 msgr2=0x7f0d5815b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:52.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 shutdown_connections 2024-09-21T05:43:52.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.573+0000 7f0d60332640 1 -- 172.21.15.22:0/556886397 wait complete. 2024-09-21T05:43:52.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 -- 172.21.15.22:0/1229904342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740d1d40 msgr2=0x7f6c740b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/1229904342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740d1d40 0x7f6c740b5210 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009830 tx=0x7f6c7002a510 comp rx=0 tx=0).stop 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 -- 172.21.15.22:0/1229904342 shutdown_connections 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/1229904342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7415c6f0 0x7f6c740ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/1229904342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740d1d40 0x7f6c740b5210 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 -- 172.21.15.22:0/1229904342 >> 172.21.15.22:0/1229904342 conn(0x7f6c740b3f10 msgr2=0x7f6c740b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 -- 172.21.15.22:0/1229904342 shutdown_connections 2024-09-21T05:43:52.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.700+0000 7f6c7c54a640 1 -- 172.21.15.22:0/1229904342 wait complete. 2024-09-21T05:43:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.702+0000 7f6c7c54a640 1 Processor -- start 2024-09-21T05:43:52.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.702+0000 7f6c7c54a640 1 -- start start 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.703+0000 7f6c7c54a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740ccba0 0x7f6c74153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.703+0000 7f6c7c54a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 0x7f6c741ef450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c7c54a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7415c6f0 0x7f6c740b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c7c54a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740d12d0 con 0x7f6c740d1d40 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c7c54a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740d1150 con 0x7f6c740ccba0 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c7c54a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c740d1450 con 0x7f6c7415c6f0 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 0x7f6c741ef450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 0x7f6c741ef450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59342/0 (socket says 172.21.15.22:59342) 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 -- 172.21.15.22:0/155367027 learned_addr learned my addr 172.21.15.22:0/155367027 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c7a2e6640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740ccba0 0x7f6c74153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c7aae7640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7415c6f0 0x7f6c740b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 -- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7415c6f0 msgr2=0x7f6c740b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7415c6f0 0x7f6c740b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 -- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740ccba0 msgr2=0x7f6c74153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740ccba0 0x7f6c74153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 -- 172.21.15.22:0/155367027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c741f6630 con 0x7f6c740d1d40 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c79ae5640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 0x7f6c741ef450 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e7c0 tx=0x7f6c6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.704+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400ccc0 con 0x7f6c740d1d40 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.705+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c741f58b0 con 0x7f6c740d1d40 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.705+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c6400ee70 con 0x7f6c740d1d40 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.705+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64010690 con 0x7f6c740d1d40 2024-09-21T05:43:52.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.705+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c741f5b50 con 0x7f6c740d1d40 2024-09-21T05:43:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.706+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c740d1970 con 0x7f6c740d1d40 2024-09-21T05:43:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.706+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c64010830 con 0x7f6c740d1d40 2024-09-21T05:43:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.707+0000 7f6c6affd640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c58078b90 0x7f6c5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.707+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f6c6409b860 con 0x7f6c740d1d40 2024-09-21T05:43:52.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.707+0000 7f6c7a2e6640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c58078b90 0x7f6c5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:52.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.708+0000 7f6c7a2e6640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c58078b90 0x7f6c5807b0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c70034000 tx=0x7f6c70004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:52.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.712+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c64063540 con 0x7f6c740d1d40 2024-09-21T05:43:52.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.897+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c74154330 con 0x7f6c740d1d40 2024-09-21T05:43:52.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.898+0000 7f6c6affd640 1 -- 172.21.15.22:0/155367027 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+16945 (secure 0 0 0) 0x7f6c64067f70 con 0x7f6c740d1d40 2024-09-21T05:43:52.900 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:52.901 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":71,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:43:03.421963+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:42:26.665513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T05:42:27.958639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.14","mappings":[{"from":6,"to":0}]},{"pgid":"11.1c","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.134:6837/1820431436":"2024-09-22T05:42:38.349537+0000","172.21.15.134:6835/1820431436":"2024-09-22T05:42:38.349537+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:43:52.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c58078b90 msgr2=0x7f6c5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c58078b90 0x7f6c5807b0a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6c70034000 tx=0x7f6c70004730 comp rx=0 tx=0).stop 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 msgr2=0x7f6c741ef450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 0x7f6c741ef450 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e7c0 tx=0x7f6c6400ec80 comp rx=0 tx=0).stop 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 shutdown_connections 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c58078b90 0x7f6c5807b0a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c7415c6f0 0x7f6c740b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c740d1d40 0x7f6c741ef450 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 --2- 172.21.15.22:0/155367027 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c740ccba0 0x7f6c74153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 >> 172.21.15.22:0/155367027 conn(0x7f6c740b3f10 msgr2=0x7f6c740b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:52.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 shutdown_connections 2024-09-21T05:43:52.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:52.900+0000 7f6c7c54a640 1 -- 172.21.15.22:0/155367027 wait complete. 2024-09-21T05:43:52.914 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:43:53.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/3199369544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0af780 msgr2=0x7f7eac0b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/3199369544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0af780 0x7f7eac0b39c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0b4f90 tx=0x7f7e9c02a560 comp rx=0 tx=0).stop 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/3199369544 shutdown_connections 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/3199369544 >> v2:172.21.15.134:3300/0 conn(0x7f7eac0cd3e0 0x7f7eac070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/3199369544 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0af780 0x7f7eac0b39c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/3199369544 >> 172.21.15.22:0/3199369544 conn(0x7f7eac06d7d0 msgr2=0x7f7eac0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/3199369544 shutdown_connections 2024-09-21T05:43:53.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.029+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/3199369544 wait complete. 2024-09-21T05:43:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.031+0000 7f7eb1ff1640 1 Processor -- start 2024-09-21T05:43:53.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.031+0000 7f7eb1ff1640 1 -- start start 2024-09-21T05:43:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eb1ff1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7eac0706c0 0x7f7eac147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eb1ff1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eac0af780 0x7f7eac1e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eb1ff1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 0x7f7eac0b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eb1ff1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac073b00 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eb1ff1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7eac073980 con 0x7f7eac0af780 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eb1ff1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7eac073c80 con 0x7f7eac0706c0 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 0x7f7eac0b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eabfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 0x7f7eac0b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59362/0 (socket says 172.21.15.22:59362) 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.032+0000 7f7eabfff640 1 -- 172.21.15.22:0/2346502237 learned_addr learned my addr 172.21.15.22:0/2346502237 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eaaffd640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eac0af780 0x7f7eac1e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eab7fe640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7eac0706c0 0x7f7eac147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eabfff640 1 -- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7eac0706c0 msgr2=0x7f7eac147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eabfff640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7eac0706c0 0x7f7eac147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eabfff640 1 -- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eac0af780 msgr2=0x7f7eac1e6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eabfff640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eac0af780 0x7f7eac1e6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eabfff640 1 -- 172.21.15.22:0/2346502237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eac1e7550 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eabfff640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 0x7f7eac0b0cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400b9e0 tx=0x7f7ea400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4003b70 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea4003e00 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eac1e7830 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400cb80 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.033+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eac15a9d0 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.034+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eac0b39c0 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.035+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ea4004000 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.036+0000 7f7e93fff640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7e98078b90 0x7f7e9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.036+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5853+0+0 (secure 0 0 0) 0x7f7ea409ab30 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.036+0000 7f7eab7fe640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7e98078b90 0x7f7e9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.036+0000 7f7eab7fe640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7e98078b90 0x7f7e9807b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c033000 tx=0x7f7e9c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:53.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.040+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ea4062920 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.264+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7eac15dd10 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.266 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:43:53.265+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:43:53.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.266+0000 7f7e93fff640 1 -- 172.21.15.22:0/2346502237 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7f7ea4067350 con 0x7f7eac0cd3e0 2024-09-21T05:43:53.268 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7e98078b90 msgr2=0x7f7e9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7e98078b90 0x7f7e9807b0a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c033000 tx=0x7f7e9c004730 comp rx=0 tx=0).stop 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 msgr2=0x7f7eac0b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 0x7f7eac0b0cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400b9e0 tx=0x7f7ea400bea0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 shutdown_connections 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7eac0cd3e0 0x7f7eac0b0cb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7eac0af780 0x7f7eac1e6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7e98078b90 0x7f7e9807b0a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 --2- 172.21.15.22:0/2346502237 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7eac0706c0 0x7f7eac147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 >> 172.21.15.22:0/2346502237 conn(0x7f7eac06d7d0 msgr2=0x7f7eac0afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:53.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 shutdown_connections 2024-09-21T05:43:53.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.269+0000 7f7eb1ff1640 1 -- 172.21.15.22:0/2346502237 wait complete. 2024-09-21T05:43:53.276 INFO:tasks.ceph.mds.d.smithi134.stderr: -3347> 2024-09-21T05:42:46.584+0000 7fe06d004640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:43:53.282 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:43:53.328 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:43:53.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.399+0000 7fb7c196e640 1 -- 172.21.15.22:0/2143803065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc0cff70 msgr2=0x7fb7bc15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.399+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2143803065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc15c830 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009830 tx=0x7fb7ac02a530 comp rx=0 tx=0).stop 2024-09-21T05:43:53.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.399+0000 7fb7c196e640 1 -- 172.21.15.22:0/2143803065 shutdown_connections 2024-09-21T05:43:53.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.399+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2143803065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc15c830 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.399+0000 7fb7c196e640 1 -- 172.21.15.22:0/2143803065 >> 172.21.15.22:0/2143803065 conn(0x7fb7bc0cee20 msgr2=0x7fb7bc0cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:53.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.400+0000 7fb7c196e640 1 -- 172.21.15.22:0/2143803065 shutdown_connections 2024-09-21T05:43:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.400+0000 7fb7c196e640 1 -- 172.21.15.22:0/2143803065 wait complete. 2024-09-21T05:43:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.401+0000 7fb7c196e640 1 Processor -- start 2024-09-21T05:43:53.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.401+0000 7fb7c196e640 1 -- start start 2024-09-21T05:43:53.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7c196e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7c196e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc160c90 0x7fb7bc0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7c196e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7bc0cc240 0x7fb7bc0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7c196e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc0d1900 con 0x7fb7bc0cff70 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7c196e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc0d1780 con 0x7fb7bc160c90 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7c196e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb7bc0d1a80 con 0x7fb7bc0cc240 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59372/0 (socket says 172.21.15.22:59372) 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 -- 172.21.15.22:0/2878414542 learned_addr learned my addr 172.21.15.22:0/2878414542 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7ba7fc640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc160c90 0x7fb7bc0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7bb7fe640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7bc0cc240 0x7fb7bc0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 -- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7bc0cc240 msgr2=0x7fb7bc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7bc0cc240 0x7fb7bc0cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 -- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc160c90 msgr2=0x7fb7bc0cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc160c90 0x7fb7bc0cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.403+0000 7fb7baffd640 1 -- 172.21.15.22:0/2878414542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc0ca700 con 0x7fb7bc0cff70 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.404+0000 7fb7baffd640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc148810 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02ab20 tx=0x7fb7ac033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.404+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac031b50 con 0x7fb7bc0cff70 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.404+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7ac031de0 con 0x7fb7bc0cff70 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.404+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc0ca9e0 con 0x7fb7bc0cff70 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.404+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac005280 con 0x7fb7bc0cff70 2024-09-21T05:43:53.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.404+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc148d50 con 0x7fb7bc0cff70 2024-09-21T05:43:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.405+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7bc15c830 con 0x7fb7bc0cff70 2024-09-21T05:43:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.406+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7ac005420 con 0x7fb7bc0cff70 2024-09-21T05:43:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.406+0000 7fb7a3fff640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb7a8078bc0 0x7fb7a807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.406+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5939+0+0 (secure 0 0 0) 0x7fb7ac0bbe00 con 0x7fb7bc0cff70 2024-09-21T05:43:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.406+0000 7fb7ba7fc640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb7a8078bc0 0x7fb7a807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:53.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.407+0000 7fb7ba7fc640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb7a8078bc0 0x7fb7a807b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc152ed0 tx=0x7fb7b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:53.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.411+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7ac03f030 con 0x7fb7bc0cff70 2024-09-21T05:43:53.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:53.635+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb7bc0b6f50 con 0x7fb7bc0cff70 2024-09-21T05:43:54.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.270+0000 7fb7a3fff640 1 -- 172.21.15.22:0/2878414542 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7fb7ac031770 con 0x7fb7bc0cff70 2024-09-21T05:43:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb7a8078bc0 msgr2=0x7fb7a807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb7a8078bc0 0x7fb7a807b0d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc152ed0 tx=0x7fb7b0009040 comp rx=0 tx=0).stop 2024-09-21T05:43:54.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 msgr2=0x7fb7bc148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc148810 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac02ab20 tx=0x7fb7ac033ba0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 shutdown_connections 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb7a8078bc0 0x7fb7a807b0d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb7bc0cc240 0x7fb7bc0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb7bc160c90 0x7fb7bc0cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 --2- 172.21.15.22:0/2878414542 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb7bc0cff70 0x7fb7bc148810 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 >> 172.21.15.22:0/2878414542 conn(0x7fb7bc0cee20 msgr2=0x7fb7bc0d23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 shutdown_connections 2024-09-21T05:43:54.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.273+0000 7fb7c196e640 1 -- 172.21.15.22:0/2878414542 wait complete. 2024-09-21T05:43:54.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:43:54.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 -- 172.21.15.22:0/3630295862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de406fa70 msgr2=0x7f3de406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:54.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/3630295862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de406fa70 0x7f3de406e160 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009830 tx=0x7f3de002a480 comp rx=0 tx=0).stop 2024-09-21T05:43:54.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 -- 172.21.15.22:0/3630295862 shutdown_connections 2024-09-21T05:43:54.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/3630295862 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de406fa70 0x7f3de406e160 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 -- 172.21.15.22:0/3630295862 >> 172.21.15.22:0/3630295862 conn(0x7f3de408dc50 msgr2=0x7f3de409d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:54.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 -- 172.21.15.22:0/3630295862 shutdown_connections 2024-09-21T05:43:54.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.411+0000 7f3dec4d5640 1 -- 172.21.15.22:0/3630295862 wait complete. 2024-09-21T05:43:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.413+0000 7f3dec4d5640 1 Processor -- start 2024-09-21T05:43:54.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.413+0000 7f3dec4d5640 1 -- start start 2024-09-21T05:43:54.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dec4d5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3de406fa70 0x7f3de415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:54.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dec4d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 0x7f3de41ea0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:54.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dec4d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de40d0ef0 0x7f3de41ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dec4d5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4162d10 con 0x7f3de40cf540 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dec4d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3de4162b90 con 0x7f3de40d0ef0 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dec4d5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3de4162e90 con 0x7f3de406fa70 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3de9a70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 0x7f3de41ea0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3de9a70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 0x7f3de41ea0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59800/0 (socket says 172.21.15.22:59800) 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3de9a70640 1 -- 172.21.15.22:0/4109752552 learned_addr learned my addr 172.21.15.22:0/4109752552 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3dea271640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3de406fa70 0x7f3de415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.414+0000 7f3deaa72640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de40d0ef0 0x7f3de41ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3de9a70640 1 -- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3de406fa70 msgr2=0x7f3de415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3de9a70640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3de406fa70 0x7f3de415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3de9a70640 1 -- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de40d0ef0 msgr2=0x7f3de41ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3de9a70640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de40d0ef0 0x7f3de41ea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:54.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3de9a70640 1 -- 172.21.15.22:0/4109752552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de40c99c0 con 0x7f3de40cf540 2024-09-21T05:43:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3de9a70640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 0x7f3de41ea0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c3f0 tx=0x7f3dd400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4017070 con 0x7f3de40cf540 2024-09-21T05:43:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dd40035f0 con 0x7f3de40cf540 2024-09-21T05:43:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de41e8fe0 con 0x7f3de40cf540 2024-09-21T05:43:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4003a40 con 0x7f3de40cf540 2024-09-21T05:43:54.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.415+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de40c8140 con 0x7f3de40cf540 2024-09-21T05:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.416+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de406e160 con 0x7f3de40cf540 2024-09-21T05:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.417+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dd401a070 con 0x7f3de40cf540 2024-09-21T05:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.417+0000 7f3ddaffd640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc8078bc0 0x7f3dc807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.417+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5939+0+0 (secure 0 0 0) 0x7f3dd4099fe0 con 0x7f3de40cf540 2024-09-21T05:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.418+0000 7f3dea271640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc8078bc0 0x7f3dc807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:54.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.418+0000 7f3dea271640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc8078bc0 0x7f3dc807b0d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3de0034000 tx=0x7f3de0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:54.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.422+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dd4061dd0 con 0x7f3de40cf540 2024-09-21T05:43:54.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:54.611+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3de4160490 con 0x7f3de40cf540 2024-09-21T05:43:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.272+0000 7f3ddaffd640 1 -- 172.21.15.22:0/4109752552 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) ==== 181+0+0 (secure 0 0 0) 0x7f3dd4066800 con 0x7f3de40cf540 2024-09-21T05:43:55.274 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.275+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc8078bc0 msgr2=0x7f3dc807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.275+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc8078bc0 0x7f3dc807b0d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3de0034000 tx=0x7f3de0004730 comp rx=0 tx=0).stop 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.275+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 msgr2=0x7f3de41ea0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.275+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 0x7f3de41ea0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c3f0 tx=0x7f3dd400c8b0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 shutdown_connections 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3de40d0ef0 0x7f3de41ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3de40cf540 0x7f3de41ea0a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc8078bc0 0x7f3dc807b0d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 --2- 172.21.15.22:0/4109752552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3de406fa70 0x7f3de415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 >> 172.21.15.22:0/4109752552 conn(0x7f3de408dc50 msgr2=0x7f3de415e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 shutdown_connections 2024-09-21T05:43:55.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.276+0000 7f3dec4d5640 1 -- 172.21.15.22:0/4109752552 wait complete. 2024-09-21T05:43:55.293 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38ceca9640 1 -- 172.21.15.22:0/2359706553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38b0004e60 msgr2=0x7f38b00052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38ceca9640 1 --2- 172.21.15.22:0/2359706553 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38b0004e60 0x7f38b00052b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f38c000bb20 tx=0x7f38b0005830 comp rx=0 tx=0).stop 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38cd246640 1 -- 172.21.15.22:0/2359706553 reap_dead start 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38ceca9640 1 -- 172.21.15.22:0/2359706553 shutdown_connections 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38ceca9640 1 -- 172.21.15.22:0/2359706553 >> 172.21.15.22:0/2359706553 conn(0x7f38c80ced50 msgr2=0x7f38c80cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38ceca9640 1 -- 172.21.15.22:0/2359706553 shutdown_connections 2024-09-21T05:43:55.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.418+0000 7f38ceca9640 1 -- 172.21.15.22:0/2359706553 wait complete. 2024-09-21T05:43:55.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.420+0000 7f38ceca9640 1 Processor -- start 2024-09-21T05:43:55.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.420+0000 7f38ceca9640 1 -- start start 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.421+0000 7f38ceca9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38b0004e60 0x7f38c81f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.421+0000 7f38ceca9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c80cbf70 0x7f38c8158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.421+0000 7f38ceca9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 0x7f38c80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.421+0000 7f38ceca9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f38c80cfe10 con 0x7f38c81f3b50 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.421+0000 7f38ceca9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f38c80717c0 con 0x7f38c80cbf70 2024-09-21T05:43:55.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.421+0000 7f38ceca9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f38c80cff90 con 0x7f38b0004e60 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 0x7f38c80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 0x7f38c80a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59812/0 (socket says 172.21.15.22:59812) 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 -- 172.21.15.22:0/4195405036 learned_addr learned my addr 172.21.15.22:0/4195405036 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38bffff640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c80cbf70 0x7f38c8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 -- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38b0004e60 msgr2=0x7f38c81f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cca45640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38b0004e60 0x7f38c81f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38b0004e60 0x7f38c81f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 -- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c80cbf70 msgr2=0x7f38c8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c80cbf70 0x7f38c8158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 -- 172.21.15.22:0/4195405036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c8158a50 con 0x7f38c81f3b50 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38bffff640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c80cbf70 0x7f38c8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:55.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cca45640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38b0004e60 0x7f38c81f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38cd246640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 0x7f38c80a94a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f38c000bc40 tx=0x7f38c0031f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0035040 con 0x7f38c81f3b50 2024-09-21T05:43:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38c0004140 con 0x7f38c81f3b50 2024-09-21T05:43:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c003a630 con 0x7f38c81f3b50 2024-09-21T05:43:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c80cb230 con 0x7f38c81f3b50 2024-09-21T05:43:55.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.422+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c81f3350 con 0x7f38c81f3b50 2024-09-21T05:43:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.423+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c80d04f0 con 0x7f38c81f3b50 2024-09-21T05:43:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.424+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38c0038020 con 0x7f38c81f3b50 2024-09-21T05:43:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.425+0000 7f38bd7fa640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f38b4078bc0 0x7f38b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.425+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5501+0+0 (secure 0 0 0) 0x7f38c00bb840 con 0x7f38c81f3b50 2024-09-21T05:43:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.425+0000 7f38cca45640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f38b4078bc0 0x7f38b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:55.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.426+0000 7f38cca45640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f38b4078bc0 0x7f38b407b0d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f38c4000f80 tx=0x7f38c4006ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:55.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.430+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38c0083d50 con 0x7f38c81f3b50 2024-09-21T05:43:55.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:55.618+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f38c80cc650 con 0x7f38c81f3b50 2024-09-21T05:43:56.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.277+0000 7f38bd7fa640 1 -- 172.21.15.22:0/4195405036 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) ==== 169+0+0 (secure 0 0 0) 0x7f38c0083650 con 0x7f38c81f3b50 2024-09-21T05:43:56.280 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:43:56.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f38b4078bc0 msgr2=0x7f38b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f38b4078bc0 0x7f38b407b0d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f38c4000f80 tx=0x7f38c4006ef0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 msgr2=0x7f38c80a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 0x7f38c80a94a0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f38c000bc40 tx=0x7f38c0031f80 comp rx=0 tx=0).stop 2024-09-21T05:43:56.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 shutdown_connections 2024-09-21T05:43:56.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38c81f3b50 0x7f38c80a94a0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38c80cbf70 0x7f38c8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f38b4078bc0 0x7f38b407b0d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 --2- 172.21.15.22:0/4195405036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38b0004e60 0x7f38c81f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 >> 172.21.15.22:0/4195405036 conn(0x7f38c80ced50 msgr2=0x7f38c815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:56.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 shutdown_connections 2024-09-21T05:43:56.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.282+0000 7f38ceca9640 1 -- 172.21.15.22:0/4195405036 wait complete. 2024-09-21T05:43:56.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:43:56.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/1924332477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06f6e0 msgr2=0x7fd2ac06e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/1924332477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06f6e0 0x7fd2ac06e610 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009830 tx=0x7fd29c02a3f0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/1924332477 shutdown_connections 2024-09-21T05:43:56.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/1924332477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06f6e0 0x7fd2ac06e610 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/1924332477 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2ac06e0a0 0x7fd2ac0b97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/1924332477 >> 172.21.15.22:0/1924332477 conn(0x7fd2ac09d840 msgr2=0x7fd2ac08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/1924332477 shutdown_connections 2024-09-21T05:43:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.422+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/1924332477 wait complete. 2024-09-21T05:43:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.424+0000 7fd2b3ef9640 1 Processor -- start 2024-09-21T05:43:56.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.424+0000 7fd2b3ef9640 1 -- start start 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b3ef9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 0x7fd2ac0ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b3ef9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2ac06f6e0 0x7fd2ac1f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b3ef9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2ac0d0e50 0x7fd2ac1f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b3ef9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac0cfcd0 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b3ef9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac0cfb50 con 0x7fd2ac0d0e50 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b3ef9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd2ac0cfe50 con 0x7fd2ac06f6e0 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 0x7fd2ac0ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 0x7fd2ac0ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59836/0 (socket says 172.21.15.22:59836) 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 -- 172.21.15.22:0/2223729962 learned_addr learned my addr 172.21.15.22:0/2223729962 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1494640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2ac06f6e0 0x7fd2ac1f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b2496640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2ac0d0e50 0x7fd2ac1f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 -- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2ac06f6e0 msgr2=0x7fd2ac1f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2ac06f6e0 0x7fd2ac1f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 -- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2ac0d0e50 msgr2=0x7fd2ac1f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2ac0d0e50 0x7fd2ac1f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 -- 172.21.15.22:0/2223729962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac1f6290 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.425+0000 7fd2b1c95640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 0x7fd2ac0ca550 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800d720 tx=0x7fd2a800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.426+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8017070 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.426+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2a8009b30 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.426+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac09e2e0 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.426+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8010460 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.426+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac09e8a0 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.427+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac0d01e0 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.427+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd2a801a070 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.428+0000 7fd2a27fc640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd290078b90 0x7fd29007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.428+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7fd2a809abe0 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.428+0000 7fd2b1494640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd290078b90 0x7fd29007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.429+0000 7fd2b1494640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd290078b90 0x7fd29007b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd29c034000 tx=0x7fd29c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:56.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.432+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2a8063350 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.620+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fd2ac0b2eb0 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.624+0000 7fd2a27fc640 1 -- 172.21.15.22:0/2223729962 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7fd2a8062c50 con 0x7fd2ac06e0a0 2024-09-21T05:43:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd290078b90 msgr2=0x7fd29007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd290078b90 0x7fd29007b0a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd29c034000 tx=0x7fd29c004730 comp rx=0 tx=0).stop 2024-09-21T05:43:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 msgr2=0x7fd2ac0ca550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 0x7fd2ac0ca550 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800d720 tx=0x7fd2a800dbe0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 shutdown_connections 2024-09-21T05:43:56.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd2ac0d0e50 0x7fd2ac1f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd290078b90 0x7fd29007b0a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd2ac06f6e0 0x7fd2ac1f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 --2- 172.21.15.22:0/2223729962 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd2ac06e0a0 0x7fd2ac0ca550 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 >> 172.21.15.22:0/2223729962 conn(0x7fd2ac09d840 msgr2=0x7fd2ac161c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 shutdown_connections 2024-09-21T05:43:56.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.626+0000 7fd2b3ef9640 1 -- 172.21.15.22:0/2223729962 wait complete. 2024-09-21T05:43:56.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 -- 172.21.15.22:0/1009273549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c8005300 msgr2=0x7f44c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 --2- 172.21.15.22:0/1009273549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c8005300 0x7f44c800b9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c610 tx=0x7f44c8005830 comp rx=0 tx=0).stop 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 -- 172.21.15.22:0/1009273549 shutdown_connections 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 --2- 172.21.15.22:0/1009273549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44c8005300 0x7f44c800b9c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 -- 172.21.15.22:0/1009273549 >> 172.21.15.22:0/1009273549 conn(0x7f44e0072ae0 msgr2=0x7f44e0072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 -- 172.21.15.22:0/1009273549 shutdown_connections 2024-09-21T05:43:56.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.755+0000 7f44e6752640 1 -- 172.21.15.22:0/1009273549 wait complete. 2024-09-21T05:43:56.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.757+0000 7f44e6752640 1 Processor -- start 2024-09-21T05:43:56.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.757+0000 7f44e6752640 1 -- start start 2024-09-21T05:43:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44c8005300 0x7f44e00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44e01539b0 0x7f44e00ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 0x7f44e006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f44e00d1a60 con 0x7f44e00ab590 2024-09-21T05:43:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f44e00d18e0 con 0x7f44c8005300 2024-09-21T05:43:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f44e00d1100 con 0x7f44e01539b0 2024-09-21T05:43:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 0x7f44e006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 0x7f44e006ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59852/0 (socket says 172.21.15.22:59852) 2024-09-21T05:43:56.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 -- 172.21.15.22:0/3507841515 learned_addr learned my addr 172.21.15.22:0/3507841515 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44df7fe640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44e01539b0 0x7f44e00ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44dffff640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44c8005300 0x7f44e00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 -- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44e01539b0 msgr2=0x7f44e00ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44e01539b0 0x7f44e00ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 -- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44c8005300 msgr2=0x7f44e00cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:56.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44c8005300 0x7f44e00cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 -- 172.21.15.22:0/3507841515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e00ab9a0 con 0x7f44e00ab590 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e4cef640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 0x7f44e006ea70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c610 tx=0x7f44d8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d802b7e0 con 0x7f44e00ab590 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44d802b980 con 0x7f44e00ab590 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e00aa520 con 0x7f44e00ab590 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.759+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d802bd90 con 0x7f44e00ab590 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.760+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e006e270 con 0x7f44e00ab590 2024-09-21T05:43:56.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.760+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e015c9d0 con 0x7f44e00ab590 2024-09-21T05:43:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.761+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f44d803c030 con 0x7f44e00ab590 2024-09-21T05:43:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.762+0000 7f44dcff9640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44cc078b90 0x7f44cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.762+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f44d80ba290 con 0x7f44e00ab590 2024-09-21T05:43:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.762+0000 7f44dffff640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44cc078b90 0x7f44cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:56.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.763+0000 7f44dffff640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44cc078b90 0x7f44cc07b0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44d0006600 tx=0x7f44d0006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:56.768 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:56.766+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f44e00ab9a0 con 0x7f44e00ab590 2024-09-21T05:43:57.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.037+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f44e00b30a0 con 0x7f44e00ab590 2024-09-21T05:43:57.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.278+0000 7f44dcff9640 1 -- 172.21.15.22:0/3507841515 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v233) ==== 166+0+0 (secure 0 0 0) 0x7f44d8082a00 con 0x7f44e00ab590 2024-09-21T05:43:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44cc078b90 msgr2=0x7f44cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44cc078b90 0x7f44cc07b0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f44d0006600 tx=0x7f44d0006b70 comp rx=0 tx=0).stop 2024-09-21T05:43:57.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 msgr2=0x7f44e006ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 0x7f44e006ea70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f44d800c610 tx=0x7f44d8003ba0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 shutdown_connections 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f44e00ab590 0x7f44e006ea70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f44cc078b90 0x7f44cc07b0a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f44e01539b0 0x7f44e00ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 --2- 172.21.15.22:0/3507841515 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f44c8005300 0x7f44e00cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 >> 172.21.15.22:0/3507841515 conn(0x7f44e0072ae0 msgr2=0x7f44e00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 shutdown_connections 2024-09-21T05:43:57.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.281+0000 7f44e6752640 1 -- 172.21.15.22:0/3507841515 wait complete. 2024-09-21T05:43:57.295 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-21T05:43:57.295 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-21T05:43:57.295 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:43:57.296 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:43:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5f01b8b640 1 -- 172.21.15.22:0/491649055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ee4004000 msgr2=0x7f5ee4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/491649055 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5ee4004000 0x7f5ee4004800 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0ced00 tx=0x7f5ee4005830 comp rx=0 tx=0).stop 2024-09-21T05:43:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5efbfff640 1 -- 172.21.15.22:0/491649055 reap_dead start 2024-09-21T05:43:57.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5f01b8b640 1 -- 172.21.15.22:0/491649055 shutdown_connections 2024-09-21T05:43:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5f01b8b640 1 -- 172.21.15.22:0/491649055 >> 172.21.15.22:0/491649055 conn(0x7f5efc0cdcc0 msgr2=0x7f5efc0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5f01b8b640 1 -- 172.21.15.22:0/491649055 shutdown_connections 2024-09-21T05:43:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.414+0000 7f5f01b8b640 1 -- 172.21.15.22:0/491649055 wait complete. 2024-09-21T05:43:57.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.416+0000 7f5f01b8b640 1 Processor -- start 2024-09-21T05:43:57.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.416+0000 7f5f01b8b640 1 -- start start 2024-09-21T05:43:57.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.417+0000 7f5f01b8b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4004000 0x7f5efc15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.417+0000 7f5f01b8b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5efc15cdc0 0x7f5efc0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.417+0000 7f5f01b8b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 0x7f5efc14c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.417+0000 7f5f01b8b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0b46a0 con 0x7f5efc0ccc10 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5f01b8b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0b39b0 con 0x7f5ee4004000 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5f01b8b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5efc0b4820 con 0x7f5efc15cdc0 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 0x7f5efc14c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 0x7f5efc14c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59864/0 (socket says 172.21.15.22:59864) 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 -- 172.21.15.22:0/2311823448 learned_addr learned my addr 172.21.15.22:0/2311823448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efaffd640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5efc15cdc0 0x7f5efc0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efb7fe640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4004000 0x7f5efc15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 -- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5efc15cdc0 msgr2=0x7f5efc0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5efc15cdc0 0x7f5efc0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 -- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4004000 msgr2=0x7f5efc15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4004000 0x7f5efc15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 -- 172.21.15.22:0/2311823448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc0cd1c0 con 0x7f5efc0ccc10 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efaffd640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5efc15cdc0 0x7f5efc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efb7fe640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4004000 0x7f5efc15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5efbfff640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 0x7f5efc14c400 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5efc15f2e0 tx=0x7f5ef400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4004b20 con 0x7f5efc0ccc10 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef4004cc0 con 0x7f5efc0ccc10 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc14f150 con 0x7f5efc0ccc10 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4003bd0 con 0x7f5efc0ccc10 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.418+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc14f710 con 0x7f5efc0ccc10 2024-09-21T05:43:57.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.419+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5efc0b4db0 con 0x7f5efc0ccc10 2024-09-21T05:43:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.420+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ef4004030 con 0x7f5efc0ccc10 2024-09-21T05:43:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.421+0000 7f5ee3fff640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ee8078b90 0x7f5ee807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.421+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f5ef409b390 con 0x7f5efc0ccc10 2024-09-21T05:43:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.421+0000 7f5efb7fe640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ee8078b90 0x7f5ee807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.422+0000 7f5efb7fe640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ee8078b90 0x7f5ee807b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5eec000fd0 tx=0x7f5eec006720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:57.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.425+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ef4063a70 con 0x7f5efc0ccc10 2024-09-21T05:43:57.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.611+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5efc0ae040 con 0x7f5efc0ccc10 2024-09-21T05:43:57.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.612+0000 7f5ee3fff640 1 -- 172.21.15.22:0/2311823448 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+13155 (secure 0 0 0) 0x7f5ef4063370 con 0x7f5efc0ccc10 2024-09-21T05:43:57.614 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:43:57.615 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":74,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:43:56.273501+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.134:6836/2827334455":"2024-09-22T05:43:53.265689+0000","172.21.15.134:6834/2827334455":"2024-09-22T05:43:53.265689+0000","172.21.15.134:6837/1820431436":"2024-09-22T05:42:38.349537+0000","172.21.15.134:6835/1820431436":"2024-09-22T05:42:38.349537+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ee8078b90 msgr2=0x7f5ee807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ee8078b90 0x7f5ee807b0a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5eec000fd0 tx=0x7f5eec006720 comp rx=0 tx=0).stop 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 msgr2=0x7f5efc14c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 0x7f5efc14c400 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5efc15f2e0 tx=0x7f5ef400f3c0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 shutdown_connections 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5efc0ccc10 0x7f5efc14c400 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ee8078b90 0x7f5ee807b0a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5efc15cdc0 0x7f5efc0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 --2- 172.21.15.22:0/2311823448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5ee4004000 0x7f5efc15c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.614+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 >> 172.21.15.22:0/2311823448 conn(0x7f5efc0cdcc0 msgr2=0x7f5efc15e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:57.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.615+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 shutdown_connections 2024-09-21T05:43:57.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.615+0000 7f5f01b8b640 1 -- 172.21.15.22:0/2311823448 wait complete. 2024-09-21T05:43:57.628 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:43:57.628 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 -- 172.21.15.22:0/3994882860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 msgr2=0x7f104c0cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 --2- 172.21.15.22:0/3994882860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0cf040 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f103c009830 tx=0x7f103c02a510 comp rx=0 tx=0).stop 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 -- 172.21.15.22:0/3994882860 shutdown_connections 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 --2- 172.21.15.22:0/3994882860 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0cf040 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 -- 172.21.15.22:0/3994882860 >> 172.21.15.22:0/3994882860 conn(0x7f104c09d880 msgr2=0x7f104c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 -- 172.21.15.22:0/3994882860 shutdown_connections 2024-09-21T05:43:57.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.746+0000 7f105128b640 1 -- 172.21.15.22:0/3994882860 wait complete. 2024-09-21T05:43:57.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.747+0000 7f105128b640 1 Processor -- start 2024-09-21T05:43:57.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.747+0000 7f105128b640 1 -- start start 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f105128b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f104c0b4a50 0x7f104c153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f105128b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f104c0d0c70 0x7f104c0ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f105128b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f105128b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f104c0b5570 con 0x7f104c0d2430 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f105128b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1624d0 con 0x7f104c0d0c70 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f105128b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f104c0b56f0 con 0x7f104c0b4a50 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59878/0 (socket says 172.21.15.22:59878) 2024-09-21T05:43:57.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 -- 172.21.15.22:0/1252304622 learned_addr learned my addr 172.21.15.22:0/1252304622 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104affd640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f104c0b4a50 0x7f104c153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f1043fff640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f104c0d0c70 0x7f104c0ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 -- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f104c0b4a50 msgr2=0x7f104c153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f104c0b4a50 0x7f104c153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 -- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f104c0d0c70 msgr2=0x7f104c0ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f104c0d0c70 0x7f104c0ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 -- 172.21.15.22:0/1252304622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c15c600 con 0x7f104c0d2430 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.749+0000 7f104b7fe640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0b18a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f104400c6c0 tx=0x7f104400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.750+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044017070 con 0x7f104c0d2430 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.750+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1044003c60 con 0x7f104c0d2430 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.750+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c0c97b0 con 0x7f104c0d2430 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.750+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440052d0 con 0x7f104c0d2430 2024-09-21T05:43:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.750+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c0c9d70 con 0x7f104c0d2430 2024-09-21T05:43:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.751+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c0c9520 con 0x7f104c0d2430 2024-09-21T05:43:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.751+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f104401a070 con 0x7f104c0d2430 2024-09-21T05:43:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.752+0000 7f10437fe640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f102c078bc0 0x7f102c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.752+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f1044099fc0 con 0x7f104c0d2430 2024-09-21T05:43:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.752+0000 7f104affd640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f102c078bc0 0x7f102c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.753+0000 7f104affd640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f102c078bc0 0x7f102c07b0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f103c034000 tx=0x7f103c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:57.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.756+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f104c15c600 con 0x7f104c0d2430 2024-09-21T05:43:57.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.944+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f104c0cf040 con 0x7f104c0d2430 2024-09-21T05:43:57.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.944+0000 7f10437fe640 1 -- 172.21.15.22:0/1252304622 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) ==== 93+0+10 (secure 0 0 0) 0x7f1044062040 con 0x7f104c0d2430 2024-09-21T05:43:57.947 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:43:57.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f102c078bc0 msgr2=0x7f102c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f102c078bc0 0x7f102c07b0d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f103c034000 tx=0x7f103c004730 comp rx=0 tx=0).stop 2024-09-21T05:43:57.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 msgr2=0x7f104c0b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:57.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0b18a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f104400c6c0 tx=0x7f104400cb80 comp rx=0 tx=0).stop 2024-09-21T05:43:57.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 shutdown_connections 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f104c0d2430 0x7f104c0b18a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f104c0d0c70 0x7f104c0ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f102c078bc0 0x7f102c07b0d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 --2- 172.21.15.22:0/1252304622 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f104c0b4a50 0x7f104c153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 >> 172.21.15.22:0/1252304622 conn(0x7f104c09d880 msgr2=0x7f104c15ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 shutdown_connections 2024-09-21T05:43:57.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:57.947+0000 7f105128b640 1 -- 172.21.15.22:0/1252304622 wait complete. 2024-09-21T05:43:57.961 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.075+0000 7f857ee4d640 1 -- 172.21.15.22:0/842582218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 msgr2=0x7f8564003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.075+0000 7f857ee4d640 1 --2- 172.21.15.22:0/842582218 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8564003ea0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8568001660 tx=0x7f8564005830 comp rx=0 tx=0).stop 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.076+0000 7f857d3ea640 1 -- 172.21.15.22:0/842582218 reap_dead start 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.076+0000 7f857ee4d640 1 -- 172.21.15.22:0/842582218 shutdown_connections 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.076+0000 7f857ee4d640 1 -- 172.21.15.22:0/842582218 >> 172.21.15.22:0/842582218 conn(0x7f85780cee20 msgr2=0x7f85780cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.076+0000 7f857ee4d640 1 -- 172.21.15.22:0/842582218 shutdown_connections 2024-09-21T05:43:58.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.076+0000 7f857ee4d640 1 -- 172.21.15.22:0/842582218 wait complete. 2024-09-21T05:43:58.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.077+0000 7f857ee4d640 1 Processor -- start 2024-09-21T05:43:58.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.078+0000 7f857ee4d640 1 -- start start 2024-09-21T05:43:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857ee4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8578140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857ee4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85781f73d0 0x7f85781e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:58.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857ee4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578159dd0 0x7f85781f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857ee4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f85780d1480 con 0x7f8564003a30 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857ee4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f85780d1300 con 0x7f8578159dd0 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857ee4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f85780d1600 con 0x7f85781f73d0 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8578140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8578140eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59894/0 (socket says 172.21.15.22:59894) 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 -- 172.21.15.22:0/3713474283 learned_addr learned my addr 172.21.15.22:0/3713474283 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f8577fff640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85781f73d0 0x7f85781e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857d3ea640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578159dd0 0x7f85781f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 -- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85781f73d0 msgr2=0x7f85781e6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85781f73d0 0x7f85781e6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 -- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578159dd0 msgr2=0x7f85781f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578159dd0 0x7f85781f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857cbe9640 1 -- 172.21.15.22:0/3713474283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8578141790 con 0x7f8564003a30 2024-09-21T05:43:58.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.079+0000 7f857d3ea640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578159dd0 0x7f85781f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:58.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.080+0000 7f857cbe9640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8578140eb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8568001780 tx=0x7f8568006360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:58.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.080+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568041070 con 0x7f8564003a30 2024-09-21T05:43:58.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.080+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8568003af0 con 0x7f8564003a30 2024-09-21T05:43:58.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.080+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8578141a70 con 0x7f8564003a30 2024-09-21T05:43:58.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.080+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568038070 con 0x7f8564003a30 2024-09-21T05:43:58.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.080+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781e7f10 con 0x7f8564003a30 2024-09-21T05:43:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.081+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85780d1b00 con 0x7f8564003a30 2024-09-21T05:43:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.081+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8568044070 con 0x7f8564003a30 2024-09-21T05:43:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.082+0000 7f85757fa640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8558078b90 0x7f855807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.082+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f85680ba2e0 con 0x7f8564003a30 2024-09-21T05:43:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.082+0000 7f8577fff640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8558078b90 0x7f855807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:58.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.083+0000 7f8577fff640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8558078b90 0x7f855807b0a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8560001330 tx=0x7f8560006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:58.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.087+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8568082a50 con 0x7f8564003a30 2024-09-21T05:43:58.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:58.346+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f8578157600 con 0x7f8564003a30 2024-09-21T05:43:59.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.285+0000 7f85757fa640 1 -- 172.21.15.22:0/3713474283 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v235) ==== 170+0+0 (secure 0 0 0) 0x7f8568082350 con 0x7f8564003a30 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.288+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8558078b90 msgr2=0x7f855807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8558078b90 0x7f855807b0a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8560001330 tx=0x7f8560006b20 comp rx=0 tx=0).stop 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 msgr2=0x7f8578140eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8578140eb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8568001780 tx=0x7f8568006360 comp rx=0 tx=0).stop 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 shutdown_connections 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8578159dd0 0x7f85781f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8558078b90 0x7f855807b0a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f85781f73d0 0x7f85781e6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 --2- 172.21.15.22:0/3713474283 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8564003a30 0x7f8578140eb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 >> 172.21.15.22:0/3713474283 conn(0x7f85780cee20 msgr2=0x7f85780c7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:59.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 shutdown_connections 2024-09-21T05:43:59.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.289+0000 7f857ee4d640 1 -- 172.21.15.22:0/3713474283 wait complete. 2024-09-21T05:43:59.305 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:43:59.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/1235108926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b80b4530 msgr2=0x7fa0b815c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/1235108926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b80b4530 0x7fa0b815c830 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009830 tx=0x7fa0a802a530 comp rx=0 tx=0).stop 2024-09-21T05:43:59.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/1235108926 shutdown_connections 2024-09-21T05:43:59.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/1235108926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b80b4530 0x7fa0b815c830 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/1235108926 >> 172.21.15.22:0/1235108926 conn(0x7fa0b80cee20 msgr2=0x7fa0b80cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:59.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/1235108926 shutdown_connections 2024-09-21T05:43:59.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.427+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/1235108926 wait complete. 2024-09-21T05:43:59.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.429+0000 7fa0bf8b6640 1 Processor -- start 2024-09-21T05:43:59.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.429+0000 7fa0bf8b6640 1 -- start start 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bf8b6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0b80cff70 0x7fa0b813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bf8b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 0x7fa0b80b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bf8b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8153be0 0x7fa0b8153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bf8b6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b80d1900 con 0x7fa0b8160c90 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bf8b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b80d1780 con 0x7fa0b8153be0 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bf8b6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b80d1a80 con 0x7fa0b80cff70 2024-09-21T05:43:59.437 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bce51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 0x7fa0b80b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bce51640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 0x7fa0b80b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59902/0 (socket says 172.21.15.22:59902) 2024-09-21T05:43:59.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.430+0000 7fa0bce51640 1 -- 172.21.15.22:0/902239989 learned_addr learned my addr 172.21.15.22:0/902239989 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:59.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bd652640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0b80cff70 0x7fa0b813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bde53640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8153be0 0x7fa0b8153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bce51640 1 -- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0b80cff70 msgr2=0x7fa0b813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bce51640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0b80cff70 0x7fa0b813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bce51640 1 -- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8153be0 msgr2=0x7fa0b8153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bce51640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8153be0 0x7fa0b8153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bce51640 1 -- 172.21.15.22:0/902239989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b8153450 con 0x7fa0b8160c90 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bde53640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8153be0 0x7fa0b8153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bd652640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0b80cff70 0x7fa0b813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:43:59.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bce51640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 0x7fa0b80b6640 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009950 tx=0x7fa0a8033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:59.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8031d60 con 0x7fa0b8160c90 2024-09-21T05:43:59.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0a8005020 con 0x7fa0b8160c90 2024-09-21T05:43:59.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b80b6f70 con 0x7fa0b8160c90 2024-09-21T05:43:59.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8005470 con 0x7fa0b8160c90 2024-09-21T05:43:59.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.431+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b8154c60 con 0x7fa0b8160c90 2024-09-21T05:43:59.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.432+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b815c830 con 0x7fa0b8160c90 2024-09-21T05:43:59.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.433+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa0a80047a0 con 0x7fa0b8160c90 2024-09-21T05:43:59.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.434+0000 7fa0adffb640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa09c078b90 0x7fa09c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.434+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7fa0a80bbdb0 con 0x7fa0b8160c90 2024-09-21T05:43:59.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.434+0000 7fa0bd652640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa09c078b90 0x7fa09c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.434+0000 7fa0bd652640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa09c078b90 0x7fa09c07b0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001d10 tx=0x7fa0b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:59.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.438+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0a80c0070 con 0x7fa0b8160c90 2024-09-21T05:43:59.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.625+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa0b80693e0 con 0x7fa0b8160c90 2024-09-21T05:43:59.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.626+0000 7fa0adffb640 1 -- 172.21.15.22:0/902239989 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7fa0a803f030 con 0x7fa0b8160c90 2024-09-21T05:43:59.628 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:43:59.628 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:43:59.628 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:43:59.628 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:43:59.628 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:43:59.629 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:43:59.630 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:43:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa09c078b90 msgr2=0x7fa09c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa09c078b90 0x7fa09c07b0a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4001d10 tx=0x7fa0b4009040 comp rx=0 tx=0).stop 2024-09-21T05:43:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 msgr2=0x7fa0b80b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 0x7fa0b80b6640 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8009950 tx=0x7fa0a8033ce0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 shutdown_connections 2024-09-21T05:43:59.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0b8153be0 0x7fa0b8153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0b8160c90 0x7fa0b80b6640 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa09c078b90 0x7fa09c07b0a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 --2- 172.21.15.22:0/902239989 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0b80cff70 0x7fa0b813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 >> 172.21.15.22:0/902239989 conn(0x7fa0b80cee20 msgr2=0x7fa0b81578f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 shutdown_connections 2024-09-21T05:43:59.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.628+0000 7fa0bf8b6640 1 -- 172.21.15.22:0/902239989 wait complete. 2024-09-21T05:43:59.642 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.760+0000 7feea8969640 1 -- 172.21.15.22:0/766885127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee900039e0 msgr2=0x7fee90003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.760+0000 7feea8969640 1 --2- 172.21.15.22:0/766885127 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fee900039e0 0x7fee90003e10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fee94009ec0 tx=0x7fee90005830 comp rx=0 tx=0).stop 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.760+0000 7feea2d9d640 1 -- 172.21.15.22:0/766885127 reap_dead start 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.760+0000 7feea8969640 1 -- 172.21.15.22:0/766885127 shutdown_connections 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.760+0000 7feea8969640 1 -- 172.21.15.22:0/766885127 >> 172.21.15.22:0/766885127 conn(0x7feea409d830 msgr2=0x7feea40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.760+0000 7feea8969640 1 -- 172.21.15.22:0/766885127 shutdown_connections 2024-09-21T05:43:59.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.761+0000 7feea8969640 1 -- 172.21.15.22:0/766885127 wait complete. 2024-09-21T05:43:59.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.762+0000 7feea8969640 1 Processor -- start 2024-09-21T05:43:59.765 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.762+0000 7feea8969640 1 -- start start 2024-09-21T05:43:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea8969640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee900039e0 0x7feea40b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea8969640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 0x7feea41f8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea8969640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea41f8eb0 0x7feea40b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea8969640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feea406dea0 con 0x7feea40b1190 2024-09-21T05:43:59.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea8969640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feea40cfed0 con 0x7feea41f8eb0 2024-09-21T05:43:59.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea8969640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feea406e020 con 0x7fee900039e0 2024-09-21T05:43:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 0x7feea41f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 0x7feea41f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59928/0 (socket says 172.21.15.22:59928) 2024-09-21T05:43:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 -- 172.21.15.22:0/2066315920 learned_addr learned my addr 172.21.15.22:0/2066315920 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:43:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea259c640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee900039e0 0x7feea40b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea2d9d640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea41f8eb0 0x7feea40b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 -- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee900039e0 msgr2=0x7feea40b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee900039e0 0x7feea40b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 -- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea41f8eb0 msgr2=0x7feea40b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea41f8eb0 0x7feea40b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 -- 172.21.15.22:0/2066315920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea40b4e10 con 0x7feea40b1190 2024-09-21T05:43:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.764+0000 7feea1d9b640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 0x7feea41f8970 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006630 tx=0x7fee8c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:59.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.765+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c015070 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.765+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee8c00f960 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.765+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea41f92c0 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.765+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00fdb0 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.765+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea40c9840 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.766+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea40d0050 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.766+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee8c011430 con 0x7feea40b1190 2024-09-21T05:43:59.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.767+0000 7fee9affd640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee84078b90 0x7fee8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:43:59.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.767+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7fee8c09b7b0 con 0x7feea40b1190 2024-09-21T05:43:59.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.767+0000 7feea259c640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee84078b90 0x7fee8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:43:59.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.768+0000 7feea259c640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee84078b90 0x7fee8407b0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fee94035000 tx=0x7fee9402d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:43:59.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.772+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feea41f92c0 con 0x7feea40b1190 2024-09-21T05:43:59.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.960+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7feea40c7bb0 con 0x7feea40b1190 2024-09-21T05:43:59.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.964+0000 7fee9affd640 1 -- 172.21.15.22:0/2066315920 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7feea40b4e10 con 0x7feea40b1190 2024-09-21T05:43:59.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee84078b90 msgr2=0x7fee8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee84078b90 0x7fee8407b0a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fee94035000 tx=0x7fee9402d040 comp rx=0 tx=0).stop 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 msgr2=0x7feea41f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 0x7feea41f8970 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fee8c006630 tx=0x7fee8c0039c0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 shutdown_connections 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feea41f8eb0 0x7feea40b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feea40b1190 0x7feea41f8970 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fee84078b90 0x7fee8407b0a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 --2- 172.21.15.22:0/2066315920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fee900039e0 0x7feea40b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 >> 172.21.15.22:0/2066315920 conn(0x7feea409d830 msgr2=0x7feea40b8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 shutdown_connections 2024-09-21T05:43:59.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:43:59.966+0000 7feea8969640 1 -- 172.21.15.22:0/2066315920 wait complete. 2024-09-21T05:43:59.980 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:43:59.980 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:44:00.021 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:44:00.071 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:44:00.071 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:44:00.072 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:00.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:44:00.088 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:44:00.088 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:00.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:44:00.107 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:44:00.107 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:00.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:44:00.143 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:44:00.144 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:00.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:44:00.164 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:44:00.164 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:00.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b3f8a640 1 -- 172.21.15.22:0/2794703291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 msgr2=0x7f8698004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/2794703291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f8698004800 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0b7120 tx=0x7f8698005830 comp rx=0 tx=0).stop 2024-09-21T05:44:00.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b2527640 1 -- 172.21.15.22:0/2794703291 reap_dead start 2024-09-21T05:44:00.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b3f8a640 1 -- 172.21.15.22:0/2794703291 shutdown_connections 2024-09-21T05:44:00.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b3f8a640 1 -- 172.21.15.22:0/2794703291 >> 172.21.15.22:0/2794703291 conn(0x7f86ac08bfc0 msgr2=0x7f86ac09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:00.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b3f8a640 1 -- 172.21.15.22:0/2794703291 shutdown_connections 2024-09-21T05:44:00.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.299+0000 7f86b3f8a640 1 -- 172.21.15.22:0/2794703291 wait complete. 2024-09-21T05:44:00.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.301+0000 7f86b3f8a640 1 Processor -- start 2024-09-21T05:44:00.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.301+0000 7f86b3f8a640 1 -- start start 2024-09-21T05:44:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.302+0000 7f86b3f8a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f86ac1e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.302+0000 7f86b3f8a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ac1ea510 0x7f86ac0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.302+0000 7f86b3f8a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ac0cd420 0x7f86ac1f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.302+0000 7f86b3f8a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac0d1c00 con 0x7f8698003fd0 2024-09-21T05:44:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.302+0000 7f86b3f8a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac0d1a80 con 0x7f86ac0cd420 2024-09-21T05:44:00.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.302+0000 7f86b3f8a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86ac0d1d80 con 0x7f86ac1ea510 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f86ac1e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f86ac1e9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59940/0 (socket says 172.21.15.22:59940) 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 -- 172.21.15.22:0/743754022 learned_addr learned my addr 172.21.15.22:0/743754022 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1525640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ac1ea510 0x7f86ac0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b2527640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ac0cd420 0x7f86ac1f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 -- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ac1ea510 msgr2=0x7f86ac0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ac1ea510 0x7f86ac0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 -- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ac0cd420 msgr2=0x7f86ac1f4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ac0cd420 0x7f86ac1f4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 -- 172.21.15.22:0/743754022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac0cb7e0 con 0x7f8698003fd0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b1d26640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f86ac1e9fd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f86a8011ab0 tx=0x7f86a8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a801e070 con 0x7f8698003fd0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a800fc70 con 0x7f8698003fd0 2024-09-21T05:44:00.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac0c9b50 con 0x7f8698003fd0 2024-09-21T05:44:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8004290 con 0x7f8698003fd0 2024-09-21T05:44:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.303+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac0ca110 con 0x7f8698003fd0 2024-09-21T05:44:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.304+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86ac1626e0 con 0x7f8698003fd0 2024-09-21T05:44:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.305+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f86a8021070 con 0x7f8698003fd0 2024-09-21T05:44:00.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.306+0000 7f86a27fc640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8690078b90 0x7f869007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.306+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f86a809b270 con 0x7f8698003fd0 2024-09-21T05:44:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.306+0000 7f86b1525640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8690078b90 0x7f869007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.306+0000 7f86b1525640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8690078b90 0x7f869007b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f869c033000 tx=0x7f869c02d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:00.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.310+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f86a800fe60 con 0x7f8698003fd0 2024-09-21T05:44:00.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.536+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86ac0b31a0 con 0x7f8698003fd0 2024-09-21T05:44:00.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.536+0000 7f86a27fc640 1 -- 172.21.15.22:0/743754022 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3914 (secure 0 0 0) 0x7f86a8063300 con 0x7f8698003fd0 2024-09-21T05:44:00.539 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:00.539 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":55,"btime":"2024-09-21T05:43:54:267682+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5462,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/380003189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":380003189},{"type":"v1","addr":"172.21.15.22:6834","nonce":380003189}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5465,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/1617177348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":1617177348},{"type":"v1","addr":"172.21.15.22:6835","nonce":1617177348}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5789,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/3372364269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":3372364269},{"type":"v1","addr":"172.21.15.134:6836","nonce":3372364269}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55},{"gid":5989,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2450635019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2450635019},{"type":"v1","addr":"172.21.15.134:6837","nonce":2450635019}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":47}],"filesystems":[]} 2024-09-21T05:44:00.540 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 55 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.538+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8690078b90 msgr2=0x7f869007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.538+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8690078b90 0x7f869007b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f869c033000 tx=0x7f869c02d000 comp rx=0 tx=0).stop 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 msgr2=0x7f86ac1e9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f86ac1e9fd0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f86a8011ab0 tx=0x7f86a8011f70 comp rx=0 tx=0).stop 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 shutdown_connections 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86ac0cd420 0x7f86ac1f4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.541 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8690078b90 0x7f869007b0a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86ac1ea510 0x7f86ac0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 --2- 172.21.15.22:0/743754022 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8698003fd0 0x7f86ac1e9fd0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 >> 172.21.15.22:0/743754022 conn(0x7f86ac08bfc0 msgr2=0x7f86ac161d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:00.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 shutdown_connections 2024-09-21T05:44:00.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.539+0000 7f86b3f8a640 1 -- 172.21.15.22:0/743754022 wait complete. 2024-09-21T05:44:00.552 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:44:00.552 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:44:00.553 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:44:00.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:44:00.553 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:44:00.553 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:44:00.553 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:44:00.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:44:00.553 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:44:00.553 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:44:00.554 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:44:00.554 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:44:00.554 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:44:00.554 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:44:00.554 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:44:00.554 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:44:00.563 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:44:00.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:44:00.564 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:44:00.564 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:44:00.568 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:44:00.568 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:44:00.569 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:44:00.569 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:44:00.607 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:44:00.607 DEBUG:teuthology.parallel:result is None 2024-09-21T05:44:00.607 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:44:00.608 DEBUG:teuthology.parallel:result is None 2024-09-21T05:44:00.611 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:44:00.611 DEBUG:teuthology.parallel:result is None 2024-09-21T05:44:00.611 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:44:00.611 DEBUG:teuthology.parallel:result is None 2024-09-21T05:44:00.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:44:00.697 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:44:00.698 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:44:00.705 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:44:00.705 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.733+0000 7f002b60a640 1 -- 172.21.15.22:0/4261814028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240b3dc0 msgr2=0x7f00240b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.733+0000 7f002b60a640 1 --2- 172.21.15.22:0/4261814028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240b3dc0 0x7f00240b41d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0018009860 tx=0x7f001802a510 comp rx=0 tx=0).stop 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.733+0000 7f002b60a640 1 -- 172.21.15.22:0/4261814028 shutdown_connections 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.734+0000 7f002b60a640 1 --2- 172.21.15.22:0/4261814028 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240b3dc0 0x7f00240b41d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.734+0000 7f002b60a640 1 -- 172.21.15.22:0/4261814028 >> 172.21.15.22:0/4261814028 conn(0x7f00240cedc0 msgr2=0x7f00240cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.734+0000 7f002b60a640 1 -- 172.21.15.22:0/4261814028 shutdown_connections 2024-09-21T05:44:00.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.734+0000 7f002b60a640 1 -- 172.21.15.22:0/4261814028 wait complete. 2024-09-21T05:44:00.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.735+0000 7f002b60a640 1 Processor -- start 2024-09-21T05:44:00.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.735+0000 7f002b60a640 1 -- start start 2024-09-21T05:44:00.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f002b60a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 0x7f0024150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f002b60a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00240d0240 0x7f002414a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f002b60a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240d1d60 0x7f0024152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f002b60a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00240cd390 con 0x7f00240b3dc0 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f002b60a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00240cd210 con 0x7f00240d0240 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f002b60a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00240d0d60 con 0x7f00240d1d60 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 0x7f0024150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 0x7f0024150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60008/0 (socket says 172.21.15.22:60008) 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 -- 172.21.15.22:0/3865151251 learned_addr learned my addr 172.21.15.22:0/3865151251 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f0028ba5640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00240d0240 0x7f002414a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f0029ba7640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240d1d60 0x7f0024152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 -- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240d1d60 msgr2=0x7f0024152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240d1d60 0x7f0024152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 -- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00240d0240 msgr2=0x7f002414a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:00.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00240d0240 0x7f002414a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 -- 172.21.15.22:0/3865151251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0024153000 con 0x7f00240b3dc0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f0029ba7640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240d1d60 0x7f0024152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f0028ba5640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00240d0240 0x7f002414a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.737+0000 7f00293a6640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 0x7f0024150400 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f002000c4d0 tx=0x7f002000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.738+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020017070 con 0x7f00240b3dc0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.738+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002414e240 con 0x7f00240b3dc0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.738+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00200052d0 con 0x7f00240b3dc0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.738+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020003810 con 0x7f00240b3dc0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.738+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00241532c0 con 0x7f00240b3dc0 2024-09-21T05:44:00.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.739+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00240b41d0 con 0x7f00240b3dc0 2024-09-21T05:44:00.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.739+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f002001a070 con 0x7f00240b3dc0 2024-09-21T05:44:00.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.740+0000 7f0015ffb640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0008078b90 0x7f000807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:00.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.740+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5029+0+0 (secure 0 0 0) 0x7f0020099e90 con 0x7f00240b3dc0 2024-09-21T05:44:00.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.741+0000 7f0028ba5640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0008078b90 0x7f000807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:00.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.741+0000 7f0028ba5640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0008078b90 0x7f000807b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0018034000 tx=0x7f0018004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:00.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.745+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00240b4710 con 0x7f00240b3dc0 2024-09-21T05:44:00.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:00.945+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0024155300 con 0x7f00240b3dc0 2024-09-21T05:44:01.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.305+0000 7f0015ffb640 1 -- 172.21.15.22:0/3865151251 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) ==== 97+0+0 (secure 0 0 0) 0x7f0020061e90 con 0x7f00240b3dc0 2024-09-21T05:44:01.307 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:44:01.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0008078b90 msgr2=0x7f000807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0008078b90 0x7f000807b0a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0018034000 tx=0x7f0018004730 comp rx=0 tx=0).stop 2024-09-21T05:44:01.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 msgr2=0x7f0024150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 0x7f0024150400 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f002000c4d0 tx=0x7f002000c990 comp rx=0 tx=0).stop 2024-09-21T05:44:01.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 shutdown_connections 2024-09-21T05:44:01.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0008078b90 0x7f000807b0a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00240d1d60 0x7f0024152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00240d0240 0x7f002414a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 --2- 172.21.15.22:0/3865151251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00240b3dc0 0x7f0024150400 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 >> 172.21.15.22:0/3865151251 conn(0x7f00240cedc0 msgr2=0x7f0024158560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 shutdown_connections 2024-09-21T05:44:01.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.309+0000 7f002b60a640 1 -- 172.21.15.22:0/3865151251 wait complete. 2024-09-21T05:44:01.325 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:44:01.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6fa07640 1 -- 172.21.15.22:0/593238359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 msgr2=0x7fae5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6fa07640 1 --2- 172.21.15.22:0/593238359 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae5000b9c0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fae60006f60 tx=0x7fae50005830 comp rx=0 tx=0).stop 2024-09-21T05:44:01.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6dfa4640 1 -- 172.21.15.22:0/593238359 reap_dead start 2024-09-21T05:44:01.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6fa07640 1 -- 172.21.15.22:0/593238359 shutdown_connections 2024-09-21T05:44:01.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6fa07640 1 -- 172.21.15.22:0/593238359 >> 172.21.15.22:0/593238359 conn(0x7fae6809d7f0 msgr2=0x7fae680d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:01.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6fa07640 1 -- 172.21.15.22:0/593238359 shutdown_connections 2024-09-21T05:44:01.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.442+0000 7fae6fa07640 1 -- 172.21.15.22:0/593238359 wait complete. 2024-09-21T05:44:01.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.444+0000 7fae6fa07640 1 Processor -- start 2024-09-21T05:44:01.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.444+0000 7fae6fa07640 1 -- start start 2024-09-21T05:44:01.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6fa07640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae6809ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6fa07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae6815dbe0 0x7fae680cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6fa07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6815dff0 0x7fae680cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6fa07640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fae680b6830 con 0x7fae500051a0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6fa07640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fae680b66b0 con 0x7fae6815dff0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6fa07640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fae680b3710 con 0x7fae6815dbe0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae6809ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae6809ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60034/0 (socket says 172.21.15.22:60034) 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 -- 172.21.15.22:0/1792278196 learned_addr learned my addr 172.21.15.22:0/1792278196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6cfa2640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae6815dbe0 0x7fae680cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6dfa4640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6815dff0 0x7fae680cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 -- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae6815dbe0 msgr2=0x7fae680cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae6815dbe0 0x7fae680cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 -- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6815dff0 msgr2=0x7fae680cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6815dff0 0x7fae680cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6d7a3640 1 -- 172.21.15.22:0/1792278196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6809f440 con 0x7fae500051a0 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.445+0000 7fae6dfa4640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6815dff0 0x7fae680cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:01.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.446+0000 7fae6d7a3640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae6809ea10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fae58006110 tx=0x7fae58006510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.446+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58015070 con 0x7fae500051a0 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.446+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae680a0e60 con 0x7fae500051a0 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.446+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae5800fab0 con 0x7fae500051a0 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.446+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58011430 con 0x7fae500051a0 2024-09-21T05:44:01.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.446+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae680a1420 con 0x7fae500051a0 2024-09-21T05:44:01.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.447+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6806ecb0 con 0x7fae500051a0 2024-09-21T05:44:01.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.448+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae580115d0 con 0x7fae500051a0 2024-09-21T05:44:01.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.448+0000 7fae65ffb640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae54078b90 0x7fae5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.448+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fae5809b980 con 0x7fae500051a0 2024-09-21T05:44:01.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.449+0000 7fae6cfa2640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae54078b90 0x7fae5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.449+0000 7fae6cfa2640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae54078b90 0x7fae5407b0a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fae680b2d90 tx=0x7fae5c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:01.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.453+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae6806ecb0 con 0x7fae500051a0 2024-09-21T05:44:01.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.712+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fae680b5fa0 con 0x7fae500051a0 2024-09-21T05:44:01.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.713+0000 7fae65ffb640 1 -- 172.21.15.22:0/1792278196 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fae58064120 con 0x7fae500051a0 2024-09-21T05:44:01.715 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:01.715 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:01.715 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:44:01.715 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.715 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.716 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.717 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.718 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.719 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.720 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:44:01.721 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.722 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:44:01.723 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow r", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.724 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.725 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.726 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:44:01.727 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:44:01.728 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:44:01.728 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:44:01.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.728 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:01.728 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:44:01.728 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:01.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae54078b90 msgr2=0x7fae5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae54078b90 0x7fae5407b0a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fae680b2d90 tx=0x7fae5c005f90 comp rx=0 tx=0).stop 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 msgr2=0x7fae6809ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae6809ea10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fae58006110 tx=0x7fae58006510 comp rx=0 tx=0).stop 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 shutdown_connections 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fae6815dff0 0x7fae680cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae54078b90 0x7fae5407b0a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fae6815dbe0 0x7fae680cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 --2- 172.21.15.22:0/1792278196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fae500051a0 0x7fae6809ea10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 >> 172.21.15.22:0/1792278196 conn(0x7fae6809d7f0 msgr2=0x7fae680b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 shutdown_connections 2024-09-21T05:44:01.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.715+0000 7fae6fa07640 1 -- 172.21.15.22:0/1792278196 wait complete. 2024-09-21T05:44:01.731 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:44:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.845+0000 7fe4ef49b640 1 -- 172.21.15.22:0/1946732943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 msgr2=0x7fe4d4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.845+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/1946732943 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4d4004c10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009b20 tx=0x7fe4d4005830 comp rx=0 tx=0).stop 2024-09-21T05:44:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.846+0000 7fe4eda38640 1 -- 172.21.15.22:0/1946732943 reap_dead start 2024-09-21T05:44:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.846+0000 7fe4ef49b640 1 -- 172.21.15.22:0/1946732943 shutdown_connections 2024-09-21T05:44:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.846+0000 7fe4ef49b640 1 -- 172.21.15.22:0/1946732943 >> 172.21.15.22:0/1946732943 conn(0x7fe4e809d7f0 msgr2=0x7fe4e80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:01.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.846+0000 7fe4ef49b640 1 -- 172.21.15.22:0/1946732943 shutdown_connections 2024-09-21T05:44:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.846+0000 7fe4ef49b640 1 -- 172.21.15.22:0/1946732943 wait complete. 2024-09-21T05:44:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.847+0000 7fe4ef49b640 1 Processor -- start 2024-09-21T05:44:01.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.847+0000 7fe4ef49b640 1 -- start start 2024-09-21T05:44:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ef49b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4e8154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ef49b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e8153a20 0x7fe4e80b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ef49b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4e80cdd80 0x7fe4e81f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ef49b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80d5110 con 0x7fe4d4004800 2024-09-21T05:44:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ef49b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e80d4f90 con 0x7fe4e8153a20 2024-09-21T05:44:01.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ef49b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e806eff0 con 0x7fe4e80cdd80 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4e8154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4e8154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60058/0 (socket says 172.21.15.22:60058) 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 -- 172.21.15.22:0/2243531461 learned_addr learned my addr 172.21.15.22:0/2243531461 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4eca36640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e8153a20 0x7fe4e80b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4eda38640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4e80cdd80 0x7fe4e81f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 -- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4e80cdd80 msgr2=0x7fe4e81f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4e80cdd80 0x7fe4e81f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 -- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e8153a20 msgr2=0x7fe4e80b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e8153a20 0x7fe4e80b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 -- 172.21.15.22:0/2243531461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e80ca890 con 0x7fe4d4004800 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4eda38640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4e80cdd80 0x7fe4e81f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ed237640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4e8154010 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40063d0 tx=0x7fe4e40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.849+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4015070 con 0x7fe4d4004800 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.850+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4e4011920 con 0x7fe4d4004800 2024-09-21T05:44:01.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.850+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e80cab70 con 0x7fe4d4004800 2024-09-21T05:44:01.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.850+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4011d70 con 0x7fe4d4004800 2024-09-21T05:44:01.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.850+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e80cd860 con 0x7fe4d4004800 2024-09-21T05:44:01.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.851+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e806eab0 con 0x7fe4d4004800 2024-09-21T05:44:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.851+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4e4018070 con 0x7fe4d4004800 2024-09-21T05:44:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.852+0000 7fe4ddffb640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe4cc078b90 0x7fe4cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.852+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fe4e409b5f0 con 0x7fe4d4004800 2024-09-21T05:44:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.852+0000 7fe4eca36640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe4cc078b90 0x7fe4cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:01.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.853+0000 7fe4eca36640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe4cc078b90 0x7fe4cc07b0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8035000 tx=0x7fe4d802d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:01.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:01.857+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe4e40637c0 con 0x7fe4d4004800 2024-09-21T05:44:02.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.046+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe4e80d3fb0 con 0x7fe4d4004800 2024-09-21T05:44:02.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.047+0000 7fe4ddffb640 1 -- 172.21.15.22:0/2243531461 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fe4e40681f0 con 0x7fe4d4004800 2024-09-21T05:44:02.049 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:02.050 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:44:02.050 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:02.050 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:44:02.050 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:44:02.050 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:02.050 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.049+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe4cc078b90 msgr2=0x7fe4cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe4cc078b90 0x7fe4cc07b0a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8035000 tx=0x7fe4d802d040 comp rx=0 tx=0).stop 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 msgr2=0x7fe4e8154010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4e8154010 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe4e40063d0 tx=0x7fe4e40039c0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 shutdown_connections 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe4cc078b90 0x7fe4cc07b0a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe4e80cdd80 0x7fe4e81f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe4e8153a20 0x7fe4e80b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 --2- 172.21.15.22:0/2243531461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe4d4004800 0x7fe4e8154010 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 >> 172.21.15.22:0/2243531461 conn(0x7fe4e809d7f0 msgr2=0x7fe4e8161560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:02.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 shutdown_connections 2024-09-21T05:44:02.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.050+0000 7fe4ef49b640 1 -- 172.21.15.22:0/2243531461 wait complete. 2024-09-21T05:44:02.063 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:44:02.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:44:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 -- 172.21.15.22:0/706584452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0d3250 msgr2=0x7fd07c0b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 --2- 172.21.15.22:0/706584452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0d3250 0x7fd07c0b9700 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fd06c005fd0 tx=0x7fd06c02a940 comp rx=0 tx=0).stop 2024-09-21T05:44:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 -- 172.21.15.22:0/706584452 shutdown_connections 2024-09-21T05:44:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 --2- 172.21.15.22:0/706584452 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0d3250 0x7fd07c0b9700 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 -- 172.21.15.22:0/706584452 >> 172.21.15.22:0/706584452 conn(0x7fd07c09d7f0 msgr2=0x7fd07c0d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:02.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 -- 172.21.15.22:0/706584452 shutdown_connections 2024-09-21T05:44:02.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.182+0000 7fd081427640 1 -- 172.21.15.22:0/706584452 wait complete. 2024-09-21T05:44:02.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.184+0000 7fd081427640 1 Processor -- start 2024-09-21T05:44:02.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.184+0000 7fd081427640 1 -- start start 2024-09-21T05:44:02.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd081427640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 0x7fd07c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:02.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd081427640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c0cef10 0x7fd07c0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:02.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd081427640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c0d3250 0x7fd07c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:02.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd081427640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c0cffc0 con 0x7fd07c0b5f80 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd081427640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c0cfe40 con 0x7fd07c0cef10 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd081427640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd07c0d0140 con 0x7fd07c0d3250 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 0x7fd07c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 0x7fd07c0ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60072/0 (socket says 172.21.15.22:60072) 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 -- 172.21.15.22:0/3770671336 learned_addr learned my addr 172.21.15.22:0/3770671336 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07a7fc640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c0cef10 0x7fd07c0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07b7fe640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c0d3250 0x7fd07c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 -- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c0d3250 msgr2=0x7fd07c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c0d3250 0x7fd07c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 -- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c0cef10 msgr2=0x7fd07c0b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c0cef10 0x7fd07c0b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07affd640 1 -- 172.21.15.22:0/3770671336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c1f93a0 con 0x7fd07c0b5f80 2024-09-21T05:44:02.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.185+0000 7fd07b7fe640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c0d3250 0x7fd07c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:02.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.186+0000 7fd07affd640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 0x7fd07c0ca1e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009cd0 tx=0x7fd06c031ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:02.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.186+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c03e070 con 0x7fd07c0b5f80 2024-09-21T05:44:02.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.186+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd06c004600 con 0x7fd07c0b5f80 2024-09-21T05:44:02.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.186+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c1f9680 con 0x7fd07c0b5f80 2024-09-21T05:44:02.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.186+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c043460 con 0x7fd07c0b5f80 2024-09-21T05:44:02.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.186+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c1f9c40 con 0x7fd07c0b5f80 2024-09-21T05:44:02.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.187+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07c1f9e80 con 0x7fd07c0b5f80 2024-09-21T05:44:02.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.188+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd06c043620 con 0x7fd07c0b5f80 2024-09-21T05:44:02.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.188+0000 7fd0737fe640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd05c078b90 0x7fd05c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:02.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.189+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fd06c0bc5d0 con 0x7fd07c0b5f80 2024-09-21T05:44:02.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.189+0000 7fd07a7fc640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd05c078b90 0x7fd05c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:02.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.190+0000 7fd07a7fc640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd05c078b90 0x7fd05c07b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0ce6c0 tx=0x7fd064009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:02.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.193+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd07c1f9e80 con 0x7fd07c0b5f80 2024-09-21T05:44:02.387 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:02.384+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd07c0b9700 con 0x7fd07c0b5f80 2024-09-21T05:44:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.317+0000 7fd0737fe640 1 -- 172.21.15.22:0/3770671336 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7fd06c084df0 con 0x7fd07c0b5f80 2024-09-21T05:44:03.319 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:44:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd05c078b90 msgr2=0x7fd05c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd05c078b90 0x7fd05c07b0a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0ce6c0 tx=0x7fd064009040 comp rx=0 tx=0).stop 2024-09-21T05:44:03.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 msgr2=0x7fd07c0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 0x7fd07c0ca1e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009cd0 tx=0x7fd06c031ea0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 shutdown_connections 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd05c078b90 0x7fd05c07b0a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd07c0d3250 0x7fd07c15b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd07c0cef10 0x7fd07c0b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 --2- 172.21.15.22:0/3770671336 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd07c0b5f80 0x7fd07c0ca1e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.321+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 >> 172.21.15.22:0/3770671336 conn(0x7fd07c09d7f0 msgr2=0x7fd07c1618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:03.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.322+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 shutdown_connections 2024-09-21T05:44:03.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.322+0000 7fd081427640 1 -- 172.21.15.22:0/3770671336 wait complete. 2024-09-21T05:44:03.341 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:44:03.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.469+0000 7fdf246f9640 1 -- 172.21.15.22:0/2045261911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 msgr2=0x7fdf1c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:03.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.469+0000 7fdf246f9640 1 --2- 172.21.15.22:0/2045261911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cf090 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdf10005fd0 tx=0x7fdf1002a670 comp rx=0 tx=0).stop 2024-09-21T05:44:03.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.470+0000 7fdf246f9640 1 -- 172.21.15.22:0/2045261911 shutdown_connections 2024-09-21T05:44:03.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.470+0000 7fdf246f9640 1 --2- 172.21.15.22:0/2045261911 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cf090 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.470+0000 7fdf246f9640 1 -- 172.21.15.22:0/2045261911 >> 172.21.15.22:0/2045261911 conn(0x7fdf1c08bfc0 msgr2=0x7fdf1c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:03.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.470+0000 7fdf246f9640 1 -- 172.21.15.22:0/2045261911 shutdown_connections 2024-09-21T05:44:03.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.470+0000 7fdf246f9640 1 -- 172.21.15.22:0/2045261911 wait complete. 2024-09-21T05:44:03.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.471+0000 7fdf246f9640 1 Processor -- start 2024-09-21T05:44:03.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.471+0000 7fdf246f9640 1 -- start start 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf246f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf1c0cf5d0 0x7fdf1c0a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf246f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf246f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf1c0d2ed0 0x7fdf1c0b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf246f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c162df0 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf246f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c162f70 con 0x7fdf1c0d2ed0 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf246f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdf1c1630f0 con 0x7fdf1c0cf5d0 2024-09-21T05:44:03.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:60092/0 (socket says 172.21.15.22:60092) 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 -- 172.21.15.22:0/1408555710 learned_addr learned my addr 172.21.15.22:0/1408555710 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf22c96640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf1c0d2ed0 0x7fdf1c0b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf22495640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf1c0cf5d0 0x7fdf1c0a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 -- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf1c0cf5d0 msgr2=0x7fdf1c0a1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf1c0cf5d0 0x7fdf1c0a1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 -- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf1c0d2ed0 msgr2=0x7fdf1c0b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf1c0d2ed0 0x7fdf1c0b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.473+0000 7fdf21c94640 1 -- 172.21.15.22:0/1408555710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c1f6730 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.474+0000 7fdf21c94640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cd210 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdf10000c00 tx=0x7fdf10003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.474+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1003e030 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.474+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf10003e60 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.474+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c1f6a10 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.474+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf10035040 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.474+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c0b86b0 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.475+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf1c0cf090 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.476+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdf10041030 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.476+0000 7fdf0effd640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdf00078bc0 0x7fdf0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:03.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.476+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5227+0+0 (secure 0 0 0) 0x7fdf100bb220 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.476+0000 7fdf22495640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdf00078bc0 0x7fdf0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:03.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.477+0000 7fdf22495640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdf00078bc0 0x7fdf0007b0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf18007fd0 tx=0x7fdf18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:03.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.481+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf10083310 con 0x7fdf1c0d1a80 2024-09-21T05:44:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:03.671+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdf1c1f64c0 con 0x7fdf1c0d1a80 2024-09-21T05:44:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.323+0000 7fdf0effd640 1 -- 172.21.15.22:0/1408555710 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fdf10087d40 con 0x7fdf1c0d1a80 2024-09-21T05:44:04.325 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:44:04.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdf00078bc0 msgr2=0x7fdf0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:04.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdf00078bc0 0x7fdf0007b0d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdf18007fd0 tx=0x7fdf18009040 comp rx=0 tx=0).stop 2024-09-21T05:44:04.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 msgr2=0x7fdf1c0cd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cd210 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdf10000c00 tx=0x7fdf10003970 comp rx=0 tx=0).stop 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 shutdown_connections 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdf1c0d2ed0 0x7fdf1c0b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdf1c0d1a80 0x7fdf1c0cd210 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdf00078bc0 0x7fdf0007b0d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 --2- 172.21.15.22:0/1408555710 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdf1c0cf5d0 0x7fdf1c0a1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 >> 172.21.15.22:0/1408555710 conn(0x7fdf1c08bfc0 msgr2=0x7fdf1c15eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 shutdown_connections 2024-09-21T05:44:04.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.327+0000 7fdf246f9640 1 -- 172.21.15.22:0/1408555710 wait complete. 2024-09-21T05:44:04.356 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:44:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.479+0000 7fd881bda640 1 -- 172.21.15.22:0/1494429949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0b3a10 msgr2=0x7fd87c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.480+0000 7fd881bda640 1 --2- 172.21.15.22:0/1494429949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0b3a10 0x7fd87c0cfa20 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0d34f0 tx=0x7fd87402a800 comp rx=0 tx=0).stop 2024-09-21T05:44:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.480+0000 7fd881bda640 1 -- 172.21.15.22:0/1494429949 shutdown_connections 2024-09-21T05:44:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.480+0000 7fd881bda640 1 --2- 172.21.15.22:0/1494429949 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0b3a10 0x7fd87c0cfa20 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.480+0000 7fd881bda640 1 -- 172.21.15.22:0/1494429949 >> 172.21.15.22:0/1494429949 conn(0x7fd87c09d8f0 msgr2=0x7fd87c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.480+0000 7fd881bda640 1 -- 172.21.15.22:0/1494429949 shutdown_connections 2024-09-21T05:44:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.480+0000 7fd881bda640 1 -- 172.21.15.22:0/1494429949 wait complete. 2024-09-21T05:44:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.481+0000 7fd881bda640 1 Processor -- start 2024-09-21T05:44:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.482+0000 7fd881bda640 1 -- start start 2024-09-21T05:44:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd881bda640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd87c0b3a10 0x7fd87c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd881bda640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd87c0b9b60 0x7fd87c0b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd881bda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 0x7fd87c0ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd881bda640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c0d02f0 con 0x7fd87c0ca260 2024-09-21T05:44:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd881bda640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd87c0d0170 con 0x7fd87c0b9b60 2024-09-21T05:44:04.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd881bda640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd87c16a5a0 con 0x7fd87c0b3a10 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 0x7fd87c0ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 0x7fd87c0ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36250/0 (socket says 172.21.15.22:36250) 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 -- 172.21.15.22:0/3866315667 learned_addr learned my addr 172.21.15.22:0/3866315667 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87b7fe640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd87c0b3a10 0x7fd87c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87affd640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd87c0b9b60 0x7fd87c0b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 -- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd87c0b3a10 msgr2=0x7fd87c154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd87c0b3a10 0x7fd87c154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 -- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd87c0b9b60 msgr2=0x7fd87c0b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd87c0b9b60 0x7fd87c0b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.483+0000 7fd87bfff640 1 -- 172.21.15.22:0/3866315667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87c0cabb0 con 0x7fd87c0ca260 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.484+0000 7fd87bfff640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 0x7fd87c0ca670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd87402adf0 tx=0x7fd874033f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.484+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874042020 con 0x7fd87c0ca260 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.484+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd874004330 con 0x7fd87c0ca260 2024-09-21T05:44:04.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.484+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87c15ac10 con 0x7fd87c0ca260 2024-09-21T05:44:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.484+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874037040 con 0x7fd87c0ca260 2024-09-21T05:44:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.484+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd87c15aeb0 con 0x7fd87c0ca260 2024-09-21T05:44:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.485+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd87c0ba0d0 con 0x7fd87c0ca260 2024-09-21T05:44:04.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.486+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd874045020 con 0x7fd87c0ca260 2024-09-21T05:44:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.486+0000 7fd863fff640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd868078b90 0x7fd86807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.487+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5609+0+0 (secure 0 0 0) 0x7fd8740bc1b0 con 0x7fd87c0ca260 2024-09-21T05:44:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.487+0000 7fd87b7fe640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd868078b90 0x7fd86807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.487+0000 7fd87b7fe640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd868078b90 0x7fd86807b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000f60 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:04.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.491+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd87404c020 con 0x7fd87c0ca260 2024-09-21T05:44:04.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:04.718+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd87c0b6b70 con 0x7fd87c0ca260 2024-09-21T05:44:05.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.325+0000 7fd863fff640 1 -- 172.21.15.22:0/3866315667 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v58) ==== 174+0+0 (secure 0 0 0) 0x7fd87c0cabb0 con 0x7fd87c0ca260 2024-09-21T05:44:05.327 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 12 and data pool 13 2024-09-21T05:44:05.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.329+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd868078b90 msgr2=0x7fd86807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.329+0000 7fd881bda640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd868078b90 0x7fd86807b0a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd86c000f60 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2024-09-21T05:44:05.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.329+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 msgr2=0x7fd87c0ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.329+0000 7fd881bda640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 0x7fd87c0ca670 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd87402adf0 tx=0x7fd874033f00 comp rx=0 tx=0).stop 2024-09-21T05:44:05.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.330+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 shutdown_connections 2024-09-21T05:44:05.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.330+0000 7fd881bda640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd87c0ca260 0x7fd87c0ca670 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.330+0000 7fd881bda640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd87c0b9b60 0x7fd87c0b6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.330+0000 7fd881bda640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd868078b90 0x7fd86807b0a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.330+0000 7fd881bda640 1 --2- 172.21.15.22:0/3866315667 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd87c0b3a10 0x7fd87c154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.330+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 >> 172.21.15.22:0/3866315667 conn(0x7fd87c09d8f0 msgr2=0x7fd87c0b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:05.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.331+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 shutdown_connections 2024-09-21T05:44:05.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.332+0000 7fd881bda640 1 -- 172.21.15.22:0/3866315667 wait complete. 2024-09-21T05:44:05.371 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:44:05.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 -- 172.21.15.22:0/3509680699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28153970 msgr2=0x7fca2815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 --2- 172.21.15.22:0/3509680699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28153970 0x7fca2815d810 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fca18005fd0 tx=0x7fca1802a7a0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 -- 172.21.15.22:0/3509680699 shutdown_connections 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 --2- 172.21.15.22:0/3509680699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca280cfc00 0x7fca280ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 --2- 172.21.15.22:0/3509680699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28153970 0x7fca2815d810 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 -- 172.21.15.22:0/3509680699 >> 172.21.15.22:0/3509680699 conn(0x7fca280d15f0 msgr2=0x7fca28073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 -- 172.21.15.22:0/3509680699 shutdown_connections 2024-09-21T05:44:05.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.489+0000 7fca2d4de640 1 -- 172.21.15.22:0/3509680699 wait complete. 2024-09-21T05:44:05.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.491+0000 7fca2d4de640 1 Processor -- start 2024-09-21T05:44:05.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.491+0000 7fca2d4de640 1 -- start start 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca2d4de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 0x7fca280cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca2d4de640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca280cfc00 0x7fca280c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca2d4de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca28153970 0x7fca28159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca2d4de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fca2815e2e0 con 0x7fca28073e50 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca2d4de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fca2815e160 con 0x7fca28153970 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca2d4de640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fca2815e460 con 0x7fca280cfc00 2024-09-21T05:44:05.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 0x7fca280cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 0x7fca280cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36270/0 (socket says 172.21.15.22:36270) 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.492+0000 7fca26ffd640 1 -- 172.21.15.22:0/2796712122 learned_addr learned my addr 172.21.15.22:0/2796712122 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca267fc640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca280cfc00 0x7fca280c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca277fe640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca28153970 0x7fca28159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca26ffd640 1 -- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca280cfc00 msgr2=0x7fca280c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca26ffd640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca280cfc00 0x7fca280c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca26ffd640 1 -- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca28153970 msgr2=0x7fca28159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca26ffd640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca28153970 0x7fca28159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca26ffd640 1 -- 172.21.15.22:0/2796712122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca280712c0 con 0x7fca28073e50 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca26ffd640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 0x7fca280cbca0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fca18009b30 tx=0x7fca18032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18041020 con 0x7fca28073e50 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca18004330 con 0x7fca28073e50 2024-09-21T05:44:05.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca28070740 con 0x7fca28073e50 2024-09-21T05:44:05.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18036040 con 0x7fca28073e50 2024-09-21T05:44:05.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.493+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca28070d00 con 0x7fca28073e50 2024-09-21T05:44:05.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.494+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2815e9a0 con 0x7fca28073e50 2024-09-21T05:44:05.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.495+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca18044020 con 0x7fca28073e50 2024-09-21T05:44:05.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.495+0000 7fca1f7fe640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca08078b90 0x7fca0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.496+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fca180bbff0 con 0x7fca28073e50 2024-09-21T05:44:05.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.496+0000 7fca267fc640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca08078b90 0x7fca0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.496+0000 7fca267fc640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca08078b90 0x7fca0807b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca10005fd0 tx=0x7fca10005850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:05.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.500+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fca1804b030 con 0x7fca28073e50 2024-09-21T05:44:05.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.690+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca2815d8b0 con 0x7fca28073e50 2024-09-21T05:44:05.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.691+0000 7fca1f7fe640 1 -- 172.21.15.22:0/2796712122 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16721 (secure 0 0 0) 0x7fca180843c0 con 0x7fca28073e50 2024-09-21T05:44:05.693 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:05.694 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:44:04.719847+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:44:02.389731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T05:44:03.675773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca08078b90 msgr2=0x7fca0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca08078b90 0x7fca0807b0a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca10005fd0 tx=0x7fca10005850 comp rx=0 tx=0).stop 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 msgr2=0x7fca280cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 0x7fca280cbca0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fca18009b30 tx=0x7fca18032f10 comp rx=0 tx=0).stop 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 shutdown_connections 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fca28153970 0x7fca28159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca08078b90 0x7fca0807b0a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fca280cfc00 0x7fca280c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 --2- 172.21.15.22:0/2796712122 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fca28073e50 0x7fca280cbca0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 >> 172.21.15.22:0/2796712122 conn(0x7fca280d15f0 msgr2=0x7fca28160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:05.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 shutdown_connections 2024-09-21T05:44:05.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.694+0000 7fca2d4de640 1 -- 172.21.15.22:0/2796712122 wait complete. 2024-09-21T05:44:05.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:44:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 -- 172.21.15.22:0/1718181672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600ccba0 msgr2=0x7f22600ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 --2- 172.21.15.22:0/1718181672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600ccba0 0x7f22600ccfb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2258009830 tx=0x7f225802a5c0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 -- 172.21.15.22:0/1718181672 shutdown_connections 2024-09-21T05:44:05.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 --2- 172.21.15.22:0/1718181672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600ccba0 0x7f22600ccfb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 -- 172.21.15.22:0/1718181672 >> 172.21.15.22:0/1718181672 conn(0x7f22600b3f10 msgr2=0x7f22600b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:05.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 -- 172.21.15.22:0/1718181672 shutdown_connections 2024-09-21T05:44:05.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.824+0000 7f226509b640 1 -- 172.21.15.22:0/1718181672 wait complete. 2024-09-21T05:44:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.826+0000 7f226509b640 1 Processor -- start 2024-09-21T05:44:05.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.826+0000 7f226509b640 1 -- start start 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.827+0000 7f226509b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 0x7f22600b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.827+0000 7f226509b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22600ccba0 0x7f22600b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.827+0000 7f226509b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600d1d40 0x7f22601f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.827+0000 7f226509b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f22600d1440 con 0x7f22600b5210 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.827+0000 7f226509b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f22600d12c0 con 0x7f22600d1d40 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.827+0000 7f226509b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f22600d15c0 con 0x7f22600ccba0 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 0x7f22600b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 0x7f22600b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36280/0 (socket says 172.21.15.22:36280) 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 -- 172.21.15.22:0/1276346040 learned_addr learned my addr 172.21.15.22:0/1276346040 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225e59c640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22600ccba0 0x7f22600b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225f59e640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600d1d40 0x7f22601f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 -- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22600ccba0 msgr2=0x7f22600b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22600ccba0 0x7f22600b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 -- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600d1d40 msgr2=0x7f22601f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600d1d40 0x7f22601f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 -- 172.21.15.22:0/1276346040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22601f9ae0 con 0x7f22600b5210 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225f59e640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600d1d40 0x7f22601f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f225ed9d640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 0x7f22600b3160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f225000c780 tx=0x7f225000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250003b70 con 0x7f22600b5210 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2250003d10 con 0x7f22600b5210 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22601f9dc0 con 0x7f22600b5210 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225000aa50 con 0x7f22600b5210 2024-09-21T05:44:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.828+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22601571d0 con 0x7f22600b5210 2024-09-21T05:44:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.829+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22600ccfb0 con 0x7f22600b5210 2024-09-21T05:44:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.830+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f225000abf0 con 0x7f22600b5210 2024-09-21T05:44:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.831+0000 7f22577fe640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2240078bc0 0x7f224007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.831+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f225009ab50 con 0x7f22600b5210 2024-09-21T05:44:05.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.831+0000 7f225e59c640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2240078bc0 0x7f224007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:05.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.832+0000 7f225e59c640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2240078bc0 0x7f224007b0d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2248007fd0 tx=0x7f2248007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:05.837 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:05.836+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2250062f00 con 0x7f22600b5210 2024-09-21T05:44:06.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.024+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22600c9130 con 0x7f22600b5210 2024-09-21T05:44:06.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.025+0000 7f22577fe640 1 -- 172.21.15.22:0/1276346040 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16721 (secure 0 0 0) 0x7f2250062800 con 0x7f22600b5210 2024-09-21T05:44:06.028 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:06.028 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":81,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:44:04.719847+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:44:02.389731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T05:44:03.675773+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:44:06.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2240078bc0 msgr2=0x7f224007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2240078bc0 0x7f224007b0d0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2248007fd0 tx=0x7f2248007b00 comp rx=0 tx=0).stop 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 msgr2=0x7f22600b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 0x7f22600b3160 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f225000c780 tx=0x7f225000cc40 comp rx=0 tx=0).stop 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 shutdown_connections 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f22600d1d40 0x7f22601f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2240078bc0 0x7f224007b0d0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f22600ccba0 0x7f22600b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 --2- 172.21.15.22:0/1276346040 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f22600b5210 0x7f22600b3160 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 >> 172.21.15.22:0/1276346040 conn(0x7f22600b3f10 msgr2=0x7f22600b7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 shutdown_connections 2024-09-21T05:44:06.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.027+0000 7f226509b640 1 -- 172.21.15.22:0/1276346040 wait complete. 2024-09-21T05:44:06.041 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc3139640 1 -- 172.21.15.22:0/2203226664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4003a00 msgr2=0x7f9fa4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc3139640 1 --2- 172.21.15.22:0/2203226664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fa4003a00 0x7f9fa4003e70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c4a0 tx=0x7f9fa4005830 comp rx=0 tx=0).stop 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc16d6640 1 -- 172.21.15.22:0/2203226664 reap_dead start 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc3139640 1 -- 172.21.15.22:0/2203226664 shutdown_connections 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc3139640 1 -- 172.21.15.22:0/2203226664 >> 172.21.15.22:0/2203226664 conn(0x7f9fbc06d550 msgr2=0x7f9fbc071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc3139640 1 -- 172.21.15.22:0/2203226664 shutdown_connections 2024-09-21T05:44:06.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.159+0000 7f9fc3139640 1 -- 172.21.15.22:0/2203226664 wait complete. 2024-09-21T05:44:06.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.161+0000 7f9fc3139640 1 Processor -- start 2024-09-21T05:44:06.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.161+0000 7f9fc3139640 1 -- start start 2024-09-21T05:44:06.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc3139640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa4003a00 0x7f9fbc0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:06.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc3139640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 0x7f9fbc0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:06.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc3139640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fbc0b2810 0x7f9fbc0b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:06.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc3139640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc0cf490 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc3139640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc0cf310 con 0x7f9fa4003a00 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc3139640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9fbc0d05d0 con 0x7f9fbc0b2810 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 0x7f9fbc0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 0x7f9fbc0b22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36300/0 (socket says 172.21.15.22:36300) 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 -- 172.21.15.22:0/1641439861 learned_addr learned my addr 172.21.15.22:0/1641439861 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc0ed5640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa4003a00 0x7f9fbc0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fc16d6640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fbc0b2810 0x7f9fbc0b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 -- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fbc0b2810 msgr2=0x7f9fbc0b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fbc0b2810 0x7f9fbc0b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 -- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa4003a00 msgr2=0x7f9fbc0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa4003a00 0x7f9fbc0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.163+0000 7f9fbbfff640 1 -- 172.21.15.22:0/1641439861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc0b3d20 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.164+0000 7f9fbbfff640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 0x7f9fbc0b22d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0011b10 tx=0x7f9fb0009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.164+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb001e070 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.164+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb0004050 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.164+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc0cafd0 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.164+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000fea0 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.164+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc0cb590 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.165+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fbc0d1900 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.166+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fb0021070 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.166+0000 7f9fb97fa640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9fa8078bc0 0x7f9fa807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.166+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f9fb009b750 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.166+0000 7f9fc0ed5640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9fa8078bc0 0x7f9fa807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:06.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.167+0000 7f9fc0ed5640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9fa8078bc0 0x7f9fa807b0d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c470 tx=0x7f9fac038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:06.173 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.171+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9fbc0d1900 con 0x7f9fbc0ca8a0 2024-09-21T05:44:06.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:06.399+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9fbc0cad60 con 0x7f9fbc0ca8a0 2024-09-21T05:44:07.308 INFO:tasks.ceph.mds.d.smithi134.stderr:2024-09-21T05:44:07.306+0000 7fefef5d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:44:07.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.348+0000 7f9fb97fa640 1 -- 172.21.15.22:0/1641439861 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) ==== 118+0+0 (secure 0 0 0) 0x7f9fbc0b3d20 con 0x7f9fbc0ca8a0 2024-09-21T05:44:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9fa8078bc0 msgr2=0x7f9fa807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9fa8078bc0 0x7f9fa807b0d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00c470 tx=0x7f9fac038040 comp rx=0 tx=0).stop 2024-09-21T05:44:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 msgr2=0x7f9fbc0b22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 0x7f9fbc0b22d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0011b10 tx=0x7f9fb0009eb0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 shutdown_connections 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9fa8078bc0 0x7f9fa807b0d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9fbc0b2810 0x7f9fbc0b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9fbc0ca8a0 0x7f9fbc0b22d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 --2- 172.21.15.22:0/1641439861 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9fa4003a00 0x7f9fbc0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 >> 172.21.15.22:0/1641439861 conn(0x7f9fbc06d550 msgr2=0x7f9fbc0b7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 shutdown_connections 2024-09-21T05:44:07.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.351+0000 7f9fc3139640 1 -- 172.21.15.22:0/1641439861 wait complete. 2024-09-21T05:44:07.367 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:44:07.367 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:44:07.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 -- 172.21.15.22:0/114414819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80cff70 msgr2=0x7f4ff80d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:07.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 --2- 172.21.15.22:0/114414819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80cff70 0x7f4ff80d1da0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009860 tx=0x7f4fe802a530 comp rx=0 tx=0).stop 2024-09-21T05:44:07.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 -- 172.21.15.22:0/114414819 shutdown_connections 2024-09-21T05:44:07.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 --2- 172.21.15.22:0/114414819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80cff70 0x7f4ff80d1da0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 -- 172.21.15.22:0/114414819 >> 172.21.15.22:0/114414819 conn(0x7f4ff80cee20 msgr2=0x7f4ff80cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:07.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 -- 172.21.15.22:0/114414819 shutdown_connections 2024-09-21T05:44:07.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.492+0000 7f4ffd825640 1 -- 172.21.15.22:0/114414819 wait complete. 2024-09-21T05:44:07.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.494+0000 7f4ffd825640 1 Processor -- start 2024-09-21T05:44:07.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.494+0000 7f4ffd825640 1 -- start start 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ffd825640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80b3e20 0x7f4ff80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ffd825640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff80cff70 0x7f4ff81f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ffd825640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 0x7f4ff814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ffd825640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff80c6710 con 0x7f4ff815c930 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ffd825640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff80c6590 con 0x7f4ff80cff70 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ffd825640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff80c6890 con 0x7f4ff80b3e20 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 0x7f4ff814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ff77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 0x7f4ff814f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36302/0 (socket says 172.21.15.22:36302) 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.495+0000 7f4ff77fe640 1 -- 172.21.15.22:0/2387291609 learned_addr learned my addr 172.21.15.22:0/2387291609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff6ffd640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80b3e20 0x7f4ff80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff67fc640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff80cff70 0x7f4ff81f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:07.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff77fe640 1 -- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80b3e20 msgr2=0x7f4ff80cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff77fe640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80b3e20 0x7f4ff80cbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff77fe640 1 -- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff80cff70 msgr2=0x7f4ff81f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff77fe640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff80cff70 0x7f4ff81f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff77fe640 1 -- 172.21.15.22:0/2387291609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff81f4c10 con 0x7f4ff815c930 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ff77fe640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 0x7f4ff814f710 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000c730 tx=0x7f4ff000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0003b70 con 0x7f4ff815c930 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff80cacc0 con 0x7f4ff815c930 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ff0003d10 con 0x7f4ff815c930 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000aab0 con 0x7f4ff815c930 2024-09-21T05:44:07.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.496+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff80cb280 con 0x7f4ff815c930 2024-09-21T05:44:07.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.497+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff80d2560 con 0x7f4ff815c930 2024-09-21T05:44:07.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.498+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ff000ac50 con 0x7f4ff815c930 2024-09-21T05:44:07.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.499+0000 7f4fdffff640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fe4078bc0 0x7f4fe407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:07.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.499+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f4ff009a950 con 0x7f4ff815c930 2024-09-21T05:44:07.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.499+0000 7f4ff6ffd640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fe4078bc0 0x7f4fe407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:07.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.500+0000 7f4ff6ffd640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fe4078bc0 0x7f4fe407b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8034000 tx=0x7f4fe8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:07.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.503+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ff0062750 con 0x7f4ff815c930 2024-09-21T05:44:07.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:07.730+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f4ff80735f0 con 0x7f4ff815c930 2024-09-21T05:44:08.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.354+0000 7f4fdffff640 1 -- 172.21.15.22:0/2387291609 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) ==== 122+0+0 (secure 0 0 0) 0x7f4ff0067180 con 0x7f4ff815c930 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.357+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fe4078bc0 msgr2=0x7f4fe407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.357+0000 7f4ffd825640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fe4078bc0 0x7f4fe407b0d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8034000 tx=0x7f4fe8004730 comp rx=0 tx=0).stop 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.357+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 msgr2=0x7f4ff814f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.357+0000 7f4ffd825640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 0x7f4ff814f710 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ff000c730 tx=0x7f4ff000cbf0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 shutdown_connections 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ff815c930 0x7f4ff814f710 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ff80cff70 0x7f4ff81f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4fe4078bc0 0x7f4fe407b0d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 --2- 172.21.15.22:0/2387291609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ff80b3e20 0x7f4ff80cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 >> 172.21.15.22:0/2387291609 conn(0x7f4ff80cee20 msgr2=0x7f4ff80d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:08.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 shutdown_connections 2024-09-21T05:44:08.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.358+0000 7f4ffd825640 1 -- 172.21.15.22:0/2387291609 wait complete. 2024-09-21T05:44:08.373 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:44:08.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 -- 172.21.15.22:0/220172052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 msgr2=0x7fc6a80b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 --2- 172.21.15.22:0/220172052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a80b5210 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8074550 tx=0x7fc69802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 -- 172.21.15.22:0/220172052 shutdown_connections 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 --2- 172.21.15.22:0/220172052 >> v2:172.21.15.134:3300/0 conn(0x7fc6a815c6f0 0x7fc6a80ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 --2- 172.21.15.22:0/220172052 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a80b5210 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 -- 172.21.15.22:0/220172052 >> 172.21.15.22:0/220172052 conn(0x7fc6a80b3f10 msgr2=0x7fc6a80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 -- 172.21.15.22:0/220172052 shutdown_connections 2024-09-21T05:44:08.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.495+0000 7fc6af438640 1 -- 172.21.15.22:0/220172052 wait complete. 2024-09-21T05:44:08.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.497+0000 7fc6af438640 1 Processor -- start 2024-09-21T05:44:08.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.497+0000 7fc6af438640 1 -- start start 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.498+0000 7fc6af438640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a80ccba0 0x7fc6a80cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.498+0000 7fc6af438640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a8158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.498+0000 7fc6af438640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a815c6f0 0x7fc6a80b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.498+0000 7fc6af438640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80d11f0 con 0x7fc6a80d1d40 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.498+0000 7fc6af438640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80d1070 con 0x7fc6a80ccba0 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.498+0000 7fc6af438640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a80d1370 con 0x7fc6a815c6f0 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a8158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a8158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36320/0 (socket says 172.21.15.22:36320) 2024-09-21T05:44:08.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 -- 172.21.15.22:0/3150213388 learned_addr learned my addr 172.21.15.22:0/3150213388 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ad1d4640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a80ccba0 0x7fc6a80cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ad9d5640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a815c6f0 0x7fc6a80b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 -- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a815c6f0 msgr2=0x7fc6a80b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a815c6f0 0x7fc6a80b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 -- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a80ccba0 msgr2=0x7fc6a80cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a80ccba0 0x7fc6a80cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 -- 172.21.15.22:0/3150213388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a81f4340 con 0x7fc6a80d1d40 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ad9d5640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a815c6f0 0x7fc6a80b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6ac9d3640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a8158180 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c970 tx=0x7fc69c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c007bb0 con 0x7fc6a80d1d40 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc69c007e40 con 0x7fc6a80d1d40 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a80ca590 con 0x7fc6a80d1d40 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.499+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69c00f690 con 0x7fc6a80d1d40 2024-09-21T05:44:08.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.500+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a80cab50 con 0x7fc6a80d1d40 2024-09-21T05:44:08.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.500+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a80d1870 con 0x7fc6a80d1d40 2024-09-21T05:44:08.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.501+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc69c00f830 con 0x7fc6a80d1d40 2024-09-21T05:44:08.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.502+0000 7fc6a5ffb640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc694078bc0 0x7fc69407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.502+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fc69c09b4e0 con 0x7fc6a80d1d40 2024-09-21T05:44:08.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.502+0000 7fc6ad1d4640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc694078bc0 0x7fc69407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.503+0000 7fc6ad1d4640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc694078bc0 0x7fc69407b0d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc698034000 tx=0x7fc698004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:08.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.506+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc69c014030 con 0x7fc6a80d1d40 2024-09-21T05:44:08.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.730+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6a80b3a40 con 0x7fc6a80d1d40 2024-09-21T05:44:08.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.730+0000 7fc6a5ffb640 1 -- 172.21.15.22:0/3150213388 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5225 (secure 0 0 0) 0x7fc69c0639b0 con 0x7fc6a80d1d40 2024-09-21T05:44:08.732 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:08.733 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":62,"btime":"2024-09-21T05:44:08:349505+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:07.731699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/2104741994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2104741994},{"type":"v1","addr":"172.21.15.134:6837","nonce":2104741994}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6720,"qdb_cluster":[6720]},"id":6}]} 2024-09-21T05:44:08.733 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 62 2024-09-21T05:44:08.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc694078bc0 msgr2=0x7fc69407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc694078bc0 0x7fc69407b0d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc698034000 tx=0x7fc698004ab0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 msgr2=0x7fc6a8158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a8158180 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00c970 tx=0x7fc69c00ce30 comp rx=0 tx=0).stop 2024-09-21T05:44:08.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 shutdown_connections 2024-09-21T05:44:08.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc694078bc0 0x7fc69407b0d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc6a815c6f0 0x7fc6a80b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc6a80d1d40 0x7fc6a8158180 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 --2- 172.21.15.22:0/3150213388 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc6a80ccba0 0x7fc6a80cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.732+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 >> 172.21.15.22:0/3150213388 conn(0x7fc6a80b3f10 msgr2=0x7fc6a80b7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.733+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 shutdown_connections 2024-09-21T05:44:08.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.733+0000 7fc6af438640 1 -- 172.21.15.22:0/3150213388 wait complete. 2024-09-21T05:44:08.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:44:08.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.859+0000 7feb7e5e6640 1 -- 172.21.15.22:0/3043910232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 msgr2=0x7feb780d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.859+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/3043910232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780d0190 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feb70005fd0 tx=0x7feb7002a820 comp rx=0 tx=0).stop 2024-09-21T05:44:08.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.859+0000 7feb7e5e6640 1 -- 172.21.15.22:0/3043910232 shutdown_connections 2024-09-21T05:44:08.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.859+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/3043910232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780d0190 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.859+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/3043910232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb780cf440 0x7feb7815c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.859+0000 7feb7e5e6640 1 -- 172.21.15.22:0/3043910232 >> 172.21.15.22:0/3043910232 conn(0x7feb780710b0 msgr2=0x7feb780714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:08.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.860+0000 7feb7e5e6640 1 -- 172.21.15.22:0/3043910232 shutdown_connections 2024-09-21T05:44:08.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.860+0000 7feb7e5e6640 1 -- 172.21.15.22:0/3043910232 wait complete. 2024-09-21T05:44:08.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.861+0000 7feb7e5e6640 1 Processor -- start 2024-09-21T05:44:08.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.861+0000 7feb7e5e6640 1 -- start start 2024-09-21T05:44:08.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7e5e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7e5e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb780cd360 0x7feb78158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7e5e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb781f66a0 0x7feb780b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7e5e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7feb780d0c00 con 0x7feb780b7fe0 2024-09-21T05:44:08.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7e5e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7feb780d0a80 con 0x7feb780cd360 2024-09-21T05:44:08.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7e5e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7feb780d0d80 con 0x7feb781f66a0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36332/0 (socket says 172.21.15.22:36332) 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 -- 172.21.15.22:0/519189686 learned_addr learned my addr 172.21.15.22:0/519189686 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb7cb83640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb781f66a0 0x7feb780b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 -- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb781f66a0 msgr2=0x7feb780b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb781f66a0 0x7feb780b6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 -- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb780cd360 msgr2=0x7feb78158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb780cd360 0x7feb78158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 -- 172.21.15.22:0/519189686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7815cef0 con 0x7feb780b7fe0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb77fff640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780cb1f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c3a0 tx=0x7feb6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.863+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68011790 con 0x7feb780b7fe0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.864+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb780b7fe0 2024-09-21T05:44:08.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.864+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68011cc0 con 0x7feb780b7fe0 2024-09-21T05:44:08.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.864+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb781f6e50 con 0x7feb780b7fe0 2024-09-21T05:44:08.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.864+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781f7410 con 0x7feb780b7fe0 2024-09-21T05:44:08.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.865+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7815d0c0 con 0x7feb780b7fe0 2024-09-21T05:44:08.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.865+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feb6801c460 con 0x7feb780b7fe0 2024-09-21T05:44:08.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.866+0000 7feb74ff9640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feb64078b90 0x7feb6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:08.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.866+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7feb6809abd0 con 0x7feb780b7fe0 2024-09-21T05:44:08.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.866+0000 7feb777fe640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feb64078b90 0x7feb6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:08.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.867+0000 7feb777fe640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feb64078b90 0x7feb6407b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0058c0 tx=0x7feb6c005850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:08.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:08.870+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb68062f70 con 0x7feb780b7fe0 2024-09-21T05:44:09.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.058+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb780d12a0 con 0x7feb780b7fe0 2024-09-21T05:44:09.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.059+0000 7feb74ff9640 1 -- 172.21.15.22:0/519189686 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16708 (secure 0 0 0) 0x7feb68062870 con 0x7feb780b7fe0 2024-09-21T05:44:09.062 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:09.062 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":82,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:44:06.335835+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:44:02.389731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T05:44:03.675773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:44:09.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feb64078b90 msgr2=0x7feb6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feb64078b90 0x7feb6407b0a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0058c0 tx=0x7feb6c005850 comp rx=0 tx=0).stop 2024-09-21T05:44:09.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 msgr2=0x7feb780cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780cb1f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feb6800c3a0 tx=0x7feb6800c860 comp rx=0 tx=0).stop 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 shutdown_connections 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7feb64078b90 0x7feb6407b0a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7feb781f66a0 0x7feb780b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7feb780cd360 0x7feb78158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 --2- 172.21.15.22:0/519189686 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7feb780b7fe0 0x7feb780cb1f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 >> 172.21.15.22:0/519189686 conn(0x7feb780710b0 msgr2=0x7feb7815a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 shutdown_connections 2024-09-21T05:44:09.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.062+0000 7feb7e5e6640 1 -- 172.21.15.22:0/519189686 wait complete. 2024-09-21T05:44:09.075 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:44:09.076 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:44:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 -- 172.21.15.22:0/1892534217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244070440 msgr2=0x7f1244074b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 --2- 172.21.15.22:0/1892534217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244070440 0x7f1244074b80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1244070e70 tx=0x7f123c02a980 comp rx=0 tx=0).stop 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 -- 172.21.15.22:0/1892534217 shutdown_connections 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 --2- 172.21.15.22:0/1892534217 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244070440 0x7f1244074b80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 --2- 172.21.15.22:0/1892534217 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1244070030 0x7f1244074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 --2- 172.21.15.22:0/1892534217 >> v2:172.21.15.134:3300/0 conn(0x7f1244073170 0x7f124415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 -- 172.21.15.22:0/1892534217 >> 172.21.15.22:0/1892534217 conn(0x7f12440b1cb0 msgr2=0x7f12440b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 -- 172.21.15.22:0/1892534217 shutdown_connections 2024-09-21T05:44:09.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.190+0000 7f124b569640 1 -- 172.21.15.22:0/1892534217 wait complete. 2024-09-21T05:44:09.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.192+0000 7f124b569640 1 Processor -- start 2024-09-21T05:44:09.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.192+0000 7f124b569640 1 -- start start 2024-09-21T05:44:09.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f124b569640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1244070030 0x7f124414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f124b569640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1244070440 0x7f12440ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f124b569640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 0x7f124414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f124b569640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f124406f690 con 0x7f1244073170 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f124b569640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f124406f510 con 0x7f1244070030 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f124b569640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f124406f810 con 0x7f1244070440 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f1249b06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 0x7f124414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f1249b06640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 0x7f124414f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36346/0 (socket says 172.21.15.22:36346) 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f1249b06640 1 -- 172.21.15.22:0/2626233816 learned_addr learned my addr 172.21.15.22:0/2626233816 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f1249305640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1244070030 0x7f124414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.193+0000 7f1248b04640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1244070440 0x7f12440ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1249b06640 1 -- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1244070440 msgr2=0x7f12440ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1249b06640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1244070440 0x7f12440ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1249b06640 1 -- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1244070030 msgr2=0x7f124414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1249b06640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1244070030 0x7f124414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1249b06640 1 -- 172.21.15.22:0/2626233816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12440cacb0 con 0x7f1244073170 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1249b06640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 0x7f124414f740 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f123c009d60 tx=0x7f123c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c034040 con 0x7f1244073170 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f123c004540 con 0x7f1244073170 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12440cb130 con 0x7f1244073170 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123c045480 con 0x7f1244073170 2024-09-21T05:44:09.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.194+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12440cb6f0 con 0x7f1244073170 2024-09-21T05:44:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.195+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12440cae80 con 0x7f1244073170 2024-09-21T05:44:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.196+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f123c037020 con 0x7f1244073170 2024-09-21T05:44:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.197+0000 7f1239ffb640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1228078b90 0x7f122807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.197+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f123c0bc540 con 0x7f1244073170 2024-09-21T05:44:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.197+0000 7f1249305640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1228078b90 0x7f122807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.197+0000 7f1249305640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1228078b90 0x7f122807b0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f124000bad0 tx=0x7f124000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:09.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.201+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f123c0842a0 con 0x7f1244073170 2024-09-21T05:44:09.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.389+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f124415c780 con 0x7f1228078b90 2024-09-21T05:44:09.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.394+0000 7f1239ffb640 1 -- 172.21.15.22:0/2626233816 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7f124415c780 con 0x7f1228078b90 2024-09-21T05:44:09.399 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:09.400 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1228078b90 msgr2=0x7f122807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1228078b90 0x7f122807b0a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f124000bad0 tx=0x7f124000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 msgr2=0x7f124414f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 0x7f124414f740 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f123c009d60 tx=0x7f123c003f90 comp rx=0 tx=0).stop 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 shutdown_connections 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1244073170 0x7f124414f740 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1228078b90 0x7f122807b0a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1244070440 0x7f12440ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 --2- 172.21.15.22:0/2626233816 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1244070030 0x7f124414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 >> 172.21.15.22:0/2626233816 conn(0x7f12440b1cb0 msgr2=0x7f12440aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 shutdown_connections 2024-09-21T05:44:09.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.397+0000 7f124b569640 1 -- 172.21.15.22:0/2626233816 wait complete. 2024-09-21T05:44:09.410 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-21T05:44:07.811988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266660,"kb_used_data":8820,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708060,"statfs":{"total":773094113280,"available":772821053440,"internally_reserved":0,"allocated":9031680,"data_stored":4502925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.492390"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353933+0000","last_change":"2024-09-21T05:44:05.349009+0000","last_active":"2024-09-21T05:44:05.353933+0000","last_peered":"2024-09-21T05:44:05.353933+0000","last_clean":"2024-09-21T05:44:05.353933+0000","last_became_active":"2024-09-21T05:44:05.348684+0000","last_became_peered":"2024-09-21T05:44:05.348684+0000","last_unstale":"2024-09-21T05:44:05.353933+0000","last_undegraded":"2024-09-21T05:44:05.353933+0000","last_fullsized":"2024-09-21T05:44:05.353933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:38.595488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.359310+0000","last_change":"2024-09-21T05:44:05.338790+0000","last_active":"2024-09-21T05:44:05.359310+0000","last_peered":"2024-09-21T05:44:05.359310+0000","last_clean":"2024-09-21T05:44:05.359310+0000","last_became_active":"2024-09-21T05:44:05.338484+0000","last_became_peered":"2024-09-21T05:44:05.338484+0000","last_unstale":"2024-09-21T05:44:05.359310+0000","last_undegraded":"2024-09-21T05:44:05.359310+0000","last_fullsized":"2024-09-21T05:44:05.359310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:09.907739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358994+0000","last_change":"2024-09-21T05:44:05.353303+0000","last_active":"2024-09-21T05:44:05.358994+0000","last_peered":"2024-09-21T05:44:05.358994+0000","last_clean":"2024-09-21T05:44:05.358994+0000","last_became_active":"2024-09-21T05:44:05.352894+0000","last_became_peered":"2024-09-21T05:44:05.352894+0000","last_unstale":"2024-09-21T05:44:05.358994+0000","last_undegraded":"2024-09-21T05:44:05.358994+0000","last_fullsized":"2024-09-21T05:44:05.358994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:42.478156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352963+0000","last_change":"2024-09-21T05:44:05.345417+0000","last_active":"2024-09-21T05:44:05.352963+0000","last_peered":"2024-09-21T05:44:05.352963+0000","last_clean":"2024-09-21T05:44:05.352963+0000","last_became_active":"2024-09-21T05:44:05.344787+0000","last_became_peered":"2024-09-21T05:44:05.344787+0000","last_unstale":"2024-09-21T05:44:05.352963+0000","last_undegraded":"2024-09-21T05:44:05.352963+0000","last_fullsized":"2024-09-21T05:44:05.352963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:54.900091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344851+0000","last_change":"2024-09-21T05:44:05.346688+0000","last_active":"2024-09-21T05:44:06.344851+0000","last_peered":"2024-09-21T05:44:06.344851+0000","last_clean":"2024-09-21T05:44:06.344851+0000","last_became_active":"2024-09-21T05:44:05.346384+0000","last_became_peered":"2024-09-21T05:44:05.346384+0000","last_unstale":"2024-09-21T05:44:06.344851+0000","last_undegraded":"2024-09-21T05:44:06.344851+0000","last_fullsized":"2024-09-21T05:44:06.344851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:40.020576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355127+0000","last_change":"2024-09-21T05:44:05.350307+0000","last_active":"2024-09-21T05:44:05.355127+0000","last_peered":"2024-09-21T05:44:05.355127+0000","last_clean":"2024-09-21T05:44:05.355127+0000","last_became_active":"2024-09-21T05:44:05.349838+0000","last_became_peered":"2024-09-21T05:44:05.349838+0000","last_unstale":"2024-09-21T05:44:05.355127+0000","last_undegraded":"2024-09-21T05:44:05.355127+0000","last_fullsized":"2024-09-21T05:44:05.355127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:06.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353162+0000","last_change":"2024-09-21T05:44:05.346679+0000","last_active":"2024-09-21T05:44:05.353162+0000","last_peered":"2024-09-21T05:44:05.353162+0000","last_clean":"2024-09-21T05:44:05.353162+0000","last_became_active":"2024-09-21T05:44:05.346365+0000","last_became_peered":"2024-09-21T05:44:05.346365+0000","last_unstale":"2024-09-21T05:44:05.353162+0000","last_undegraded":"2024-09-21T05:44:05.353162+0000","last_fullsized":"2024-09-21T05:44:05.353162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:29.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361481+0000","last_change":"2024-09-21T05:44:05.354937+0000","last_active":"2024-09-21T05:44:05.361481+0000","last_peered":"2024-09-21T05:44:05.361481+0000","last_clean":"2024-09-21T05:44:05.361481+0000","last_became_active":"2024-09-21T05:44:05.354632+0000","last_became_peered":"2024-09-21T05:44:05.354632+0000","last_unstale":"2024-09-21T05:44:05.361481+0000","last_undegraded":"2024-09-21T05:44:05.361481+0000","last_fullsized":"2024-09-21T05:44:05.361481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:38.677586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355935+0000","last_change":"2024-09-21T05:44:05.351441+0000","last_active":"2024-09-21T05:44:05.355935+0000","last_peered":"2024-09-21T05:44:05.355935+0000","last_clean":"2024-09-21T05:44:05.355935+0000","last_became_active":"2024-09-21T05:44:05.351008+0000","last_became_peered":"2024-09-21T05:44:05.351008+0000","last_unstale":"2024-09-21T05:44:05.355935+0000","last_undegraded":"2024-09-21T05:44:05.355935+0000","last_fullsized":"2024-09-21T05:44:05.355935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:44.610721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.349724+0000","last_change":"2024-09-21T05:44:05.345624+0000","last_active":"2024-09-21T05:44:05.349724+0000","last_peered":"2024-09-21T05:44:05.349724+0000","last_clean":"2024-09-21T05:44:05.349724+0000","last_became_active":"2024-09-21T05:44:05.345177+0000","last_became_peered":"2024-09-21T05:44:05.345177+0000","last_unstale":"2024-09-21T05:44:05.349724+0000","last_undegraded":"2024-09-21T05:44:05.349724+0000","last_fullsized":"2024-09-21T05:44:05.349724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:13.813363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361064+0000","last_change":"2024-09-21T05:44:05.355053+0000","last_active":"2024-09-21T05:44:05.361064+0000","last_peered":"2024-09-21T05:44:05.361064+0000","last_clean":"2024-09-21T05:44:05.361064+0000","last_became_active":"2024-09-21T05:44:05.354482+0000","last_became_peered":"2024-09-21T05:44:05.354482+0000","last_unstale":"2024-09-21T05:44:05.361064+0000","last_undegraded":"2024-09-21T05:44:05.361064+0000","last_fullsized":"2024-09-21T05:44:05.361064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:10.035432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353876+0000","last_change":"2024-09-21T05:44:05.345959+0000","last_active":"2024-09-21T05:44:05.353876+0000","last_peered":"2024-09-21T05:44:05.353876+0000","last_clean":"2024-09-21T05:44:05.353876+0000","last_became_active":"2024-09-21T05:44:05.345647+0000","last_became_peered":"2024-09-21T05:44:05.345647+0000","last_unstale":"2024-09-21T05:44:05.353876+0000","last_undegraded":"2024-09-21T05:44:05.353876+0000","last_fullsized":"2024-09-21T05:44:05.353876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:06.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358986+0000","last_change":"2024-09-21T05:44:05.354156+0000","last_active":"2024-09-21T05:44:05.358986+0000","last_peered":"2024-09-21T05:44:05.358986+0000","last_clean":"2024-09-21T05:44:05.358986+0000","last_became_active":"2024-09-21T05:44:05.353827+0000","last_became_peered":"2024-09-21T05:44:05.353827+0000","last_unstale":"2024-09-21T05:44:05.358986+0000","last_undegraded":"2024-09-21T05:44:05.358986+0000","last_fullsized":"2024-09-21T05:44:05.358986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:25.027057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352929+0000","last_change":"2024-09-21T05:44:05.346007+0000","last_active":"2024-09-21T05:44:05.352929+0000","last_peered":"2024-09-21T05:44:05.352929+0000","last_clean":"2024-09-21T05:44:05.352929+0000","last_became_active":"2024-09-21T05:44:05.344913+0000","last_became_peered":"2024-09-21T05:44:05.344913+0000","last_unstale":"2024-09-21T05:44:05.352929+0000","last_undegraded":"2024-09-21T05:44:05.352929+0000","last_fullsized":"2024-09-21T05:44:05.352929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:04.577555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353460+0000","last_change":"2024-09-21T05:44:05.346788+0000","last_active":"2024-09-21T05:44:05.353460+0000","last_peered":"2024-09-21T05:44:05.353460+0000","last_clean":"2024-09-21T05:44:05.353460+0000","last_became_active":"2024-09-21T05:44:05.346475+0000","last_became_peered":"2024-09-21T05:44:05.346475+0000","last_unstale":"2024-09-21T05:44:05.353460+0000","last_undegraded":"2024-09-21T05:44:05.353460+0000","last_fullsized":"2024-09-21T05:44:05.353460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:41.613302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344263+0000","last_change":"2024-09-21T05:44:05.347529+0000","last_active":"2024-09-21T05:44:06.344263+0000","last_peered":"2024-09-21T05:44:06.344263+0000","last_clean":"2024-09-21T05:44:06.344263+0000","last_became_active":"2024-09-21T05:44:05.347178+0000","last_became_peered":"2024-09-21T05:44:05.347178+0000","last_unstale":"2024-09-21T05:44:06.344263+0000","last_undegraded":"2024-09-21T05:44:06.344263+0000","last_fullsized":"2024-09-21T05:44:06.344263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:42.533823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344713+0000","last_change":"2024-09-21T05:44:05.340682+0000","last_active":"2024-09-21T05:44:06.344713+0000","last_peered":"2024-09-21T05:44:06.344713+0000","last_clean":"2024-09-21T05:44:06.344713+0000","last_became_active":"2024-09-21T05:44:05.340090+0000","last_became_peered":"2024-09-21T05:44:05.340090+0000","last_unstale":"2024-09-21T05:44:06.344713+0000","last_undegraded":"2024-09-21T05:44:06.344713+0000","last_fullsized":"2024-09-21T05:44:06.344713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:42.870502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353010+0000","last_change":"2024-09-21T05:44:05.349019+0000","last_active":"2024-09-21T05:44:05.353010+0000","last_peered":"2024-09-21T05:44:05.353010+0000","last_clean":"2024-09-21T05:44:05.353010+0000","last_became_active":"2024-09-21T05:44:05.348619+0000","last_became_peered":"2024-09-21T05:44:05.348619+0000","last_unstale":"2024-09-21T05:44:05.353010+0000","last_undegraded":"2024-09-21T05:44:05.353010+0000","last_fullsized":"2024-09-21T05:44:05.353010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:15.642288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358516+0000","last_change":"2024-09-21T05:44:05.355517+0000","last_active":"2024-09-21T05:44:05.358516+0000","last_peered":"2024-09-21T05:44:05.358516+0000","last_clean":"2024-09-21T05:44:05.358516+0000","last_became_active":"2024-09-21T05:44:05.355081+0000","last_became_peered":"2024-09-21T05:44:05.355081+0000","last_unstale":"2024-09-21T05:44:05.358516+0000","last_undegraded":"2024-09-21T05:44:05.358516+0000","last_fullsized":"2024-09-21T05:44:05.358516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:31.701087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358655+0000","last_change":"2024-09-21T05:44:05.356583+0000","last_active":"2024-09-21T05:44:05.358655+0000","last_peered":"2024-09-21T05:44:05.358655+0000","last_clean":"2024-09-21T05:44:05.358655+0000","last_became_active":"2024-09-21T05:44:05.356252+0000","last_became_peered":"2024-09-21T05:44:05.356252+0000","last_unstale":"2024-09-21T05:44:05.358655+0000","last_undegraded":"2024-09-21T05:44:05.358655+0000","last_fullsized":"2024-09-21T05:44:05.358655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:28.285040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356627+0000","last_change":"2024-09-21T05:44:05.349964+0000","last_active":"2024-09-21T05:44:05.356627+0000","last_peered":"2024-09-21T05:44:05.356627+0000","last_clean":"2024-09-21T05:44:05.356627+0000","last_became_active":"2024-09-21T05:44:05.349547+0000","last_became_peered":"2024-09-21T05:44:05.349547+0000","last_unstale":"2024-09-21T05:44:05.356627+0000","last_undegraded":"2024-09-21T05:44:05.356627+0000","last_fullsized":"2024-09-21T05:44:05.356627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:08.822727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356308+0000","last_change":"2024-09-21T05:44:05.351112+0000","last_active":"2024-09-21T05:44:05.356308+0000","last_peered":"2024-09-21T05:44:05.356308+0000","last_clean":"2024-09-21T05:44:05.356308+0000","last_became_active":"2024-09-21T05:44:05.350782+0000","last_became_peered":"2024-09-21T05:44:05.350782+0000","last_unstale":"2024-09-21T05:44:05.356308+0000","last_undegraded":"2024-09-21T05:44:05.356308+0000","last_fullsized":"2024-09-21T05:44:05.356308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:55.842648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356769+0000","last_change":"2024-09-21T05:44:05.351172+0000","last_active":"2024-09-21T05:44:05.356769+0000","last_peered":"2024-09-21T05:44:05.356769+0000","last_clean":"2024-09-21T05:44:05.356769+0000","last_became_active":"2024-09-21T05:44:05.350867+0000","last_became_peered":"2024-09-21T05:44:05.350867+0000","last_unstale":"2024-09-21T05:44:05.356769+0000","last_undegraded":"2024-09-21T05:44:05.356769+0000","last_fullsized":"2024-09-21T05:44:05.356769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:38.677586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356742+0000","last_change":"2024-09-21T05:44:05.351193+0000","last_active":"2024-09-21T05:44:05.356742+0000","last_peered":"2024-09-21T05:44:05.356742+0000","last_clean":"2024-09-21T05:44:05.356742+0000","last_became_active":"2024-09-21T05:44:05.350886+0000","last_became_peered":"2024-09-21T05:44:05.350886+0000","last_unstale":"2024-09-21T05:44:05.356742+0000","last_undegraded":"2024-09-21T05:44:05.356742+0000","last_fullsized":"2024-09-21T05:44:05.356742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:12.196197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342205+0000","last_change":"2024-09-21T05:44:05.337141+0000","last_active":"2024-09-21T05:44:05.342205+0000","last_peered":"2024-09-21T05:44:05.342205+0000","last_clean":"2024-09-21T05:44:05.342205+0000","last_became_active":"2024-09-21T05:44:05.336815+0000","last_became_peered":"2024-09-21T05:44:05.336815+0000","last_unstale":"2024-09-21T05:44:05.342205+0000","last_undegraded":"2024-09-21T05:44:05.342205+0000","last_fullsized":"2024-09-21T05:44:05.342205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:22.477887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.364506+0000","last_change":"2024-09-21T05:44:05.361335+0000","last_active":"2024-09-21T05:44:05.364506+0000","last_peered":"2024-09-21T05:44:05.364506+0000","last_clean":"2024-09-21T05:44:05.364506+0000","last_became_active":"2024-09-21T05:44:05.361084+0000","last_became_peered":"2024-09-21T05:44:05.361084+0000","last_unstale":"2024-09-21T05:44:05.364506+0000","last_undegraded":"2024-09-21T05:44:05.364506+0000","last_fullsized":"2024-09-21T05:44:05.364506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:36.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344488+0000","last_change":"2024-09-21T05:44:05.346863+0000","last_active":"2024-09-21T05:44:06.344488+0000","last_peered":"2024-09-21T05:44:06.344488+0000","last_clean":"2024-09-21T05:44:06.344488+0000","last_became_active":"2024-09-21T05:44:05.346535+0000","last_became_peered":"2024-09-21T05:44:05.346535+0000","last_unstale":"2024-09-21T05:44:06.344488+0000","last_undegraded":"2024-09-21T05:44:06.344488+0000","last_fullsized":"2024-09-21T05:44:06.344488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:23.974145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356119+0000","last_change":"2024-09-21T05:44:05.349982+0000","last_active":"2024-09-21T05:44:05.356119+0000","last_peered":"2024-09-21T05:44:05.356119+0000","last_clean":"2024-09-21T05:44:05.356119+0000","last_became_active":"2024-09-21T05:44:05.349586+0000","last_became_peered":"2024-09-21T05:44:05.349586+0000","last_unstale":"2024-09-21T05:44:05.356119+0000","last_undegraded":"2024-09-21T05:44:05.356119+0000","last_fullsized":"2024-09-21T05:44:05.356119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:33.579521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.354545+0000","last_change":"2024-09-21T05:44:05.343070+0000","last_active":"2024-09-21T05:44:05.354545+0000","last_peered":"2024-09-21T05:44:05.354545+0000","last_clean":"2024-09-21T05:44:05.354545+0000","last_became_active":"2024-09-21T05:44:05.342757+0000","last_became_peered":"2024-09-21T05:44:05.342757+0000","last_unstale":"2024-09-21T05:44:05.354545+0000","last_undegraded":"2024-09-21T05:44:05.354545+0000","last_fullsized":"2024-09-21T05:44:05.354545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.192054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355708+0000","last_change":"2024-09-21T05:44:05.346217+0000","last_active":"2024-09-21T05:44:05.355708+0000","last_peered":"2024-09-21T05:44:05.355708+0000","last_clean":"2024-09-21T05:44:05.355708+0000","last_became_active":"2024-09-21T05:44:05.345936+0000","last_became_peered":"2024-09-21T05:44:05.345936+0000","last_unstale":"2024-09-21T05:44:05.355708+0000","last_undegraded":"2024-09-21T05:44:05.355708+0000","last_fullsized":"2024-09-21T05:44:05.355708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:48.479671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344231+0000","last_change":"2024-09-21T05:44:05.351496+0000","last_active":"2024-09-21T05:44:06.344231+0000","last_peered":"2024-09-21T05:44:06.344231+0000","last_clean":"2024-09-21T05:44:06.344231+0000","last_became_active":"2024-09-21T05:44:05.351142+0000","last_became_peered":"2024-09-21T05:44:05.351142+0000","last_unstale":"2024-09-21T05:44:06.344231+0000","last_undegraded":"2024-09-21T05:44:06.344231+0000","last_fullsized":"2024-09-21T05:44:06.344231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:40.154352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.359410+0000","last_change":"2024-09-21T05:44:05.355633+0000","last_active":"2024-09-21T05:44:05.359410+0000","last_peered":"2024-09-21T05:44:05.359410+0000","last_clean":"2024-09-21T05:44:05.359410+0000","last_became_active":"2024-09-21T05:44:05.354585+0000","last_became_peered":"2024-09-21T05:44:05.354585+0000","last_unstale":"2024-09-21T05:44:05.359410+0000","last_undegraded":"2024-09-21T05:44:05.359410+0000","last_fullsized":"2024-09-21T05:44:05.359410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:12.939705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344987+0000","last_change":"2024-09-21T05:44:05.347900+0000","last_active":"2024-09-21T05:44:06.344987+0000","last_peered":"2024-09-21T05:44:06.344987+0000","last_clean":"2024-09-21T05:44:06.344987+0000","last_became_active":"2024-09-21T05:44:05.347597+0000","last_became_peered":"2024-09-21T05:44:05.347597+0000","last_unstale":"2024-09-21T05:44:06.344987+0000","last_undegraded":"2024-09-21T05:44:06.344987+0000","last_fullsized":"2024-09-21T05:44:06.344987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:54.890455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358782+0000","last_change":"2024-09-21T05:44:04.340613+0000","last_active":"2024-09-21T05:44:05.358782+0000","last_peered":"2024-09-21T05:44:05.358782+0000","last_clean":"2024-09-21T05:44:05.358782+0000","last_became_active":"2024-09-21T05:44:04.340249+0000","last_became_peered":"2024-09-21T05:44:04.340249+0000","last_unstale":"2024-09-21T05:44:05.358782+0000","last_undegraded":"2024-09-21T05:44:05.358782+0000","last_fullsized":"2024-09-21T05:44:05.358782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:55.434596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356980+0000","last_change":"2024-09-21T05:44:05.351385+0000","last_active":"2024-09-21T05:44:05.356980+0000","last_peered":"2024-09-21T05:44:05.356980+0000","last_clean":"2024-09-21T05:44:05.356980+0000","last_became_active":"2024-09-21T05:44:05.350942+0000","last_became_peered":"2024-09-21T05:44:05.350942+0000","last_unstale":"2024-09-21T05:44:05.356980+0000","last_undegraded":"2024-09-21T05:44:05.356980+0000","last_fullsized":"2024-09-21T05:44:05.356980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:44.944845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344924+0000","last_change":"2024-09-21T05:44:04.341819+0000","last_active":"2024-09-21T05:44:06.344924+0000","last_peered":"2024-09-21T05:44:06.344924+0000","last_clean":"2024-09-21T05:44:06.344924+0000","last_became_active":"2024-09-21T05:44:04.339841+0000","last_became_peered":"2024-09-21T05:44:04.339841+0000","last_unstale":"2024-09-21T05:44:06.344924+0000","last_undegraded":"2024-09-21T05:44:06.344924+0000","last_fullsized":"2024-09-21T05:44:06.344924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:24.773340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.350208+0000","last_change":"2024-09-21T05:44:05.345013+0000","last_active":"2024-09-21T05:44:05.350208+0000","last_peered":"2024-09-21T05:44:05.350208+0000","last_clean":"2024-09-21T05:44:05.350208+0000","last_became_active":"2024-09-21T05:44:05.344705+0000","last_became_peered":"2024-09-21T05:44:05.344705+0000","last_unstale":"2024-09-21T05:44:05.350208+0000","last_undegraded":"2024-09-21T05:44:05.350208+0000","last_fullsized":"2024-09-21T05:44:05.350208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:19.294975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358435+0000","last_change":"2024-09-21T05:44:04.348855+0000","last_active":"2024-09-21T05:44:05.358435+0000","last_peered":"2024-09-21T05:44:05.358435+0000","last_clean":"2024-09-21T05:44:05.358435+0000","last_became_active":"2024-09-21T05:44:04.347770+0000","last_became_peered":"2024-09-21T05:44:04.347770+0000","last_unstale":"2024-09-21T05:44:05.358435+0000","last_undegraded":"2024-09-21T05:44:05.358435+0000","last_fullsized":"2024-09-21T05:44:05.358435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:28.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361780+0000","last_change":"2024-09-21T05:44:05.354889+0000","last_active":"2024-09-21T05:44:05.361780+0000","last_peered":"2024-09-21T05:44:05.361780+0000","last_clean":"2024-09-21T05:44:05.361780+0000","last_became_active":"2024-09-21T05:44:05.354585+0000","last_became_peered":"2024-09-21T05:44:05.354585+0000","last_unstale":"2024-09-21T05:44:05.361780+0000","last_undegraded":"2024-09-21T05:44:05.361780+0000","last_fullsized":"2024-09-21T05:44:05.361780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:40.154352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358973+0000","last_change":"2024-09-21T05:44:04.348039+0000","last_active":"2024-09-21T05:44:05.358973+0000","last_peered":"2024-09-21T05:44:05.358973+0000","last_clean":"2024-09-21T05:44:05.358973+0000","last_became_active":"2024-09-21T05:44:04.347671+0000","last_became_peered":"2024-09-21T05:44:04.347671+0000","last_unstale":"2024-09-21T05:44:05.358973+0000","last_undegraded":"2024-09-21T05:44:05.358973+0000","last_fullsized":"2024-09-21T05:44:05.358973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:11.937258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355675+0000","last_change":"2024-09-21T05:44:05.345429+0000","last_active":"2024-09-21T05:44:05.355675+0000","last_peered":"2024-09-21T05:44:05.355675+0000","last_clean":"2024-09-21T05:44:05.355675+0000","last_became_active":"2024-09-21T05:44:05.345143+0000","last_became_peered":"2024-09-21T05:44:05.345143+0000","last_unstale":"2024-09-21T05:44:05.355675+0000","last_undegraded":"2024-09-21T05:44:05.355675+0000","last_fullsized":"2024-09-21T05:44:05.355675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:09.786674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.345254+0000","last_change":"2024-09-21T05:44:04.348145+0000","last_active":"2024-09-21T05:44:06.345254+0000","last_peered":"2024-09-21T05:44:06.345254+0000","last_clean":"2024-09-21T05:44:06.345254+0000","last_became_active":"2024-09-21T05:44:04.347611+0000","last_became_peered":"2024-09-21T05:44:04.347611+0000","last_unstale":"2024-09-21T05:44:06.345254+0000","last_undegraded":"2024-09-21T05:44:06.345254+0000","last_fullsized":"2024-09-21T05:44:06.345254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:12.180410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.345121+0000","last_change":"2024-09-21T05:44:05.343853+0000","last_active":"2024-09-21T05:44:06.345121+0000","last_peered":"2024-09-21T05:44:06.345121+0000","last_clean":"2024-09-21T05:44:06.345121+0000","last_became_active":"2024-09-21T05:44:05.343546+0000","last_became_peered":"2024-09-21T05:44:05.343546+0000","last_unstale":"2024-09-21T05:44:06.345121+0000","last_undegraded":"2024-09-21T05:44:06.345121+0000","last_fullsized":"2024-09-21T05:44:06.345121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:54.420097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344440+0000","last_change":"2024-09-21T05:44:04.341770+0000","last_active":"2024-09-21T05:44:06.344440+0000","last_peered":"2024-09-21T05:44:06.344440+0000","last_clean":"2024-09-21T05:44:06.344440+0000","last_became_active":"2024-09-21T05:44:04.339648+0000","last_became_peered":"2024-09-21T05:44:04.339648+0000","last_unstale":"2024-09-21T05:44:06.344440+0000","last_undegraded":"2024-09-21T05:44:06.344440+0000","last_fullsized":"2024-09-21T05:44:06.344440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:52.442075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361744+0000","last_change":"2024-09-21T05:44:05.355168+0000","last_active":"2024-09-21T05:44:05.361744+0000","last_peered":"2024-09-21T05:44:05.361744+0000","last_clean":"2024-09-21T05:44:05.361744+0000","last_became_active":"2024-09-21T05:44:05.354808+0000","last_became_peered":"2024-09-21T05:44:05.354808+0000","last_unstale":"2024-09-21T05:44:05.361744+0000","last_undegraded":"2024-09-21T05:44:05.361744+0000","last_fullsized":"2024-09-21T05:44:05.361744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:27.630528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334485+0000","last_change":"2024-09-21T05:44:04.350597+0000","last_active":"2024-09-21T05:44:05.334485+0000","last_peered":"2024-09-21T05:44:05.334485+0000","last_clean":"2024-09-21T05:44:05.334485+0000","last_became_active":"2024-09-21T05:44:04.350315+0000","last_became_peered":"2024-09-21T05:44:04.350315+0000","last_unstale":"2024-09-21T05:44:05.334485+0000","last_undegraded":"2024-09-21T05:44:05.334485+0000","last_fullsized":"2024-09-21T05:44:05.334485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:32.577075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353422+0000","last_change":"2024-09-21T05:44:05.345901+0000","last_active":"2024-09-21T05:44:05.353422+0000","last_peered":"2024-09-21T05:44:05.353422+0000","last_clean":"2024-09-21T05:44:05.353422+0000","last_became_active":"2024-09-21T05:44:05.345539+0000","last_became_peered":"2024-09-21T05:44:05.345539+0000","last_unstale":"2024-09-21T05:44:05.353422+0000","last_undegraded":"2024-09-21T05:44:05.353422+0000","last_fullsized":"2024-09-21T05:44:05.353422+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:27.185130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.328438+0000","last_change":"2024-09-21T05:44:04.337933+0000","last_active":"2024-09-21T05:44:05.328438+0000","last_peered":"2024-09-21T05:44:05.328438+0000","last_clean":"2024-09-21T05:44:05.328438+0000","last_became_active":"2024-09-21T05:44:04.337601+0000","last_became_peered":"2024-09-21T05:44:04.337601+0000","last_unstale":"2024-09-21T05:44:05.328438+0000","last_undegraded":"2024-09-21T05:44:05.328438+0000","last_fullsized":"2024-09-21T05:44:05.328438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:42.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342619+0000","last_change":"2024-09-21T05:44:05.341219+0000","last_active":"2024-09-21T05:44:05.342619+0000","last_peered":"2024-09-21T05:44:05.342619+0000","last_clean":"2024-09-21T05:44:05.342619+0000","last_became_active":"2024-09-21T05:44:05.340194+0000","last_became_peered":"2024-09-21T05:44:05.340194+0000","last_unstale":"2024-09-21T05:44:05.342619+0000","last_undegraded":"2024-09-21T05:44:05.342619+0000","last_fullsized":"2024-09-21T05:44:05.342619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:44.720004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.335763+0000","last_change":"2024-09-21T05:44:04.343910+0000","last_active":"2024-09-21T05:44:05.335763+0000","last_peered":"2024-09-21T05:44:05.335763+0000","last_clean":"2024-09-21T05:44:05.335763+0000","last_became_active":"2024-09-21T05:44:04.343564+0000","last_became_peered":"2024-09-21T05:44:04.343564+0000","last_unstale":"2024-09-21T05:44:05.335763+0000","last_undegraded":"2024-09-21T05:44:05.335763+0000","last_fullsized":"2024-09-21T05:44:05.335763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:21.725240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358896+0000","last_change":"2024-09-21T05:44:05.352689+0000","last_active":"2024-09-21T05:44:05.358896+0000","last_peered":"2024-09-21T05:44:05.358896+0000","last_clean":"2024-09-21T05:44:05.358896+0000","last_became_active":"2024-09-21T05:44:05.352326+0000","last_became_peered":"2024-09-21T05:44:05.352326+0000","last_unstale":"2024-09-21T05:44:05.358896+0000","last_undegraded":"2024-09-21T05:44:05.358896+0000","last_fullsized":"2024-09-21T05:44:05.358896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:33.579521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334295+0000","last_change":"2024-09-21T05:44:04.340286+0000","last_active":"2024-09-21T05:44:05.334295+0000","last_peered":"2024-09-21T05:44:05.334295+0000","last_clean":"2024-09-21T05:44:05.334295+0000","last_became_active":"2024-09-21T05:44:04.339041+0000","last_became_peered":"2024-09-21T05:44:04.339041+0000","last_unstale":"2024-09-21T05:44:05.334295+0000","last_undegraded":"2024-09-21T05:44:05.334295+0000","last_fullsized":"2024-09-21T05:44:05.334295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:42.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356934+0000","last_change":"2024-09-21T05:44:05.353220+0000","last_active":"2024-09-21T05:44:05.356934+0000","last_peered":"2024-09-21T05:44:05.356934+0000","last_clean":"2024-09-21T05:44:05.356934+0000","last_became_active":"2024-09-21T05:44:05.352886+0000","last_became_peered":"2024-09-21T05:44:05.352886+0000","last_unstale":"2024-09-21T05:44:05.356934+0000","last_undegraded":"2024-09-21T05:44:05.356934+0000","last_fullsized":"2024-09-21T05:44:05.356934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:21.308548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362627+0000","last_change":"2024-09-21T05:44:04.342388+0000","last_active":"2024-09-21T05:44:05.362627+0000","last_peered":"2024-09-21T05:44:05.362627+0000","last_clean":"2024-09-21T05:44:05.362627+0000","last_became_active":"2024-09-21T05:44:04.341840+0000","last_became_peered":"2024-09-21T05:44:04.341840+0000","last_unstale":"2024-09-21T05:44:05.362627+0000","last_undegraded":"2024-09-21T05:44:05.362627+0000","last_fullsized":"2024-09-21T05:44:05.362627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:13.602788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344698+0000","last_change":"2024-09-21T05:44:05.357914+0000","last_active":"2024-09-21T05:44:06.344698+0000","last_peered":"2024-09-21T05:44:06.344698+0000","last_clean":"2024-09-21T05:44:06.344698+0000","last_became_active":"2024-09-21T05:44:05.357600+0000","last_became_peered":"2024-09-21T05:44:05.357600+0000","last_unstale":"2024-09-21T05:44:06.344698+0000","last_undegraded":"2024-09-21T05:44:06.344698+0000","last_fullsized":"2024-09-21T05:44:06.344698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:09.074958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361530+0000","last_change":"2024-09-21T05:44:04.342115+0000","last_active":"2024-09-21T05:44:05.361530+0000","last_peered":"2024-09-21T05:44:05.361530+0000","last_clean":"2024-09-21T05:44:05.361530+0000","last_became_active":"2024-09-21T05:44:04.341803+0000","last_became_peered":"2024-09-21T05:44:04.341803+0000","last_unstale":"2024-09-21T05:44:05.361530+0000","last_undegraded":"2024-09-21T05:44:05.361530+0000","last_fullsized":"2024-09-21T05:44:05.361530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:12.356953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352577+0000","last_change":"2024-09-21T05:44:05.349982+0000","last_active":"2024-09-21T05:44:05.352577+0000","last_peered":"2024-09-21T05:44:05.352577+0000","last_clean":"2024-09-21T05:44:05.352577+0000","last_became_active":"2024-09-21T05:44:05.349667+0000","last_became_peered":"2024-09-21T05:44:05.349667+0000","last_unstale":"2024-09-21T05:44:05.352577+0000","last_undegraded":"2024-09-21T05:44:05.352577+0000","last_fullsized":"2024-09-21T05:44:05.352577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:06.256673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360966+0000","last_change":"2024-09-21T05:44:04.348660+0000","last_active":"2024-09-21T05:44:05.360966+0000","last_peered":"2024-09-21T05:44:05.360966+0000","last_clean":"2024-09-21T05:44:05.360966+0000","last_became_active":"2024-09-21T05:44:04.348188+0000","last_became_peered":"2024-09-21T05:44:04.348188+0000","last_unstale":"2024-09-21T05:44:05.360966+0000","last_undegraded":"2024-09-21T05:44:05.360966+0000","last_fullsized":"2024-09-21T05:44:05.360966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:30.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361858+0000","last_change":"2024-09-21T05:44:05.358289+0000","last_active":"2024-09-21T05:44:05.361858+0000","last_peered":"2024-09-21T05:44:05.361858+0000","last_clean":"2024-09-21T05:44:05.361858+0000","last_became_active":"2024-09-21T05:44:05.357967+0000","last_became_peered":"2024-09-21T05:44:05.357967+0000","last_unstale":"2024-09-21T05:44:05.361858+0000","last_undegraded":"2024-09-21T05:44:05.361858+0000","last_fullsized":"2024-09-21T05:44:05.361858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:23.974145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334363+0000","last_change":"2024-09-21T05:44:04.345322+0000","last_active":"2024-09-21T05:44:05.334363+0000","last_peered":"2024-09-21T05:44:05.334363+0000","last_clean":"2024-09-21T05:44:05.334363+0000","last_became_active":"2024-09-21T05:44:04.344943+0000","last_became_peered":"2024-09-21T05:44:04.344943+0000","last_unstale":"2024-09-21T05:44:05.334363+0000","last_undegraded":"2024-09-21T05:44:05.334363+0000","last_fullsized":"2024-09-21T05:44:05.334363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:40.582348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360680+0000","last_change":"2024-09-21T05:44:05.344618+0000","last_active":"2024-09-21T05:44:05.360680+0000","last_peered":"2024-09-21T05:44:05.360680+0000","last_clean":"2024-09-21T05:44:05.360680+0000","last_became_active":"2024-09-21T05:44:05.344190+0000","last_became_peered":"2024-09-21T05:44:05.344190+0000","last_unstale":"2024-09-21T05:44:05.360680+0000","last_undegraded":"2024-09-21T05:44:05.360680+0000","last_fullsized":"2024-09-21T05:44:05.360680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.626250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.339199+0000","last_change":"2024-09-21T05:44:04.340367+0000","last_active":"2024-09-21T05:44:05.339199+0000","last_peered":"2024-09-21T05:44:05.339199+0000","last_clean":"2024-09-21T05:44:05.339199+0000","last_became_active":"2024-09-21T05:44:04.339997+0000","last_became_peered":"2024-09-21T05:44:04.339997+0000","last_unstale":"2024-09-21T05:44:05.339199+0000","last_undegraded":"2024-09-21T05:44:05.339199+0000","last_fullsized":"2024-09-21T05:44:05.339199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:07.581968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342160+0000","last_change":"2024-09-21T05:44:05.337922+0000","last_active":"2024-09-21T05:44:05.342160+0000","last_peered":"2024-09-21T05:44:05.342160+0000","last_clean":"2024-09-21T05:44:05.342160+0000","last_became_active":"2024-09-21T05:44:05.337625+0000","last_became_peered":"2024-09-21T05:44:05.337625+0000","last_unstale":"2024-09-21T05:44:05.342160+0000","last_undegraded":"2024-09-21T05:44:05.342160+0000","last_fullsized":"2024-09-21T05:44:05.342160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:22.850598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360581+0000","last_change":"2024-09-21T05:44:04.351389+0000","last_active":"2024-09-21T05:44:05.360581+0000","last_peered":"2024-09-21T05:44:05.360581+0000","last_clean":"2024-09-21T05:44:05.360581+0000","last_became_active":"2024-09-21T05:44:04.351055+0000","last_became_peered":"2024-09-21T05:44:04.351055+0000","last_unstale":"2024-09-21T05:44:05.360581+0000","last_undegraded":"2024-09-21T05:44:05.360581+0000","last_fullsized":"2024-09-21T05:44:05.360581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:54.441990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352538+0000","last_change":"2024-09-21T05:44:05.350309+0000","last_active":"2024-09-21T05:44:05.352538+0000","last_peered":"2024-09-21T05:44:05.352538+0000","last_clean":"2024-09-21T05:44:05.352538+0000","last_became_active":"2024-09-21T05:44:05.349963+0000","last_became_peered":"2024-09-21T05:44:05.349963+0000","last_unstale":"2024-09-21T05:44:05.352538+0000","last_undegraded":"2024-09-21T05:44:05.352538+0000","last_fullsized":"2024-09-21T05:44:05.352538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:19.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356597+0000","last_change":"2024-09-21T05:44:04.343110+0000","last_active":"2024-09-21T05:44:05.356597+0000","last_peered":"2024-09-21T05:44:05.356597+0000","last_clean":"2024-09-21T05:44:05.356597+0000","last_became_active":"2024-09-21T05:44:04.342707+0000","last_became_peered":"2024-09-21T05:44:04.342707+0000","last_unstale":"2024-09-21T05:44:05.356597+0000","last_undegraded":"2024-09-21T05:44:05.356597+0000","last_fullsized":"2024-09-21T05:44:05.356597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:07.581968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.354858+0000","last_change":"2024-09-21T05:44:05.349097+0000","last_active":"2024-09-21T05:44:05.354858+0000","last_peered":"2024-09-21T05:44:05.354858+0000","last_clean":"2024-09-21T05:44:05.354858+0000","last_became_active":"2024-09-21T05:44:05.348615+0000","last_became_peered":"2024-09-21T05:44:05.348615+0000","last_unstale":"2024-09-21T05:44:05.354858+0000","last_undegraded":"2024-09-21T05:44:05.354858+0000","last_fullsized":"2024-09-21T05:44:05.354858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:44.610721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362220+0000","last_change":"2024-09-21T05:44:04.343679+0000","last_active":"2024-09-21T05:44:05.362220+0000","last_peered":"2024-09-21T05:44:05.362220+0000","last_clean":"2024-09-21T05:44:05.362220+0000","last_became_active":"2024-09-21T05:44:04.342809+0000","last_became_peered":"2024-09-21T05:44:04.342809+0000","last_unstale":"2024-09-21T05:44:05.362220+0000","last_undegraded":"2024-09-21T05:44:05.362220+0000","last_fullsized":"2024-09-21T05:44:05.362220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:37.675139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344836+0000","last_change":"2024-09-21T05:44:05.345277+0000","last_active":"2024-09-21T05:44:06.344836+0000","last_peered":"2024-09-21T05:44:06.344836+0000","last_clean":"2024-09-21T05:44:06.344836+0000","last_became_active":"2024-09-21T05:44:05.344853+0000","last_became_peered":"2024-09-21T05:44:05.344853+0000","last_unstale":"2024-09-21T05:44:06.344836+0000","last_undegraded":"2024-09-21T05:44:06.344836+0000","last_fullsized":"2024-09-21T05:44:06.344836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:47.242980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342582+0000","last_change":"2024-09-21T05:44:04.349271+0000","last_active":"2024-09-21T05:44:05.342582+0000","last_peered":"2024-09-21T05:44:05.342582+0000","last_clean":"2024-09-21T05:44:05.342582+0000","last_became_active":"2024-09-21T05:44:04.348960+0000","last_became_peered":"2024-09-21T05:44:04.348960+0000","last_unstale":"2024-09-21T05:44:05.342582+0000","last_undegraded":"2024-09-21T05:44:05.342582+0000","last_fullsized":"2024-09-21T05:44:05.342582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:03.238378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352960+0000","last_change":"2024-09-21T05:44:05.345213+0000","last_active":"2024-09-21T05:44:05.352960+0000","last_peered":"2024-09-21T05:44:05.352960+0000","last_clean":"2024-09-21T05:44:05.352960+0000","last_became_active":"2024-09-21T05:44:05.344501+0000","last_became_peered":"2024-09-21T05:44:05.344501+0000","last_unstale":"2024-09-21T05:44:05.352960+0000","last_undegraded":"2024-09-21T05:44:05.352960+0000","last_fullsized":"2024-09-21T05:44:05.352960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:42.592592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.341220+0000","last_change":"2024-09-21T05:44:04.347643+0000","last_active":"2024-09-21T05:44:05.341220+0000","last_peered":"2024-09-21T05:44:05.341220+0000","last_clean":"2024-09-21T05:44:05.341220+0000","last_became_active":"2024-09-21T05:44:04.347300+0000","last_became_peered":"2024-09-21T05:44:04.347300+0000","last_unstale":"2024-09-21T05:44:05.341220+0000","last_undegraded":"2024-09-21T05:44:05.341220+0000","last_fullsized":"2024-09-21T05:44:05.341220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:19.427383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.350296+0000","last_change":"2024-09-21T05:44:05.346586+0000","last_active":"2024-09-21T05:44:05.350296+0000","last_peered":"2024-09-21T05:44:05.350296+0000","last_clean":"2024-09-21T05:44:05.350296+0000","last_became_active":"2024-09-21T05:44:05.346279+0000","last_became_peered":"2024-09-21T05:44:05.346279+0000","last_unstale":"2024-09-21T05:44:05.350296+0000","last_undegraded":"2024-09-21T05:44:05.350296+0000","last_fullsized":"2024-09-21T05:44:05.350296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:19.294975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.336633+0000","last_change":"2024-09-21T05:44:04.343612+0000","last_active":"2024-09-21T05:44:05.336633+0000","last_peered":"2024-09-21T05:44:05.336633+0000","last_clean":"2024-09-21T05:44:05.336633+0000","last_became_active":"2024-09-21T05:44:04.343269+0000","last_became_peered":"2024-09-21T05:44:04.343269+0000","last_unstale":"2024-09-21T05:44:05.336633+0000","last_undegraded":"2024-09-21T05:44:05.336633+0000","last_fullsized":"2024-09-21T05:44:05.336633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:44.285360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.350431+0000","last_change":"2024-09-21T05:44:05.342070+0000","last_active":"2024-09-21T05:44:05.350431+0000","last_peered":"2024-09-21T05:44:05.350431+0000","last_clean":"2024-09-21T05:44:05.350431+0000","last_became_active":"2024-09-21T05:44:05.341731+0000","last_became_peered":"2024-09-21T05:44:05.341731+0000","last_unstale":"2024-09-21T05:44:05.350431+0000","last_undegraded":"2024-09-21T05:44:05.350431+0000","last_fullsized":"2024-09-21T05:44:05.350431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:25.063564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.333971+0000","last_change":"2024-09-21T05:44:04.348898+0000","last_active":"2024-09-21T05:44:05.333971+0000","last_peered":"2024-09-21T05:44:05.333971+0000","last_clean":"2024-09-21T05:44:05.333971+0000","last_became_active":"2024-09-21T05:44:04.348578+0000","last_became_peered":"2024-09-21T05:44:04.348578+0000","last_unstale":"2024-09-21T05:44:05.333971+0000","last_undegraded":"2024-09-21T05:44:05.333971+0000","last_fullsized":"2024-09-21T05:44:05.333971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:07.755527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355226+0000","last_change":"2024-09-21T05:44:05.351283+0000","last_active":"2024-09-21T05:44:05.355226+0000","last_peered":"2024-09-21T05:44:05.355226+0000","last_clean":"2024-09-21T05:44:05.355226+0000","last_became_active":"2024-09-21T05:44:05.350959+0000","last_became_peered":"2024-09-21T05:44:05.350959+0000","last_unstale":"2024-09-21T05:44:05.355226+0000","last_undegraded":"2024-09-21T05:44:05.355226+0000","last_fullsized":"2024-09-21T05:44:05.355226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:38.595488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.333862+0000","last_change":"2024-09-21T05:44:04.351604+0000","last_active":"2024-09-21T05:44:05.333862+0000","last_peered":"2024-09-21T05:44:05.333862+0000","last_clean":"2024-09-21T05:44:05.333862+0000","last_became_active":"2024-09-21T05:44:04.351220+0000","last_became_peered":"2024-09-21T05:44:04.351220+0000","last_unstale":"2024-09-21T05:44:05.333862+0000","last_undegraded":"2024-09-21T05:44:05.333862+0000","last_fullsized":"2024-09-21T05:44:05.333862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:34.908441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344660+0000","last_change":"2024-09-21T05:44:05.346613+0000","last_active":"2024-09-21T05:44:06.344660+0000","last_peered":"2024-09-21T05:44:06.344660+0000","last_clean":"2024-09-21T05:44:06.344660+0000","last_became_active":"2024-09-21T05:44:05.346224+0000","last_became_peered":"2024-09-21T05:44:05.346224+0000","last_unstale":"2024-09-21T05:44:06.344660+0000","last_undegraded":"2024-09-21T05:44:06.344660+0000","last_fullsized":"2024-09-21T05:44:06.344660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:36.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362330+0000","last_change":"2024-09-21T05:44:04.338665+0000","last_active":"2024-09-21T05:44:05.362330+0000","last_peered":"2024-09-21T05:44:05.362330+0000","last_clean":"2024-09-21T05:44:05.362330+0000","last_became_active":"2024-09-21T05:44:04.338352+0000","last_became_peered":"2024-09-21T05:44:04.338352+0000","last_unstale":"2024-09-21T05:44:05.362330+0000","last_undegraded":"2024-09-21T05:44:05.362330+0000","last_fullsized":"2024-09-21T05:44:05.362330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:19.783430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361840+0000","last_change":"2024-09-21T05:44:05.356138+0000","last_active":"2024-09-21T05:44:05.361840+0000","last_peered":"2024-09-21T05:44:05.361840+0000","last_clean":"2024-09-21T05:44:05.361840+0000","last_became_active":"2024-09-21T05:44:05.355848+0000","last_became_peered":"2024-09-21T05:44:05.355848+0000","last_unstale":"2024-09-21T05:44:05.361840+0000","last_undegraded":"2024-09-21T05:44:05.361840+0000","last_fullsized":"2024-09-21T05:44:05.361840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:42.533823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.329531+0000","last_change":"2024-09-21T05:44:04.346344+0000","last_active":"2024-09-21T05:44:05.329531+0000","last_peered":"2024-09-21T05:44:05.329531+0000","last_clean":"2024-09-21T05:44:05.329531+0000","last_became_active":"2024-09-21T05:44:04.345922+0000","last_became_peered":"2024-09-21T05:44:04.345922+0000","last_unstale":"2024-09-21T05:44:05.329531+0000","last_undegraded":"2024-09-21T05:44:05.329531+0000","last_fullsized":"2024-09-21T05:44:05.329531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:54.633986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.357019+0000","last_change":"2024-09-21T05:44:05.353659+0000","last_active":"2024-09-21T05:44:05.357019+0000","last_peered":"2024-09-21T05:44:05.357019+0000","last_clean":"2024-09-21T05:44:05.357019+0000","last_became_active":"2024-09-21T05:44:05.352836+0000","last_became_peered":"2024-09-21T05:44:05.352836+0000","last_unstale":"2024-09-21T05:44:05.357019+0000","last_undegraded":"2024-09-21T05:44:05.357019+0000","last_fullsized":"2024-09-21T05:44:05.357019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:55.444437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.330794+0000","last_change":"2024-09-21T05:44:04.342072+0000","last_active":"2024-09-21T05:44:05.330794+0000","last_peered":"2024-09-21T05:44:05.330794+0000","last_clean":"2024-09-21T05:44:05.330794+0000","last_became_active":"2024-09-21T05:44:04.341437+0000","last_became_peered":"2024-09-21T05:44:04.341437+0000","last_unstale":"2024-09-21T05:44:05.330794+0000","last_undegraded":"2024-09-21T05:44:05.330794+0000","last_fullsized":"2024-09-21T05:44:05.330794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:01.352882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344963+0000","last_change":"2024-09-21T05:44:05.359063+0000","last_active":"2024-09-21T05:44:06.344963+0000","last_peered":"2024-09-21T05:44:06.344963+0000","last_clean":"2024-09-21T05:44:06.344963+0000","last_became_active":"2024-09-21T05:44:05.357828+0000","last_became_peered":"2024-09-21T05:44:05.357828+0000","last_unstale":"2024-09-21T05:44:06.344963+0000","last_undegraded":"2024-09-21T05:44:06.344963+0000","last_fullsized":"2024-09-21T05:44:06.344963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:01.373062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334669+0000","last_change":"2024-09-21T05:44:04.343454+0000","last_active":"2024-09-21T05:44:05.334669+0000","last_peered":"2024-09-21T05:44:05.334669+0000","last_clean":"2024-09-21T05:44:05.334669+0000","last_became_active":"2024-09-21T05:44:04.343082+0000","last_became_peered":"2024-09-21T05:44:04.343082+0000","last_unstale":"2024-09-21T05:44:05.334669+0000","last_undegraded":"2024-09-21T05:44:05.334669+0000","last_fullsized":"2024-09-21T05:44:05.334669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:39.774961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353296+0000","last_change":"2024-09-21T05:44:05.348065+0000","last_active":"2024-09-21T05:44:05.353296+0000","last_peered":"2024-09-21T05:44:05.353296+0000","last_clean":"2024-09-21T05:44:05.353296+0000","last_became_active":"2024-09-21T05:44:05.347767+0000","last_became_peered":"2024-09-21T05:44:05.347767+0000","last_unstale":"2024-09-21T05:44:05.353296+0000","last_undegraded":"2024-09-21T05:44:05.353296+0000","last_fullsized":"2024-09-21T05:44:05.353296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:13.813363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.338779+0000","last_change":"2024-09-21T05:44:04.350309+0000","last_active":"2024-09-21T05:44:05.338779+0000","last_peered":"2024-09-21T05:44:05.338779+0000","last_clean":"2024-09-21T05:44:05.338779+0000","last_became_active":"2024-09-21T05:44:04.349925+0000","last_became_peered":"2024-09-21T05:44:04.349925+0000","last_unstale":"2024-09-21T05:44:05.338779+0000","last_undegraded":"2024-09-21T05:44:05.338779+0000","last_fullsized":"2024-09-21T05:44:05.338779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:43.942398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.359127+0000","last_change":"2024-09-21T05:44:05.358027+0000","last_active":"2024-09-21T05:44:05.359127+0000","last_peered":"2024-09-21T05:44:05.359127+0000","last_clean":"2024-09-21T05:44:05.359127+0000","last_became_active":"2024-09-21T05:44:05.357688+0000","last_became_peered":"2024-09-21T05:44:05.357688+0000","last_unstale":"2024-09-21T05:44:05.359127+0000","last_undegraded":"2024-09-21T05:44:05.359127+0000","last_fullsized":"2024-09-21T05:44:05.359127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:14.179685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344594+0000","last_change":"2024-09-21T05:44:04.341481+0000","last_active":"2024-09-21T05:44:06.344594+0000","last_peered":"2024-09-21T05:44:06.344594+0000","last_clean":"2024-09-21T05:44:06.344594+0000","last_became_active":"2024-09-21T05:44:04.340189+0000","last_became_peered":"2024-09-21T05:44:04.340189+0000","last_unstale":"2024-09-21T05:44:06.344594+0000","last_undegraded":"2024-09-21T05:44:06.344594+0000","last_fullsized":"2024-09-21T05:44:06.344594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:20.102950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.364848+0000","last_change":"2024-09-21T05:44:05.362339+0000","last_active":"2024-09-21T05:44:05.364848+0000","last_peered":"2024-09-21T05:44:05.364848+0000","last_clean":"2024-09-21T05:44:05.364848+0000","last_became_active":"2024-09-21T05:44:05.362056+0000","last_became_peered":"2024-09-21T05:44:05.362056+0000","last_unstale":"2024-09-21T05:44:05.364848+0000","last_undegraded":"2024-09-21T05:44:05.364848+0000","last_fullsized":"2024-09-21T05:44:05.364848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:09.074958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.335523+0000","last_change":"2024-09-21T05:44:04.343291+0000","last_active":"2024-09-21T05:44:05.335523+0000","last_peered":"2024-09-21T05:44:05.335523+0000","last_clean":"2024-09-21T05:44:05.335523+0000","last_became_active":"2024-09-21T05:44:04.342978+0000","last_became_peered":"2024-09-21T05:44:04.342978+0000","last_unstale":"2024-09-21T05:44:05.335523+0000","last_undegraded":"2024-09-21T05:44:05.335523+0000","last_fullsized":"2024-09-21T05:44:05.335523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:15.776057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.349721+0000","last_change":"2024-09-21T05:44:05.344130+0000","last_active":"2024-09-21T05:44:05.349721+0000","last_peered":"2024-09-21T05:44:05.349721+0000","last_clean":"2024-09-21T05:44:05.349721+0000","last_became_active":"2024-09-21T05:44:05.343768+0000","last_became_peered":"2024-09-21T05:44:05.343768+0000","last_unstale":"2024-09-21T05:44:05.349721+0000","last_undegraded":"2024-09-21T05:44:05.349721+0000","last_fullsized":"2024-09-21T05:44:05.349721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:13.359400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344128+0000","last_change":"2024-09-21T05:44:04.339456+0000","last_active":"2024-09-21T05:44:06.344128+0000","last_peered":"2024-09-21T05:44:06.344128+0000","last_clean":"2024-09-21T05:44:06.344128+0000","last_became_active":"2024-09-21T05:44:04.339043+0000","last_became_peered":"2024-09-21T05:44:04.339043+0000","last_unstale":"2024-09-21T05:44:06.344128+0000","last_undegraded":"2024-09-21T05:44:06.344128+0000","last_fullsized":"2024-09-21T05:44:06.344128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:28.570238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360867+0000","last_change":"2024-09-21T05:44:05.357732+0000","last_active":"2024-09-21T05:44:05.360867+0000","last_peered":"2024-09-21T05:44:05.360867+0000","last_clean":"2024-09-21T05:44:05.360867+0000","last_became_active":"2024-09-21T05:44:05.357359+0000","last_became_peered":"2024-09-21T05:44:05.357359+0000","last_unstale":"2024-09-21T05:44:05.360867+0000","last_undegraded":"2024-09-21T05:44:05.360867+0000","last_fullsized":"2024-09-21T05:44:05.360867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:00.479100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362005+0000","last_change":"2024-09-21T05:44:04.344203+0000","last_active":"2024-09-21T05:44:05.362005+0000","last_peered":"2024-09-21T05:44:05.362005+0000","last_clean":"2024-09-21T05:44:05.362005+0000","last_became_active":"2024-09-21T05:44:04.343874+0000","last_became_peered":"2024-09-21T05:44:04.343874+0000","last_unstale":"2024-09-21T05:44:05.362005+0000","last_undegraded":"2024-09-21T05:44:05.362005+0000","last_fullsized":"2024-09-21T05:44:05.362005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:18.030294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":217,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.340021+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:44:05.340021+0000","last_peered":"2024-09-21T05:44:05.340021+0000","last_clean":"2024-09-21T05:44:05.340021+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:44:05.340021+0000","last_undegraded":"2024-09-21T05:44:05.340021+0000","last_fullsized":"2024-09-21T05:44:05.340021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":1009933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":419495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36896,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1585152,"data_stored":1009897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:44:09.415 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2386951427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 msgr2=0x7fd0f40ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2386951427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f40ceb50 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009830 tx=0x7fd0e802a510 comp rx=0 tx=0).stop 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2386951427 shutdown_connections 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2386951427 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0f40ccba0 0x7fd0f40ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2386951427 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f40ceb50 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2386951427 >> 172.21.15.22:0/2386951427 conn(0x7fd0f40b3f10 msgr2=0x7fd0f40b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2386951427 shutdown_connections 2024-09-21T05:44:09.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.565+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2386951427 wait complete. 2024-09-21T05:44:09.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.567+0000 7fd0fc00a640 1 Processor -- start 2024-09-21T05:44:09.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.567+0000 7fd0fc00a640 1 -- start start 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0fc00a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f41443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0fc00a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f40ccba0 0x7fd0f4155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0fc00a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0f40d1d40 0x7fd0f41f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0fc00a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40d1230 con 0x7fd0f40b5210 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0fc00a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f40d10b0 con 0x7fd0f40ccba0 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0fc00a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f40d13b0 con 0x7fd0f40d1d40 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0f9da6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f41443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0f9da6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f41443b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36360/0 (socket says 172.21.15.22:36360) 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.568+0000 7fd0f9da6640 1 -- 172.21.15.22:0/2184821429 learned_addr learned my addr 172.21.15.22:0/2184821429 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f95a5640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f40ccba0 0x7fd0f4155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0fa5a7640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0f40d1d40 0x7fd0f41f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f9da6640 1 -- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0f40d1d40 msgr2=0x7fd0f41f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f9da6640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0f40d1d40 0x7fd0f41f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f9da6640 1 -- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f40ccba0 msgr2=0x7fd0f4155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f9da6640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f40ccba0 0x7fd0f4155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f9da6640 1 -- 172.21.15.22:0/2184821429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f41566d0 con 0x7fd0f40b5210 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f9da6640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f41443b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c3a0 tx=0x7fd0ec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec011790 con 0x7fd0f40b5210 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0ec017070 con 0x7fd0f40b5210 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f4155e30 con 0x7fd0f40b5210 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec011cc0 con 0x7fd0f40b5210 2024-09-21T05:44:09.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.569+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f41563f0 con 0x7fd0f40b5210 2024-09-21T05:44:09.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.570+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f40ccfb0 con 0x7fd0f40b5210 2024-09-21T05:44:09.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.571+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0ec01c460 con 0x7fd0f40b5210 2024-09-21T05:44:09.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.571+0000 7fd0f27fc640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd0e0078bc0 0x7fd0e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.572+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec09a980 con 0x7fd0f40b5210 2024-09-21T05:44:09.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.572+0000 7fd0f95a5640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd0e0078bc0 0x7fd0e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.572+0000 7fd0f95a5640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd0e0078bc0 0x7fd0e007b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8034000 tx=0x7fd0e8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:09.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.576+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0f40ccfb0 con 0x7fd0f40b5210 2024-09-21T05:44:09.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.758+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0f40d18d0 con 0x7fd0e0078bc0 2024-09-21T05:44:09.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.762+0000 7fd0f27fc640 1 -- 172.21.15.22:0/2184821429 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7fd0f40d18d0 con 0x7fd0e0078bc0 2024-09-21T05:44:09.767 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:09.768 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:44:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd0e0078bc0 msgr2=0x7fd0e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd0e0078bc0 0x7fd0e007b0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8034000 tx=0x7fd0e8004730 comp rx=0 tx=0).stop 2024-09-21T05:44:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 msgr2=0x7fd0f41443b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f41443b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c3a0 tx=0x7fd0ec00c860 comp rx=0 tx=0).stop 2024-09-21T05:44:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 shutdown_connections 2024-09-21T05:44:09.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd0e0078bc0 0x7fd0e007b0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd0f40d1d40 0x7fd0f41f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0f40ccba0 0x7fd0f4155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 --2- 172.21.15.22:0/2184821429 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0f40b5210 0x7fd0f41443b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 >> 172.21.15.22:0/2184821429 conn(0x7fd0f40b3f10 msgr2=0x7fd0f415a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 shutdown_connections 2024-09-21T05:44:09.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.765+0000 7fd0fc00a640 1 -- 172.21.15.22:0/2184821429 wait complete. 2024-09-21T05:44:09.779 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-21T05:44:07.811988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266660,"kb_used_data":8820,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754708060,"statfs":{"total":773094113280,"available":772821053440,"internally_reserved":0,"allocated":9031680,"data_stored":4502925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.492390"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353933+0000","last_change":"2024-09-21T05:44:05.349009+0000","last_active":"2024-09-21T05:44:05.353933+0000","last_peered":"2024-09-21T05:44:05.353933+0000","last_clean":"2024-09-21T05:44:05.353933+0000","last_became_active":"2024-09-21T05:44:05.348684+0000","last_became_peered":"2024-09-21T05:44:05.348684+0000","last_unstale":"2024-09-21T05:44:05.353933+0000","last_undegraded":"2024-09-21T05:44:05.353933+0000","last_fullsized":"2024-09-21T05:44:05.353933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:38.595488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.359310+0000","last_change":"2024-09-21T05:44:05.338790+0000","last_active":"2024-09-21T05:44:05.359310+0000","last_peered":"2024-09-21T05:44:05.359310+0000","last_clean":"2024-09-21T05:44:05.359310+0000","last_became_active":"2024-09-21T05:44:05.338484+0000","last_became_peered":"2024-09-21T05:44:05.338484+0000","last_unstale":"2024-09-21T05:44:05.359310+0000","last_undegraded":"2024-09-21T05:44:05.359310+0000","last_fullsized":"2024-09-21T05:44:05.359310+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:48:09.907739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358994+0000","last_change":"2024-09-21T05:44:05.353303+0000","last_active":"2024-09-21T05:44:05.358994+0000","last_peered":"2024-09-21T05:44:05.358994+0000","last_clean":"2024-09-21T05:44:05.358994+0000","last_became_active":"2024-09-21T05:44:05.352894+0000","last_became_peered":"2024-09-21T05:44:05.352894+0000","last_unstale":"2024-09-21T05:44:05.358994+0000","last_undegraded":"2024-09-21T05:44:05.358994+0000","last_fullsized":"2024-09-21T05:44:05.358994+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:42.478156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352963+0000","last_change":"2024-09-21T05:44:05.345417+0000","last_active":"2024-09-21T05:44:05.352963+0000","last_peered":"2024-09-21T05:44:05.352963+0000","last_clean":"2024-09-21T05:44:05.352963+0000","last_became_active":"2024-09-21T05:44:05.344787+0000","last_became_peered":"2024-09-21T05:44:05.344787+0000","last_unstale":"2024-09-21T05:44:05.352963+0000","last_undegraded":"2024-09-21T05:44:05.352963+0000","last_fullsized":"2024-09-21T05:44:05.352963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:17:54.900091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344851+0000","last_change":"2024-09-21T05:44:05.346688+0000","last_active":"2024-09-21T05:44:06.344851+0000","last_peered":"2024-09-21T05:44:06.344851+0000","last_clean":"2024-09-21T05:44:06.344851+0000","last_became_active":"2024-09-21T05:44:05.346384+0000","last_became_peered":"2024-09-21T05:44:05.346384+0000","last_unstale":"2024-09-21T05:44:06.344851+0000","last_undegraded":"2024-09-21T05:44:06.344851+0000","last_fullsized":"2024-09-21T05:44:06.344851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:40.020576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355127+0000","last_change":"2024-09-21T05:44:05.350307+0000","last_active":"2024-09-21T05:44:05.355127+0000","last_peered":"2024-09-21T05:44:05.355127+0000","last_clean":"2024-09-21T05:44:05.355127+0000","last_became_active":"2024-09-21T05:44:05.349838+0000","last_became_peered":"2024-09-21T05:44:05.349838+0000","last_unstale":"2024-09-21T05:44:05.355127+0000","last_undegraded":"2024-09-21T05:44:05.355127+0000","last_fullsized":"2024-09-21T05:44:05.355127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:06.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353162+0000","last_change":"2024-09-21T05:44:05.346679+0000","last_active":"2024-09-21T05:44:05.353162+0000","last_peered":"2024-09-21T05:44:05.353162+0000","last_clean":"2024-09-21T05:44:05.353162+0000","last_became_active":"2024-09-21T05:44:05.346365+0000","last_became_peered":"2024-09-21T05:44:05.346365+0000","last_unstale":"2024-09-21T05:44:05.353162+0000","last_undegraded":"2024-09-21T05:44:05.353162+0000","last_fullsized":"2024-09-21T05:44:05.353162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:29.711495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361481+0000","last_change":"2024-09-21T05:44:05.354937+0000","last_active":"2024-09-21T05:44:05.361481+0000","last_peered":"2024-09-21T05:44:05.361481+0000","last_clean":"2024-09-21T05:44:05.361481+0000","last_became_active":"2024-09-21T05:44:05.354632+0000","last_became_peered":"2024-09-21T05:44:05.354632+0000","last_unstale":"2024-09-21T05:44:05.361481+0000","last_undegraded":"2024-09-21T05:44:05.361481+0000","last_fullsized":"2024-09-21T05:44:05.361481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:38.677586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355935+0000","last_change":"2024-09-21T05:44:05.351441+0000","last_active":"2024-09-21T05:44:05.355935+0000","last_peered":"2024-09-21T05:44:05.355935+0000","last_clean":"2024-09-21T05:44:05.355935+0000","last_became_active":"2024-09-21T05:44:05.351008+0000","last_became_peered":"2024-09-21T05:44:05.351008+0000","last_unstale":"2024-09-21T05:44:05.355935+0000","last_undegraded":"2024-09-21T05:44:05.355935+0000","last_fullsized":"2024-09-21T05:44:05.355935+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:44.610721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.349724+0000","last_change":"2024-09-21T05:44:05.345624+0000","last_active":"2024-09-21T05:44:05.349724+0000","last_peered":"2024-09-21T05:44:05.349724+0000","last_clean":"2024-09-21T05:44:05.349724+0000","last_became_active":"2024-09-21T05:44:05.345177+0000","last_became_peered":"2024-09-21T05:44:05.345177+0000","last_unstale":"2024-09-21T05:44:05.349724+0000","last_undegraded":"2024-09-21T05:44:05.349724+0000","last_fullsized":"2024-09-21T05:44:05.349724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:13.813363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361064+0000","last_change":"2024-09-21T05:44:05.355053+0000","last_active":"2024-09-21T05:44:05.361064+0000","last_peered":"2024-09-21T05:44:05.361064+0000","last_clean":"2024-09-21T05:44:05.361064+0000","last_became_active":"2024-09-21T05:44:05.354482+0000","last_became_peered":"2024-09-21T05:44:05.354482+0000","last_unstale":"2024-09-21T05:44:05.361064+0000","last_undegraded":"2024-09-21T05:44:05.361064+0000","last_fullsized":"2024-09-21T05:44:05.361064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:31:10.035432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353876+0000","last_change":"2024-09-21T05:44:05.345959+0000","last_active":"2024-09-21T05:44:05.353876+0000","last_peered":"2024-09-21T05:44:05.353876+0000","last_clean":"2024-09-21T05:44:05.353876+0000","last_became_active":"2024-09-21T05:44:05.345647+0000","last_became_peered":"2024-09-21T05:44:05.345647+0000","last_unstale":"2024-09-21T05:44:05.353876+0000","last_undegraded":"2024-09-21T05:44:05.353876+0000","last_fullsized":"2024-09-21T05:44:05.353876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:06.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358986+0000","last_change":"2024-09-21T05:44:05.354156+0000","last_active":"2024-09-21T05:44:05.358986+0000","last_peered":"2024-09-21T05:44:05.358986+0000","last_clean":"2024-09-21T05:44:05.358986+0000","last_became_active":"2024-09-21T05:44:05.353827+0000","last_became_peered":"2024-09-21T05:44:05.353827+0000","last_unstale":"2024-09-21T05:44:05.358986+0000","last_undegraded":"2024-09-21T05:44:05.358986+0000","last_fullsized":"2024-09-21T05:44:05.358986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:25.027057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352929+0000","last_change":"2024-09-21T05:44:05.346007+0000","last_active":"2024-09-21T05:44:05.352929+0000","last_peered":"2024-09-21T05:44:05.352929+0000","last_clean":"2024-09-21T05:44:05.352929+0000","last_became_active":"2024-09-21T05:44:05.344913+0000","last_became_peered":"2024-09-21T05:44:05.344913+0000","last_unstale":"2024-09-21T05:44:05.352929+0000","last_undegraded":"2024-09-21T05:44:05.352929+0000","last_fullsized":"2024-09-21T05:44:05.352929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:04.577555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353460+0000","last_change":"2024-09-21T05:44:05.346788+0000","last_active":"2024-09-21T05:44:05.353460+0000","last_peered":"2024-09-21T05:44:05.353460+0000","last_clean":"2024-09-21T05:44:05.353460+0000","last_became_active":"2024-09-21T05:44:05.346475+0000","last_became_peered":"2024-09-21T05:44:05.346475+0000","last_unstale":"2024-09-21T05:44:05.353460+0000","last_undegraded":"2024-09-21T05:44:05.353460+0000","last_fullsized":"2024-09-21T05:44:05.353460+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:41.613302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344263+0000","last_change":"2024-09-21T05:44:05.347529+0000","last_active":"2024-09-21T05:44:06.344263+0000","last_peered":"2024-09-21T05:44:06.344263+0000","last_clean":"2024-09-21T05:44:06.344263+0000","last_became_active":"2024-09-21T05:44:05.347178+0000","last_became_peered":"2024-09-21T05:44:05.347178+0000","last_unstale":"2024-09-21T05:44:06.344263+0000","last_undegraded":"2024-09-21T05:44:06.344263+0000","last_fullsized":"2024-09-21T05:44:06.344263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:42.533823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344713+0000","last_change":"2024-09-21T05:44:05.340682+0000","last_active":"2024-09-21T05:44:06.344713+0000","last_peered":"2024-09-21T05:44:06.344713+0000","last_clean":"2024-09-21T05:44:06.344713+0000","last_became_active":"2024-09-21T05:44:05.340090+0000","last_became_peered":"2024-09-21T05:44:05.340090+0000","last_unstale":"2024-09-21T05:44:06.344713+0000","last_undegraded":"2024-09-21T05:44:06.344713+0000","last_fullsized":"2024-09-21T05:44:06.344713+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:42.870502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353010+0000","last_change":"2024-09-21T05:44:05.349019+0000","last_active":"2024-09-21T05:44:05.353010+0000","last_peered":"2024-09-21T05:44:05.353010+0000","last_clean":"2024-09-21T05:44:05.353010+0000","last_became_active":"2024-09-21T05:44:05.348619+0000","last_became_peered":"2024-09-21T05:44:05.348619+0000","last_unstale":"2024-09-21T05:44:05.353010+0000","last_undegraded":"2024-09-21T05:44:05.353010+0000","last_fullsized":"2024-09-21T05:44:05.353010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:40:15.642288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358516+0000","last_change":"2024-09-21T05:44:05.355517+0000","last_active":"2024-09-21T05:44:05.358516+0000","last_peered":"2024-09-21T05:44:05.358516+0000","last_clean":"2024-09-21T05:44:05.358516+0000","last_became_active":"2024-09-21T05:44:05.355081+0000","last_became_peered":"2024-09-21T05:44:05.355081+0000","last_unstale":"2024-09-21T05:44:05.358516+0000","last_undegraded":"2024-09-21T05:44:05.358516+0000","last_fullsized":"2024-09-21T05:44:05.358516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:31.701087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358655+0000","last_change":"2024-09-21T05:44:05.356583+0000","last_active":"2024-09-21T05:44:05.358655+0000","last_peered":"2024-09-21T05:44:05.358655+0000","last_clean":"2024-09-21T05:44:05.358655+0000","last_became_active":"2024-09-21T05:44:05.356252+0000","last_became_peered":"2024-09-21T05:44:05.356252+0000","last_unstale":"2024-09-21T05:44:05.358655+0000","last_undegraded":"2024-09-21T05:44:05.358655+0000","last_fullsized":"2024-09-21T05:44:05.358655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:28.285040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356627+0000","last_change":"2024-09-21T05:44:05.349964+0000","last_active":"2024-09-21T05:44:05.356627+0000","last_peered":"2024-09-21T05:44:05.356627+0000","last_clean":"2024-09-21T05:44:05.356627+0000","last_became_active":"2024-09-21T05:44:05.349547+0000","last_became_peered":"2024-09-21T05:44:05.349547+0000","last_unstale":"2024-09-21T05:44:05.356627+0000","last_undegraded":"2024-09-21T05:44:05.356627+0000","last_fullsized":"2024-09-21T05:44:05.356627+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:05:08.822727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356308+0000","last_change":"2024-09-21T05:44:05.351112+0000","last_active":"2024-09-21T05:44:05.356308+0000","last_peered":"2024-09-21T05:44:05.356308+0000","last_clean":"2024-09-21T05:44:05.356308+0000","last_became_active":"2024-09-21T05:44:05.350782+0000","last_became_peered":"2024-09-21T05:44:05.350782+0000","last_unstale":"2024-09-21T05:44:05.356308+0000","last_undegraded":"2024-09-21T05:44:05.356308+0000","last_fullsized":"2024-09-21T05:44:05.356308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:55.842648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356769+0000","last_change":"2024-09-21T05:44:05.351172+0000","last_active":"2024-09-21T05:44:05.356769+0000","last_peered":"2024-09-21T05:44:05.356769+0000","last_clean":"2024-09-21T05:44:05.356769+0000","last_became_active":"2024-09-21T05:44:05.350867+0000","last_became_peered":"2024-09-21T05:44:05.350867+0000","last_unstale":"2024-09-21T05:44:05.356769+0000","last_undegraded":"2024-09-21T05:44:05.356769+0000","last_fullsized":"2024-09-21T05:44:05.356769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:38.677586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356742+0000","last_change":"2024-09-21T05:44:05.351193+0000","last_active":"2024-09-21T05:44:05.356742+0000","last_peered":"2024-09-21T05:44:05.356742+0000","last_clean":"2024-09-21T05:44:05.356742+0000","last_became_active":"2024-09-21T05:44:05.350886+0000","last_became_peered":"2024-09-21T05:44:05.350886+0000","last_unstale":"2024-09-21T05:44:05.356742+0000","last_undegraded":"2024-09-21T05:44:05.356742+0000","last_fullsized":"2024-09-21T05:44:05.356742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:12.196197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342205+0000","last_change":"2024-09-21T05:44:05.337141+0000","last_active":"2024-09-21T05:44:05.342205+0000","last_peered":"2024-09-21T05:44:05.342205+0000","last_clean":"2024-09-21T05:44:05.342205+0000","last_became_active":"2024-09-21T05:44:05.336815+0000","last_became_peered":"2024-09-21T05:44:05.336815+0000","last_unstale":"2024-09-21T05:44:05.342205+0000","last_undegraded":"2024-09-21T05:44:05.342205+0000","last_fullsized":"2024-09-21T05:44:05.342205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:22.477887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.364506+0000","last_change":"2024-09-21T05:44:05.361335+0000","last_active":"2024-09-21T05:44:05.364506+0000","last_peered":"2024-09-21T05:44:05.364506+0000","last_clean":"2024-09-21T05:44:05.364506+0000","last_became_active":"2024-09-21T05:44:05.361084+0000","last_became_peered":"2024-09-21T05:44:05.361084+0000","last_unstale":"2024-09-21T05:44:05.364506+0000","last_undegraded":"2024-09-21T05:44:05.364506+0000","last_fullsized":"2024-09-21T05:44:05.364506+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:36.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344488+0000","last_change":"2024-09-21T05:44:05.346863+0000","last_active":"2024-09-21T05:44:06.344488+0000","last_peered":"2024-09-21T05:44:06.344488+0000","last_clean":"2024-09-21T05:44:06.344488+0000","last_became_active":"2024-09-21T05:44:05.346535+0000","last_became_peered":"2024-09-21T05:44:05.346535+0000","last_unstale":"2024-09-21T05:44:06.344488+0000","last_undegraded":"2024-09-21T05:44:06.344488+0000","last_fullsized":"2024-09-21T05:44:06.344488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:23.974145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356119+0000","last_change":"2024-09-21T05:44:05.349982+0000","last_active":"2024-09-21T05:44:05.356119+0000","last_peered":"2024-09-21T05:44:05.356119+0000","last_clean":"2024-09-21T05:44:05.356119+0000","last_became_active":"2024-09-21T05:44:05.349586+0000","last_became_peered":"2024-09-21T05:44:05.349586+0000","last_unstale":"2024-09-21T05:44:05.356119+0000","last_undegraded":"2024-09-21T05:44:05.356119+0000","last_fullsized":"2024-09-21T05:44:05.356119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:33.579521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.354545+0000","last_change":"2024-09-21T05:44:05.343070+0000","last_active":"2024-09-21T05:44:05.354545+0000","last_peered":"2024-09-21T05:44:05.354545+0000","last_clean":"2024-09-21T05:44:05.354545+0000","last_became_active":"2024-09-21T05:44:05.342757+0000","last_became_peered":"2024-09-21T05:44:05.342757+0000","last_unstale":"2024-09-21T05:44:05.354545+0000","last_undegraded":"2024-09-21T05:44:05.354545+0000","last_fullsized":"2024-09-21T05:44:05.354545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:01.192054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355708+0000","last_change":"2024-09-21T05:44:05.346217+0000","last_active":"2024-09-21T05:44:05.355708+0000","last_peered":"2024-09-21T05:44:05.355708+0000","last_clean":"2024-09-21T05:44:05.355708+0000","last_became_active":"2024-09-21T05:44:05.345936+0000","last_became_peered":"2024-09-21T05:44:05.345936+0000","last_unstale":"2024-09-21T05:44:05.355708+0000","last_undegraded":"2024-09-21T05:44:05.355708+0000","last_fullsized":"2024-09-21T05:44:05.355708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:50:48.479671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344231+0000","last_change":"2024-09-21T05:44:05.351496+0000","last_active":"2024-09-21T05:44:06.344231+0000","last_peered":"2024-09-21T05:44:06.344231+0000","last_clean":"2024-09-21T05:44:06.344231+0000","last_became_active":"2024-09-21T05:44:05.351142+0000","last_became_peered":"2024-09-21T05:44:05.351142+0000","last_unstale":"2024-09-21T05:44:06.344231+0000","last_undegraded":"2024-09-21T05:44:06.344231+0000","last_fullsized":"2024-09-21T05:44:06.344231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:40.154352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.359410+0000","last_change":"2024-09-21T05:44:05.355633+0000","last_active":"2024-09-21T05:44:05.359410+0000","last_peered":"2024-09-21T05:44:05.359410+0000","last_clean":"2024-09-21T05:44:05.359410+0000","last_became_active":"2024-09-21T05:44:05.354585+0000","last_became_peered":"2024-09-21T05:44:05.354585+0000","last_unstale":"2024-09-21T05:44:05.359410+0000","last_undegraded":"2024-09-21T05:44:05.359410+0000","last_fullsized":"2024-09-21T05:44:05.359410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:12.939705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344987+0000","last_change":"2024-09-21T05:44:05.347900+0000","last_active":"2024-09-21T05:44:06.344987+0000","last_peered":"2024-09-21T05:44:06.344987+0000","last_clean":"2024-09-21T05:44:06.344987+0000","last_became_active":"2024-09-21T05:44:05.347597+0000","last_became_peered":"2024-09-21T05:44:05.347597+0000","last_unstale":"2024-09-21T05:44:06.344987+0000","last_undegraded":"2024-09-21T05:44:06.344987+0000","last_fullsized":"2024-09-21T05:44:06.344987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:54.890455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"81'4","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358782+0000","last_change":"2024-09-21T05:44:04.340613+0000","last_active":"2024-09-21T05:44:05.358782+0000","last_peered":"2024-09-21T05:44:05.358782+0000","last_clean":"2024-09-21T05:44:05.358782+0000","last_became_active":"2024-09-21T05:44:04.340249+0000","last_became_peered":"2024-09-21T05:44:04.340249+0000","last_unstale":"2024-09-21T05:44:05.358782+0000","last_undegraded":"2024-09-21T05:44:05.358782+0000","last_fullsized":"2024-09-21T05:44:05.358782+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:55.434596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356980+0000","last_change":"2024-09-21T05:44:05.351385+0000","last_active":"2024-09-21T05:44:05.356980+0000","last_peered":"2024-09-21T05:44:05.356980+0000","last_clean":"2024-09-21T05:44:05.356980+0000","last_became_active":"2024-09-21T05:44:05.350942+0000","last_became_peered":"2024-09-21T05:44:05.350942+0000","last_unstale":"2024-09-21T05:44:05.356980+0000","last_undegraded":"2024-09-21T05:44:05.356980+0000","last_fullsized":"2024-09-21T05:44:05.356980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:44.944845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344924+0000","last_change":"2024-09-21T05:44:04.341819+0000","last_active":"2024-09-21T05:44:06.344924+0000","last_peered":"2024-09-21T05:44:06.344924+0000","last_clean":"2024-09-21T05:44:06.344924+0000","last_became_active":"2024-09-21T05:44:04.339841+0000","last_became_peered":"2024-09-21T05:44:04.339841+0000","last_unstale":"2024-09-21T05:44:06.344924+0000","last_undegraded":"2024-09-21T05:44:06.344924+0000","last_fullsized":"2024-09-21T05:44:06.344924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:24.773340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.350208+0000","last_change":"2024-09-21T05:44:05.345013+0000","last_active":"2024-09-21T05:44:05.350208+0000","last_peered":"2024-09-21T05:44:05.350208+0000","last_clean":"2024-09-21T05:44:05.350208+0000","last_became_active":"2024-09-21T05:44:05.344705+0000","last_became_peered":"2024-09-21T05:44:05.344705+0000","last_unstale":"2024-09-21T05:44:05.350208+0000","last_undegraded":"2024-09-21T05:44:05.350208+0000","last_fullsized":"2024-09-21T05:44:05.350208+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:19.294975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358435+0000","last_change":"2024-09-21T05:44:04.348855+0000","last_active":"2024-09-21T05:44:05.358435+0000","last_peered":"2024-09-21T05:44:05.358435+0000","last_clean":"2024-09-21T05:44:05.358435+0000","last_became_active":"2024-09-21T05:44:04.347770+0000","last_became_peered":"2024-09-21T05:44:04.347770+0000","last_unstale":"2024-09-21T05:44:05.358435+0000","last_undegraded":"2024-09-21T05:44:05.358435+0000","last_fullsized":"2024-09-21T05:44:05.358435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:01:28.270079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361780+0000","last_change":"2024-09-21T05:44:05.354889+0000","last_active":"2024-09-21T05:44:05.361780+0000","last_peered":"2024-09-21T05:44:05.361780+0000","last_clean":"2024-09-21T05:44:05.361780+0000","last_became_active":"2024-09-21T05:44:05.354585+0000","last_became_peered":"2024-09-21T05:44:05.354585+0000","last_unstale":"2024-09-21T05:44:05.361780+0000","last_undegraded":"2024-09-21T05:44:05.361780+0000","last_fullsized":"2024-09-21T05:44:05.361780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:40.154352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358973+0000","last_change":"2024-09-21T05:44:04.348039+0000","last_active":"2024-09-21T05:44:05.358973+0000","last_peered":"2024-09-21T05:44:05.358973+0000","last_clean":"2024-09-21T05:44:05.358973+0000","last_became_active":"2024-09-21T05:44:04.347671+0000","last_became_peered":"2024-09-21T05:44:04.347671+0000","last_unstale":"2024-09-21T05:44:05.358973+0000","last_undegraded":"2024-09-21T05:44:05.358973+0000","last_fullsized":"2024-09-21T05:44:05.358973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:11.937258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355675+0000","last_change":"2024-09-21T05:44:05.345429+0000","last_active":"2024-09-21T05:44:05.355675+0000","last_peered":"2024-09-21T05:44:05.355675+0000","last_clean":"2024-09-21T05:44:05.355675+0000","last_became_active":"2024-09-21T05:44:05.345143+0000","last_became_peered":"2024-09-21T05:44:05.345143+0000","last_unstale":"2024-09-21T05:44:05.355675+0000","last_undegraded":"2024-09-21T05:44:05.355675+0000","last_fullsized":"2024-09-21T05:44:05.355675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:09.786674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.345254+0000","last_change":"2024-09-21T05:44:04.348145+0000","last_active":"2024-09-21T05:44:06.345254+0000","last_peered":"2024-09-21T05:44:06.345254+0000","last_clean":"2024-09-21T05:44:06.345254+0000","last_became_active":"2024-09-21T05:44:04.347611+0000","last_became_peered":"2024-09-21T05:44:04.347611+0000","last_unstale":"2024-09-21T05:44:06.345254+0000","last_undegraded":"2024-09-21T05:44:06.345254+0000","last_fullsized":"2024-09-21T05:44:06.345254+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:12.180410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.345121+0000","last_change":"2024-09-21T05:44:05.343853+0000","last_active":"2024-09-21T05:44:06.345121+0000","last_peered":"2024-09-21T05:44:06.345121+0000","last_clean":"2024-09-21T05:44:06.345121+0000","last_became_active":"2024-09-21T05:44:05.343546+0000","last_became_peered":"2024-09-21T05:44:05.343546+0000","last_unstale":"2024-09-21T05:44:06.345121+0000","last_undegraded":"2024-09-21T05:44:06.345121+0000","last_fullsized":"2024-09-21T05:44:06.345121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:49:54.420097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344440+0000","last_change":"2024-09-21T05:44:04.341770+0000","last_active":"2024-09-21T05:44:06.344440+0000","last_peered":"2024-09-21T05:44:06.344440+0000","last_clean":"2024-09-21T05:44:06.344440+0000","last_became_active":"2024-09-21T05:44:04.339648+0000","last_became_peered":"2024-09-21T05:44:04.339648+0000","last_unstale":"2024-09-21T05:44:06.344440+0000","last_undegraded":"2024-09-21T05:44:06.344440+0000","last_fullsized":"2024-09-21T05:44:06.344440+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:10:52.442075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361744+0000","last_change":"2024-09-21T05:44:05.355168+0000","last_active":"2024-09-21T05:44:05.361744+0000","last_peered":"2024-09-21T05:44:05.361744+0000","last_clean":"2024-09-21T05:44:05.361744+0000","last_became_active":"2024-09-21T05:44:05.354808+0000","last_became_peered":"2024-09-21T05:44:05.354808+0000","last_unstale":"2024-09-21T05:44:05.361744+0000","last_undegraded":"2024-09-21T05:44:05.361744+0000","last_fullsized":"2024-09-21T05:44:05.361744+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:27.630528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334485+0000","last_change":"2024-09-21T05:44:04.350597+0000","last_active":"2024-09-21T05:44:05.334485+0000","last_peered":"2024-09-21T05:44:05.334485+0000","last_clean":"2024-09-21T05:44:05.334485+0000","last_became_active":"2024-09-21T05:44:04.350315+0000","last_became_peered":"2024-09-21T05:44:04.350315+0000","last_unstale":"2024-09-21T05:44:05.334485+0000","last_undegraded":"2024-09-21T05:44:05.334485+0000","last_fullsized":"2024-09-21T05:44:05.334485+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:32.577075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353422+0000","last_change":"2024-09-21T05:44:05.345901+0000","last_active":"2024-09-21T05:44:05.353422+0000","last_peered":"2024-09-21T05:44:05.353422+0000","last_clean":"2024-09-21T05:44:05.353422+0000","last_became_active":"2024-09-21T05:44:05.345539+0000","last_became_peered":"2024-09-21T05:44:05.345539+0000","last_unstale":"2024-09-21T05:44:05.353422+0000","last_undegraded":"2024-09-21T05:44:05.353422+0000","last_fullsized":"2024-09-21T05:44:05.353422+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:57:27.185130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.328438+0000","last_change":"2024-09-21T05:44:04.337933+0000","last_active":"2024-09-21T05:44:05.328438+0000","last_peered":"2024-09-21T05:44:05.328438+0000","last_clean":"2024-09-21T05:44:05.328438+0000","last_became_active":"2024-09-21T05:44:04.337601+0000","last_became_peered":"2024-09-21T05:44:04.337601+0000","last_unstale":"2024-09-21T05:44:05.328438+0000","last_undegraded":"2024-09-21T05:44:05.328438+0000","last_fullsized":"2024-09-21T05:44:05.328438+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:42.293854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342619+0000","last_change":"2024-09-21T05:44:05.341219+0000","last_active":"2024-09-21T05:44:05.342619+0000","last_peered":"2024-09-21T05:44:05.342619+0000","last_clean":"2024-09-21T05:44:05.342619+0000","last_became_active":"2024-09-21T05:44:05.340194+0000","last_became_peered":"2024-09-21T05:44:05.340194+0000","last_unstale":"2024-09-21T05:44:05.342619+0000","last_undegraded":"2024-09-21T05:44:05.342619+0000","last_fullsized":"2024-09-21T05:44:05.342619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:44.720004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.335763+0000","last_change":"2024-09-21T05:44:04.343910+0000","last_active":"2024-09-21T05:44:05.335763+0000","last_peered":"2024-09-21T05:44:05.335763+0000","last_clean":"2024-09-21T05:44:05.335763+0000","last_became_active":"2024-09-21T05:44:04.343564+0000","last_became_peered":"2024-09-21T05:44:04.343564+0000","last_unstale":"2024-09-21T05:44:05.335763+0000","last_undegraded":"2024-09-21T05:44:05.335763+0000","last_fullsized":"2024-09-21T05:44:05.335763+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:36:21.725240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.358896+0000","last_change":"2024-09-21T05:44:05.352689+0000","last_active":"2024-09-21T05:44:05.358896+0000","last_peered":"2024-09-21T05:44:05.358896+0000","last_clean":"2024-09-21T05:44:05.358896+0000","last_became_active":"2024-09-21T05:44:05.352326+0000","last_became_peered":"2024-09-21T05:44:05.352326+0000","last_unstale":"2024-09-21T05:44:05.358896+0000","last_undegraded":"2024-09-21T05:44:05.358896+0000","last_fullsized":"2024-09-21T05:44:05.358896+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:33.579521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334295+0000","last_change":"2024-09-21T05:44:04.340286+0000","last_active":"2024-09-21T05:44:05.334295+0000","last_peered":"2024-09-21T05:44:05.334295+0000","last_clean":"2024-09-21T05:44:05.334295+0000","last_became_active":"2024-09-21T05:44:04.339041+0000","last_became_peered":"2024-09-21T05:44:04.339041+0000","last_unstale":"2024-09-21T05:44:05.334295+0000","last_undegraded":"2024-09-21T05:44:05.334295+0000","last_fullsized":"2024-09-21T05:44:05.334295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:42.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356934+0000","last_change":"2024-09-21T05:44:05.353220+0000","last_active":"2024-09-21T05:44:05.356934+0000","last_peered":"2024-09-21T05:44:05.356934+0000","last_clean":"2024-09-21T05:44:05.356934+0000","last_became_active":"2024-09-21T05:44:05.352886+0000","last_became_peered":"2024-09-21T05:44:05.352886+0000","last_unstale":"2024-09-21T05:44:05.356934+0000","last_undegraded":"2024-09-21T05:44:05.356934+0000","last_fullsized":"2024-09-21T05:44:05.356934+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:21.308548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"81'3","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362627+0000","last_change":"2024-09-21T05:44:04.342388+0000","last_active":"2024-09-21T05:44:05.362627+0000","last_peered":"2024-09-21T05:44:05.362627+0000","last_clean":"2024-09-21T05:44:05.362627+0000","last_became_active":"2024-09-21T05:44:04.341840+0000","last_became_peered":"2024-09-21T05:44:04.341840+0000","last_unstale":"2024-09-21T05:44:05.362627+0000","last_undegraded":"2024-09-21T05:44:05.362627+0000","last_fullsized":"2024-09-21T05:44:05.362627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:13.602788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344698+0000","last_change":"2024-09-21T05:44:05.357914+0000","last_active":"2024-09-21T05:44:06.344698+0000","last_peered":"2024-09-21T05:44:06.344698+0000","last_clean":"2024-09-21T05:44:06.344698+0000","last_became_active":"2024-09-21T05:44:05.357600+0000","last_became_peered":"2024-09-21T05:44:05.357600+0000","last_unstale":"2024-09-21T05:44:06.344698+0000","last_undegraded":"2024-09-21T05:44:06.344698+0000","last_fullsized":"2024-09-21T05:44:06.344698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:09.074958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"81'6","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361530+0000","last_change":"2024-09-21T05:44:04.342115+0000","last_active":"2024-09-21T05:44:05.361530+0000","last_peered":"2024-09-21T05:44:05.361530+0000","last_clean":"2024-09-21T05:44:05.361530+0000","last_became_active":"2024-09-21T05:44:04.341803+0000","last_became_peered":"2024-09-21T05:44:04.341803+0000","last_unstale":"2024-09-21T05:44:05.361530+0000","last_undegraded":"2024-09-21T05:44:05.361530+0000","last_fullsized":"2024-09-21T05:44:05.361530+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:12.356953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352577+0000","last_change":"2024-09-21T05:44:05.349982+0000","last_active":"2024-09-21T05:44:05.352577+0000","last_peered":"2024-09-21T05:44:05.352577+0000","last_clean":"2024-09-21T05:44:05.352577+0000","last_became_active":"2024-09-21T05:44:05.349667+0000","last_became_peered":"2024-09-21T05:44:05.349667+0000","last_unstale":"2024-09-21T05:44:05.352577+0000","last_undegraded":"2024-09-21T05:44:05.352577+0000","last_fullsized":"2024-09-21T05:44:05.352577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:06.256673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360966+0000","last_change":"2024-09-21T05:44:04.348660+0000","last_active":"2024-09-21T05:44:05.360966+0000","last_peered":"2024-09-21T05:44:05.360966+0000","last_clean":"2024-09-21T05:44:05.360966+0000","last_became_active":"2024-09-21T05:44:04.348188+0000","last_became_peered":"2024-09-21T05:44:04.348188+0000","last_unstale":"2024-09-21T05:44:05.360966+0000","last_undegraded":"2024-09-21T05:44:05.360966+0000","last_fullsized":"2024-09-21T05:44:05.360966+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:30.741577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361858+0000","last_change":"2024-09-21T05:44:05.358289+0000","last_active":"2024-09-21T05:44:05.361858+0000","last_peered":"2024-09-21T05:44:05.361858+0000","last_clean":"2024-09-21T05:44:05.361858+0000","last_became_active":"2024-09-21T05:44:05.357967+0000","last_became_peered":"2024-09-21T05:44:05.357967+0000","last_unstale":"2024-09-21T05:44:05.361858+0000","last_undegraded":"2024-09-21T05:44:05.361858+0000","last_fullsized":"2024-09-21T05:44:05.361858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:23.974145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334363+0000","last_change":"2024-09-21T05:44:04.345322+0000","last_active":"2024-09-21T05:44:05.334363+0000","last_peered":"2024-09-21T05:44:05.334363+0000","last_clean":"2024-09-21T05:44:05.334363+0000","last_became_active":"2024-09-21T05:44:04.344943+0000","last_became_peered":"2024-09-21T05:44:04.344943+0000","last_unstale":"2024-09-21T05:44:05.334363+0000","last_undegraded":"2024-09-21T05:44:05.334363+0000","last_fullsized":"2024-09-21T05:44:05.334363+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:40.582348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360680+0000","last_change":"2024-09-21T05:44:05.344618+0000","last_active":"2024-09-21T05:44:05.360680+0000","last_peered":"2024-09-21T05:44:05.360680+0000","last_clean":"2024-09-21T05:44:05.360680+0000","last_became_active":"2024-09-21T05:44:05.344190+0000","last_became_peered":"2024-09-21T05:44:05.344190+0000","last_unstale":"2024-09-21T05:44:05.360680+0000","last_undegraded":"2024-09-21T05:44:05.360680+0000","last_fullsized":"2024-09-21T05:44:05.360680+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:33.626250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.339199+0000","last_change":"2024-09-21T05:44:04.340367+0000","last_active":"2024-09-21T05:44:05.339199+0000","last_peered":"2024-09-21T05:44:05.339199+0000","last_clean":"2024-09-21T05:44:05.339199+0000","last_became_active":"2024-09-21T05:44:04.339997+0000","last_became_peered":"2024-09-21T05:44:04.339997+0000","last_unstale":"2024-09-21T05:44:05.339199+0000","last_undegraded":"2024-09-21T05:44:05.339199+0000","last_fullsized":"2024-09-21T05:44:05.339199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:07.581968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342160+0000","last_change":"2024-09-21T05:44:05.337922+0000","last_active":"2024-09-21T05:44:05.342160+0000","last_peered":"2024-09-21T05:44:05.342160+0000","last_clean":"2024-09-21T05:44:05.342160+0000","last_became_active":"2024-09-21T05:44:05.337625+0000","last_became_peered":"2024-09-21T05:44:05.337625+0000","last_unstale":"2024-09-21T05:44:05.342160+0000","last_undegraded":"2024-09-21T05:44:05.342160+0000","last_fullsized":"2024-09-21T05:44:05.342160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:28:22.850598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"81'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360581+0000","last_change":"2024-09-21T05:44:04.351389+0000","last_active":"2024-09-21T05:44:05.360581+0000","last_peered":"2024-09-21T05:44:05.360581+0000","last_clean":"2024-09-21T05:44:05.360581+0000","last_became_active":"2024-09-21T05:44:04.351055+0000","last_became_peered":"2024-09-21T05:44:04.351055+0000","last_unstale":"2024-09-21T05:44:05.360581+0000","last_undegraded":"2024-09-21T05:44:05.360581+0000","last_fullsized":"2024-09-21T05:44:05.360581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:54.441990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352538+0000","last_change":"2024-09-21T05:44:05.350309+0000","last_active":"2024-09-21T05:44:05.352538+0000","last_peered":"2024-09-21T05:44:05.352538+0000","last_clean":"2024-09-21T05:44:05.352538+0000","last_became_active":"2024-09-21T05:44:05.349963+0000","last_became_peered":"2024-09-21T05:44:05.349963+0000","last_unstale":"2024-09-21T05:44:05.352538+0000","last_undegraded":"2024-09-21T05:44:05.352538+0000","last_fullsized":"2024-09-21T05:44:05.352538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:56:19.504906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.356597+0000","last_change":"2024-09-21T05:44:04.343110+0000","last_active":"2024-09-21T05:44:05.356597+0000","last_peered":"2024-09-21T05:44:05.356597+0000","last_clean":"2024-09-21T05:44:05.356597+0000","last_became_active":"2024-09-21T05:44:04.342707+0000","last_became_peered":"2024-09-21T05:44:04.342707+0000","last_unstale":"2024-09-21T05:44:05.356597+0000","last_undegraded":"2024-09-21T05:44:05.356597+0000","last_fullsized":"2024-09-21T05:44:05.356597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:07.581968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.354858+0000","last_change":"2024-09-21T05:44:05.349097+0000","last_active":"2024-09-21T05:44:05.354858+0000","last_peered":"2024-09-21T05:44:05.354858+0000","last_clean":"2024-09-21T05:44:05.354858+0000","last_became_active":"2024-09-21T05:44:05.348615+0000","last_became_peered":"2024-09-21T05:44:05.348615+0000","last_unstale":"2024-09-21T05:44:05.354858+0000","last_undegraded":"2024-09-21T05:44:05.354858+0000","last_fullsized":"2024-09-21T05:44:05.354858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:44.610721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362220+0000","last_change":"2024-09-21T05:44:04.343679+0000","last_active":"2024-09-21T05:44:05.362220+0000","last_peered":"2024-09-21T05:44:05.362220+0000","last_clean":"2024-09-21T05:44:05.362220+0000","last_became_active":"2024-09-21T05:44:04.342809+0000","last_became_peered":"2024-09-21T05:44:04.342809+0000","last_unstale":"2024-09-21T05:44:05.362220+0000","last_undegraded":"2024-09-21T05:44:05.362220+0000","last_fullsized":"2024-09-21T05:44:05.362220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:37.675139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344836+0000","last_change":"2024-09-21T05:44:05.345277+0000","last_active":"2024-09-21T05:44:06.344836+0000","last_peered":"2024-09-21T05:44:06.344836+0000","last_clean":"2024-09-21T05:44:06.344836+0000","last_became_active":"2024-09-21T05:44:05.344853+0000","last_became_peered":"2024-09-21T05:44:05.344853+0000","last_unstale":"2024-09-21T05:44:06.344836+0000","last_undegraded":"2024-09-21T05:44:06.344836+0000","last_fullsized":"2024-09-21T05:44:06.344836+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:47.242980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.342582+0000","last_change":"2024-09-21T05:44:04.349271+0000","last_active":"2024-09-21T05:44:05.342582+0000","last_peered":"2024-09-21T05:44:05.342582+0000","last_clean":"2024-09-21T05:44:05.342582+0000","last_became_active":"2024-09-21T05:44:04.348960+0000","last_became_peered":"2024-09-21T05:44:04.348960+0000","last_unstale":"2024-09-21T05:44:05.342582+0000","last_undegraded":"2024-09-21T05:44:05.342582+0000","last_fullsized":"2024-09-21T05:44:05.342582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:03.238378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.352960+0000","last_change":"2024-09-21T05:44:05.345213+0000","last_active":"2024-09-21T05:44:05.352960+0000","last_peered":"2024-09-21T05:44:05.352960+0000","last_clean":"2024-09-21T05:44:05.352960+0000","last_became_active":"2024-09-21T05:44:05.344501+0000","last_became_peered":"2024-09-21T05:44:05.344501+0000","last_unstale":"2024-09-21T05:44:05.352960+0000","last_undegraded":"2024-09-21T05:44:05.352960+0000","last_fullsized":"2024-09-21T05:44:05.352960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:42.592592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.341220+0000","last_change":"2024-09-21T05:44:04.347643+0000","last_active":"2024-09-21T05:44:05.341220+0000","last_peered":"2024-09-21T05:44:05.341220+0000","last_clean":"2024-09-21T05:44:05.341220+0000","last_became_active":"2024-09-21T05:44:04.347300+0000","last_became_peered":"2024-09-21T05:44:04.347300+0000","last_unstale":"2024-09-21T05:44:05.341220+0000","last_undegraded":"2024-09-21T05:44:05.341220+0000","last_fullsized":"2024-09-21T05:44:05.341220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:19.427383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.350296+0000","last_change":"2024-09-21T05:44:05.346586+0000","last_active":"2024-09-21T05:44:05.350296+0000","last_peered":"2024-09-21T05:44:05.350296+0000","last_clean":"2024-09-21T05:44:05.350296+0000","last_became_active":"2024-09-21T05:44:05.346279+0000","last_became_peered":"2024-09-21T05:44:05.346279+0000","last_unstale":"2024-09-21T05:44:05.350296+0000","last_undegraded":"2024-09-21T05:44:05.350296+0000","last_fullsized":"2024-09-21T05:44:05.350296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:19.294975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.336633+0000","last_change":"2024-09-21T05:44:04.343612+0000","last_active":"2024-09-21T05:44:05.336633+0000","last_peered":"2024-09-21T05:44:05.336633+0000","last_clean":"2024-09-21T05:44:05.336633+0000","last_became_active":"2024-09-21T05:44:04.343269+0000","last_became_peered":"2024-09-21T05:44:04.343269+0000","last_unstale":"2024-09-21T05:44:05.336633+0000","last_undegraded":"2024-09-21T05:44:05.336633+0000","last_fullsized":"2024-09-21T05:44:05.336633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:44.285360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.350431+0000","last_change":"2024-09-21T05:44:05.342070+0000","last_active":"2024-09-21T05:44:05.350431+0000","last_peered":"2024-09-21T05:44:05.350431+0000","last_clean":"2024-09-21T05:44:05.350431+0000","last_became_active":"2024-09-21T05:44:05.341731+0000","last_became_peered":"2024-09-21T05:44:05.341731+0000","last_unstale":"2024-09-21T05:44:05.350431+0000","last_undegraded":"2024-09-21T05:44:05.350431+0000","last_fullsized":"2024-09-21T05:44:05.350431+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:25.063564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.333971+0000","last_change":"2024-09-21T05:44:04.348898+0000","last_active":"2024-09-21T05:44:05.333971+0000","last_peered":"2024-09-21T05:44:05.333971+0000","last_clean":"2024-09-21T05:44:05.333971+0000","last_became_active":"2024-09-21T05:44:04.348578+0000","last_became_peered":"2024-09-21T05:44:04.348578+0000","last_unstale":"2024-09-21T05:44:05.333971+0000","last_undegraded":"2024-09-21T05:44:05.333971+0000","last_fullsized":"2024-09-21T05:44:05.333971+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:30:07.755527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.355226+0000","last_change":"2024-09-21T05:44:05.351283+0000","last_active":"2024-09-21T05:44:05.355226+0000","last_peered":"2024-09-21T05:44:05.355226+0000","last_clean":"2024-09-21T05:44:05.355226+0000","last_became_active":"2024-09-21T05:44:05.350959+0000","last_became_peered":"2024-09-21T05:44:05.350959+0000","last_unstale":"2024-09-21T05:44:05.355226+0000","last_undegraded":"2024-09-21T05:44:05.355226+0000","last_fullsized":"2024-09-21T05:44:05.355226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:38.595488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.333862+0000","last_change":"2024-09-21T05:44:04.351604+0000","last_active":"2024-09-21T05:44:05.333862+0000","last_peered":"2024-09-21T05:44:05.333862+0000","last_clean":"2024-09-21T05:44:05.333862+0000","last_became_active":"2024-09-21T05:44:04.351220+0000","last_became_peered":"2024-09-21T05:44:04.351220+0000","last_unstale":"2024-09-21T05:44:05.333862+0000","last_undegraded":"2024-09-21T05:44:05.333862+0000","last_fullsized":"2024-09-21T05:44:05.333862+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:34.908441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344660+0000","last_change":"2024-09-21T05:44:05.346613+0000","last_active":"2024-09-21T05:44:06.344660+0000","last_peered":"2024-09-21T05:44:06.344660+0000","last_clean":"2024-09-21T05:44:06.344660+0000","last_became_active":"2024-09-21T05:44:05.346224+0000","last_became_peered":"2024-09-21T05:44:05.346224+0000","last_unstale":"2024-09-21T05:44:06.344660+0000","last_undegraded":"2024-09-21T05:44:06.344660+0000","last_fullsized":"2024-09-21T05:44:06.344660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:36.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"81'2","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362330+0000","last_change":"2024-09-21T05:44:04.338665+0000","last_active":"2024-09-21T05:44:05.362330+0000","last_peered":"2024-09-21T05:44:05.362330+0000","last_clean":"2024-09-21T05:44:05.362330+0000","last_became_active":"2024-09-21T05:44:04.338352+0000","last_became_peered":"2024-09-21T05:44:04.338352+0000","last_unstale":"2024-09-21T05:44:05.362330+0000","last_undegraded":"2024-09-21T05:44:05.362330+0000","last_fullsized":"2024-09-21T05:44:05.362330+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:19.783430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.361840+0000","last_change":"2024-09-21T05:44:05.356138+0000","last_active":"2024-09-21T05:44:05.361840+0000","last_peered":"2024-09-21T05:44:05.361840+0000","last_clean":"2024-09-21T05:44:05.361840+0000","last_became_active":"2024-09-21T05:44:05.355848+0000","last_became_peered":"2024-09-21T05:44:05.355848+0000","last_unstale":"2024-09-21T05:44:05.361840+0000","last_undegraded":"2024-09-21T05:44:05.361840+0000","last_fullsized":"2024-09-21T05:44:05.361840+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:42.533823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.329531+0000","last_change":"2024-09-21T05:44:04.346344+0000","last_active":"2024-09-21T05:44:05.329531+0000","last_peered":"2024-09-21T05:44:05.329531+0000","last_clean":"2024-09-21T05:44:05.329531+0000","last_became_active":"2024-09-21T05:44:04.345922+0000","last_became_peered":"2024-09-21T05:44:04.345922+0000","last_unstale":"2024-09-21T05:44:05.329531+0000","last_undegraded":"2024-09-21T05:44:05.329531+0000","last_fullsized":"2024-09-21T05:44:05.329531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:26:54.633986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.357019+0000","last_change":"2024-09-21T05:44:05.353659+0000","last_active":"2024-09-21T05:44:05.357019+0000","last_peered":"2024-09-21T05:44:05.357019+0000","last_clean":"2024-09-21T05:44:05.357019+0000","last_became_active":"2024-09-21T05:44:05.352836+0000","last_became_peered":"2024-09-21T05:44:05.352836+0000","last_unstale":"2024-09-21T05:44:05.357019+0000","last_undegraded":"2024-09-21T05:44:05.357019+0000","last_fullsized":"2024-09-21T05:44:05.357019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:55.444437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.330794+0000","last_change":"2024-09-21T05:44:04.342072+0000","last_active":"2024-09-21T05:44:05.330794+0000","last_peered":"2024-09-21T05:44:05.330794+0000","last_clean":"2024-09-21T05:44:05.330794+0000","last_became_active":"2024-09-21T05:44:04.341437+0000","last_became_peered":"2024-09-21T05:44:04.341437+0000","last_unstale":"2024-09-21T05:44:05.330794+0000","last_undegraded":"2024-09-21T05:44:05.330794+0000","last_fullsized":"2024-09-21T05:44:05.330794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:01.352882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344963+0000","last_change":"2024-09-21T05:44:05.359063+0000","last_active":"2024-09-21T05:44:06.344963+0000","last_peered":"2024-09-21T05:44:06.344963+0000","last_clean":"2024-09-21T05:44:06.344963+0000","last_became_active":"2024-09-21T05:44:05.357828+0000","last_became_peered":"2024-09-21T05:44:05.357828+0000","last_unstale":"2024-09-21T05:44:06.344963+0000","last_undegraded":"2024-09-21T05:44:06.344963+0000","last_fullsized":"2024-09-21T05:44:06.344963+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:01.373062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.334669+0000","last_change":"2024-09-21T05:44:04.343454+0000","last_active":"2024-09-21T05:44:05.334669+0000","last_peered":"2024-09-21T05:44:05.334669+0000","last_clean":"2024-09-21T05:44:05.334669+0000","last_became_active":"2024-09-21T05:44:04.343082+0000","last_became_peered":"2024-09-21T05:44:04.343082+0000","last_unstale":"2024-09-21T05:44:05.334669+0000","last_undegraded":"2024-09-21T05:44:05.334669+0000","last_fullsized":"2024-09-21T05:44:05.334669+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:39.774961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.353296+0000","last_change":"2024-09-21T05:44:05.348065+0000","last_active":"2024-09-21T05:44:05.353296+0000","last_peered":"2024-09-21T05:44:05.353296+0000","last_clean":"2024-09-21T05:44:05.353296+0000","last_became_active":"2024-09-21T05:44:05.347767+0000","last_became_peered":"2024-09-21T05:44:05.347767+0000","last_unstale":"2024-09-21T05:44:05.353296+0000","last_undegraded":"2024-09-21T05:44:05.353296+0000","last_fullsized":"2024-09-21T05:44:05.353296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:13.813363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.338779+0000","last_change":"2024-09-21T05:44:04.350309+0000","last_active":"2024-09-21T05:44:05.338779+0000","last_peered":"2024-09-21T05:44:05.338779+0000","last_clean":"2024-09-21T05:44:05.338779+0000","last_became_active":"2024-09-21T05:44:04.349925+0000","last_became_peered":"2024-09-21T05:44:04.349925+0000","last_unstale":"2024-09-21T05:44:05.338779+0000","last_undegraded":"2024-09-21T05:44:05.338779+0000","last_fullsized":"2024-09-21T05:44:05.338779+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:35:43.942398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.359127+0000","last_change":"2024-09-21T05:44:05.358027+0000","last_active":"2024-09-21T05:44:05.359127+0000","last_peered":"2024-09-21T05:44:05.359127+0000","last_clean":"2024-09-21T05:44:05.359127+0000","last_became_active":"2024-09-21T05:44:05.357688+0000","last_became_peered":"2024-09-21T05:44:05.357688+0000","last_unstale":"2024-09-21T05:44:05.359127+0000","last_undegraded":"2024-09-21T05:44:05.359127+0000","last_fullsized":"2024-09-21T05:44:05.359127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:31:14.179685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344594+0000","last_change":"2024-09-21T05:44:04.341481+0000","last_active":"2024-09-21T05:44:06.344594+0000","last_peered":"2024-09-21T05:44:06.344594+0000","last_clean":"2024-09-21T05:44:06.344594+0000","last_became_active":"2024-09-21T05:44:04.340189+0000","last_became_peered":"2024-09-21T05:44:04.340189+0000","last_unstale":"2024-09-21T05:44:06.344594+0000","last_undegraded":"2024-09-21T05:44:06.344594+0000","last_fullsized":"2024-09-21T05:44:06.344594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:20.102950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.364848+0000","last_change":"2024-09-21T05:44:05.362339+0000","last_active":"2024-09-21T05:44:05.364848+0000","last_peered":"2024-09-21T05:44:05.364848+0000","last_clean":"2024-09-21T05:44:05.364848+0000","last_became_active":"2024-09-21T05:44:05.362056+0000","last_became_peered":"2024-09-21T05:44:05.362056+0000","last_unstale":"2024-09-21T05:44:05.364848+0000","last_undegraded":"2024-09-21T05:44:05.364848+0000","last_fullsized":"2024-09-21T05:44:05.364848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:09.074958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.335523+0000","last_change":"2024-09-21T05:44:04.343291+0000","last_active":"2024-09-21T05:44:05.335523+0000","last_peered":"2024-09-21T05:44:05.335523+0000","last_clean":"2024-09-21T05:44:05.335523+0000","last_became_active":"2024-09-21T05:44:04.342978+0000","last_became_peered":"2024-09-21T05:44:04.342978+0000","last_unstale":"2024-09-21T05:44:05.335523+0000","last_undegraded":"2024-09-21T05:44:05.335523+0000","last_fullsized":"2024-09-21T05:44:05.335523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:15.776057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.349721+0000","last_change":"2024-09-21T05:44:05.344130+0000","last_active":"2024-09-21T05:44:05.349721+0000","last_peered":"2024-09-21T05:44:05.349721+0000","last_clean":"2024-09-21T05:44:05.349721+0000","last_became_active":"2024-09-21T05:44:05.343768+0000","last_became_peered":"2024-09-21T05:44:05.343768+0000","last_unstale":"2024-09-21T05:44:05.349721+0000","last_undegraded":"2024-09-21T05:44:05.349721+0000","last_fullsized":"2024-09-21T05:44:05.349721+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:13.359400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-21T05:44:06.344128+0000","last_change":"2024-09-21T05:44:04.339456+0000","last_active":"2024-09-21T05:44:06.344128+0000","last_peered":"2024-09-21T05:44:06.344128+0000","last_clean":"2024-09-21T05:44:06.344128+0000","last_became_active":"2024-09-21T05:44:04.339043+0000","last_became_peered":"2024-09-21T05:44:04.339043+0000","last_unstale":"2024-09-21T05:44:06.344128+0000","last_undegraded":"2024-09-21T05:44:06.344128+0000","last_fullsized":"2024-09-21T05:44:06.344128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:32:28.570238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.360867+0000","last_change":"2024-09-21T05:44:05.357732+0000","last_active":"2024-09-21T05:44:05.360867+0000","last_peered":"2024-09-21T05:44:05.360867+0000","last_clean":"2024-09-21T05:44:05.360867+0000","last_became_active":"2024-09-21T05:44:05.357359+0000","last_became_peered":"2024-09-21T05:44:05.357359+0000","last_unstale":"2024-09-21T05:44:05.360867+0000","last_undegraded":"2024-09-21T05:44:05.360867+0000","last_fullsized":"2024-09-21T05:44:05.360867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:04.318387+0000","last_clean_scrub_stamp":"2024-09-21T05:44:04.318387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:35:00.479100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.362005+0000","last_change":"2024-09-21T05:44:04.344203+0000","last_active":"2024-09-21T05:44:05.362005+0000","last_peered":"2024-09-21T05:44:05.362005+0000","last_clean":"2024-09-21T05:44:05.362005+0000","last_became_active":"2024-09-21T05:44:04.343874+0000","last_became_peered":"2024-09-21T05:44:04.343874+0000","last_unstale":"2024-09-21T05:44:05.362005+0000","last_undegraded":"2024-09-21T05:44:05.362005+0000","last_fullsized":"2024-09-21T05:44:05.362005+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:44:03.315940+0000","last_clean_scrub_stamp":"2024-09-21T05:44:03.315940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:03:18.030294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":217,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-21T05:44:05.340021+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:44:05.340021+0000","last_peered":"2024-09-21T05:44:05.340021+0000","last_clean":"2024-09-21T05:44:05.340021+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:44:05.340021+0000","last_undegraded":"2024-09-21T05:44:05.340021+0000","last_fullsized":"2024-09-21T05:44:05.340021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1585152,"data_stored":1009933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":419495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36304,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335536,"statfs":{"total":96636764160,"available":96599588864,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36300,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335540,"statfs":{"total":96636764160,"available":96599592960,"internally_reserved":0,"allocated":974848,"data_stored":412720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36896,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334944,"statfs":{"total":96636764160,"available":96598982656,"internally_reserved":0,"allocated":1585152,"data_stored":1009897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:44:09.782 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:44:09.782 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.940+0000 7f529d063640 1 -- 172.21.15.22:0/2883399061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980b9b70 msgr2=0x7f52980cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.940+0000 7f529d063640 1 --2- 172.21.15.22:0/2883399061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980b9b70 0x7f52980cf090 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f52980c8020 tx=0x7f528c02a560 comp rx=0 tx=0).stop 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 -- 172.21.15.22:0/2883399061 shutdown_connections 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 --2- 172.21.15.22:0/2883399061 >> v2:172.21.15.134:3300/0 conn(0x7f52980cf5d0 0x7f52980cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 --2- 172.21.15.22:0/2883399061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980b9b70 0x7f52980cf090 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 --2- 172.21.15.22:0/2883399061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980d2ed0 0x7f52980d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 -- 172.21.15.22:0/2883399061 >> 172.21.15.22:0/2883399061 conn(0x7f529808bfc0 msgr2=0x7f529809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:09.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 -- 172.21.15.22:0/2883399061 shutdown_connections 2024-09-21T05:44:09.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.941+0000 7f529d063640 1 -- 172.21.15.22:0/2883399061 wait complete. 2024-09-21T05:44:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.943+0000 7f529d063640 1 Processor -- start 2024-09-21T05:44:09.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.943+0000 7f529d063640 1 -- start start 2024-09-21T05:44:09.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529d063640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980b9b70 0x7f5298159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529d063640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 0x7f529815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529d063640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52980d2ed0 0x7f52980c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529d063640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52980d4640 con 0x7f52980cf5d0 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529d063640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52980d44c0 con 0x7f52980d2ed0 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529d063640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f52980d47c0 con 0x7f52980b9b70 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 0x7f529815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 0x7f529815d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36364/0 (socket says 172.21.15.22:36364) 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529659c640 1 -- 172.21.15.22:0/2083336889 learned_addr learned my addr 172.21.15.22:0/2083336889 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f5296d9d640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980b9b70 0x7f5298159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.944+0000 7f529759e640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52980d2ed0 0x7f52980c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529659c640 1 -- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980b9b70 msgr2=0x7f5298159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529659c640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980b9b70 0x7f5298159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529659c640 1 -- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52980d2ed0 msgr2=0x7f52980c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529659c640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52980d2ed0 0x7f52980c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529659c640 1 -- 172.21.15.22:0/2083336889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52981f61d0 con 0x7f52980cf5d0 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f5296d9d640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980b9b70 0x7f5298159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529659c640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 0x7f529815d220 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f528c034040 tx=0x7f528c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c03e030 con 0x7f52980cf5d0 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f528c003e00 con 0x7f52980cf5d0 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528c035040 con 0x7f52980cf5d0 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529815a170 con 0x7f52980cf5d0 2024-09-21T05:44:09.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.945+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52981f6750 con 0x7f52980cf5d0 2024-09-21T05:44:09.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.946+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52980d27a0 con 0x7f52980cf5d0 2024-09-21T05:44:09.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.947+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f528c041030 con 0x7f52980cf5d0 2024-09-21T05:44:09.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.947+0000 7f527f7fe640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5284078b90 0x7f528407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:09.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.948+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f528c0bb5e0 con 0x7f52980cf5d0 2024-09-21T05:44:09.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.948+0000 7f5296d9d640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5284078b90 0x7f528407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:09.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.948+0000 7f5296d9d640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5284078b90 0x7f528407b0a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5288005e10 tx=0x7f5288005da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:09.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:09.952+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f52980d27a0 con 0x7f52980cf5d0 2024-09-21T05:44:10.212 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.210+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f52980cf9e0 con 0x7f52980cf5d0 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.213+0000 7f527f7fe640 1 -- 172.21.15.22:0/2083336889 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f528c083a10 con 0x7f52980cf5d0 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:44:10.215 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5284078b90 msgr2=0x7f528407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5284078b90 0x7f528407b0a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5288005e10 tx=0x7f5288005da0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 msgr2=0x7f529815d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 0x7f529815d220 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f528c034040 tx=0x7f528c003910 comp rx=0 tx=0).stop 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 shutdown_connections 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52980d2ed0 0x7f52980c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52980cf5d0 0x7f529815d220 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5284078b90 0x7f528407b0a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 --2- 172.21.15.22:0/2083336889 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52980b9b70 0x7f5298159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 >> 172.21.15.22:0/2083336889 conn(0x7f529808bfc0 msgr2=0x7f52980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 shutdown_connections 2024-09-21T05:44:10.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.215+0000 7f529d063640 1 -- 172.21.15.22:0/2083336889 wait complete. 2024-09-21T05:44:10.229 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f94807640 1 -- 172.21.15.22:0/26119741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f7c004d40 msgr2=0x7f8f7c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f94807640 1 --2- 172.21.15.22:0/26119741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f7c004d40 0x7f8f7c005150 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c290 tx=0x7f8f7c005830 comp rx=0 tx=0).stop 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f8ed9d640 1 -- 172.21.15.22:0/26119741 reap_dead start 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f94807640 1 -- 172.21.15.22:0/26119741 shutdown_connections 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f94807640 1 -- 172.21.15.22:0/26119741 >> 172.21.15.22:0/26119741 conn(0x7f8f90072ae0 msgr2=0x7f8f90072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f94807640 1 -- 172.21.15.22:0/26119741 shutdown_connections 2024-09-21T05:44:10.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.344+0000 7f8f94807640 1 -- 172.21.15.22:0/26119741 wait complete. 2024-09-21T05:44:10.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.346+0000 7f8f94807640 1 Processor -- start 2024-09-21T05:44:10.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.346+0000 7f8f94807640 1 -- start start 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.347+0000 7f8f94807640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f7c004d40 0x7f8f90156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.347+0000 7f8f94807640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9015f6b0 0x7f8f90152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f94807640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 0x7f8f9006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f94807640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900d16d0 con 0x7f8f900cb310 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f94807640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900d0fe0 con 0x7f8f7c004d40 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f94807640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8f900d1850 con 0x7f8f9015f6b0 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 0x7f8f9006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:10.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 0x7f8f9006def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36376/0 (socket says 172.21.15.22:36376) 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 -- 172.21.15.22:0/1054600523 learned_addr learned my addr 172.21.15.22:0/1054600523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8e59c640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f7c004d40 0x7f8f90156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8dd9b640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9015f6b0 0x7f8f90152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 -- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9015f6b0 msgr2=0x7f8f90152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9015f6b0 0x7f8f90152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 -- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f7c004d40 msgr2=0x7f8f90156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f7c004d40 0x7f8f90156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 -- 172.21.15.22:0/1054600523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f900ca290 con 0x7f8f900cb310 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f8ed9d640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 0x7f8f9006def0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c3b0 tx=0x7f8f88036c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88031070 con 0x7f8f900cb310 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f880014f0 con 0x7f8f900cb310 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f900ca550 con 0x7f8f900cb310 2024-09-21T05:44:10.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.348+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88006c50 con 0x7f8f900cb310 2024-09-21T05:44:10.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.349+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f900cab10 con 0x7f8f900cb310 2024-09-21T05:44:10.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.350+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9015c6e0 con 0x7f8f900cb310 2024-09-21T05:44:10.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.350+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8f88034070 con 0x7f8f900cb310 2024-09-21T05:44:10.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.351+0000 7f8f86ffd640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f70078bc0 0x7f8f7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:10.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.351+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8f880ba820 con 0x7f8f900cb310 2024-09-21T05:44:10.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.351+0000 7f8f8e59c640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f70078bc0 0x7f8f7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:10.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.351+0000 7f8f8e59c640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f70078bc0 0x7f8f7007b0d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8f80005fd0 tx=0x7f8f80009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:10.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.355+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f88006630 con 0x7f8f900cb310 2024-09-21T05:44:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.579+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f900b7ca0 con 0x7f8f900cb310 2024-09-21T05:44:10.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.579+0000 7f8f86ffd640 1 -- 172.21.15.22:0/1054600523 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5225 (secure 0 0 0) 0x7f8f88082610 con 0x7f8f900cb310 2024-09-21T05:44:10.582 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:10.582 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":62,"btime":"2024-09-21T05:44:08:349505+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:07.731699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/2104741994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2104741994},{"type":"v1","addr":"172.21.15.134:6837","nonce":2104741994}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6720,"qdb_cluster":[6720]},"id":6}]} 2024-09-21T05:44:10.582 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 62 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.581+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f70078bc0 msgr2=0x7f8f7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.581+0000 7f8f94807640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f70078bc0 0x7f8f7007b0d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8f80005fd0 tx=0x7f8f80009500 comp rx=0 tx=0).stop 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.581+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 msgr2=0x7f8f9006def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.581+0000 7f8f94807640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 0x7f8f9006def0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800c3b0 tx=0x7f8f88036c00 comp rx=0 tx=0).stop 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.581+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 shutdown_connections 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.581+0000 7f8f94807640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8f900cb310 0x7f8f9006def0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.582+0000 7f8f94807640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8f70078bc0 0x7f8f7007b0d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.582+0000 7f8f94807640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8f9015f6b0 0x7f8f90152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.582+0000 7f8f94807640 1 --2- 172.21.15.22:0/1054600523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8f7c004d40 0x7f8f90156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:10.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.582+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 >> 172.21.15.22:0/1054600523 conn(0x7f8f90072ae0 msgr2=0x7f8f90068d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.582+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 shutdown_connections 2024-09-21T05:44:10.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:10.582+0000 7f8f94807640 1 -- 172.21.15.22:0/1054600523 wait complete. 2024-09-21T05:44:10.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:44:07.731699+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6720}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6720': {'gid': 6720, 'name': 'd', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6837/2104741994', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6835', 'nonce': 2104741994}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 2104741994}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6720, 'qdb_cluster': [6720]} 2024-09-21T05:44:10.998 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:44:10.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T05:44:11.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 -- 172.21.15.22:0/3186697658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0d1a80 msgr2=0x7fe38c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 --2- 172.21.15.22:0/3186697658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0d1a80 0x7fe38c0cf090 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe380009830 tx=0x7fe38002a510 comp rx=0 tx=0).stop 2024-09-21T05:44:11.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 -- 172.21.15.22:0/3186697658 shutdown_connections 2024-09-21T05:44:11.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 --2- 172.21.15.22:0/3186697658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe38c0cf5d0 0x7fe38c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 --2- 172.21.15.22:0/3186697658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0d1a80 0x7fe38c0cf090 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 -- 172.21.15.22:0/3186697658 >> 172.21.15.22:0/3186697658 conn(0x7fe38c08bfc0 msgr2=0x7fe38c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 -- 172.21.15.22:0/3186697658 shutdown_connections 2024-09-21T05:44:11.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.119+0000 7fe391d5e640 1 -- 172.21.15.22:0/3186697658 wait complete. 2024-09-21T05:44:11.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.120+0000 7fe391d5e640 1 Processor -- start 2024-09-21T05:44:11.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.121+0000 7fe391d5e640 1 -- start start 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe391d5e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0cf5d0 0x7fe38c151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe391d5e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe38c0d1a80 0x7fe38c160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe391d5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 0x7fe38c160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe391d5e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c162c60 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe391d5e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c162ae0 con 0x7fe38c0cf5d0 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe391d5e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe38c162de0 con 0x7fe38c0d1a80 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 0x7fe38c160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 0x7fe38c160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36390/0 (socket says 172.21.15.22:36390) 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 -- 172.21.15.22:0/214828179 learned_addr learned my addr 172.21.15.22:0/214828179 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:11.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38affd640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe38c0d1a80 0x7fe38c160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0cf5d0 0x7fe38c151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 -- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe38c0d1a80 msgr2=0x7fe38c160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe38c0d1a80 0x7fe38c160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 -- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0cf5d0 msgr2=0x7fe38c151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0cf5d0 0x7fe38c151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.122+0000 7fe38bfff640 1 -- 172.21.15.22:0/214828179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c0caec0 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.123+0000 7fe38bfff640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 0x7fe38c160f60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe384009f80 tx=0x7fe38400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.123+0000 7fe373fff640 1 -- 172.21.15.22:0/214828179 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38401b070 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.123+0000 7fe373fff640 1 -- 172.21.15.22:0/214828179 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe384003b80 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.123+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c0cabe0 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.123+0000 7fe373fff640 1 -- 172.21.15.22:0/214828179 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384004030 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.123+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c0c84e0 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.125+0000 7fe373fff640 1 -- 172.21.15.22:0/214828179 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe38401e070 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.125+0000 7fe373fff640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe378078b90 0x7fe37807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.125+0000 7fe373fff640 1 -- 172.21.15.22:0/214828179 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe38409b4e0 con 0x7fe38c0d2ed0 2024-09-21T05:44:11.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.126+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe378078b90 0x7fe37807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.126+0000 7fe398ccd740 1 Processor -- start 2024-09-21T05:44:11.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.126+0000 7fe398ccd740 1 -- start start 2024-09-21T05:44:11.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.126+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe378078b90 0x7fe37807b0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe37c001740 tx=0x7fe37c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.127+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b2275cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.127+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.127+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275e83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.127+0000 7fe398ccd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b227513820 con 0x55b2271d8590 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.127+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b227502c50 con 0x55b2271d03b0 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.127+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b227506a60 con 0x55b2271f9270 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275e83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275e83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36400/0 (socket says 172.21.15.22:36400) 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 -- 172.21.15.22:0/1133488080 learned_addr learned my addr 172.21.15.22:0/1133488080 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b2275cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38affd640 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 -- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 msgr2=0x55b2275cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b2275cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 -- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 msgr2=0x55b2275cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 -- 172.21.15.22:0/1133488080 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b227264850 con 0x55b2271d8590 2024-09-21T05:44:11.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe38bfff640 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275e83e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe38405f2f0 tx=0x7fe38405f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7907c1aaf45a40d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe3717fa640 1 -- 172.21.15.22:0/1133488080 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384067c70 con 0x55b2271d8590 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe3717fa640 1 -- 172.21.15.22:0/1133488080 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe38406cbb0 con 0x55b2271d8590 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.128+0000 7fe3717fa640 1 -- 172.21.15.22:0/1133488080 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38409b2c0 con 0x55b2271d8590 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 -- 172.21.15.22:0/1133488080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 msgr2=0x55b2275e83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275e83e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe38405f2f0 tx=0x7fe38405f210 comp rx=0 tx=0).stop 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 -- 172.21.15.22:0/1133488080 shutdown_connections 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275e83e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1133488080 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b2275cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 -- 172.21.15.22:0/1133488080 >> 172.21.15.22:0/1133488080 conn(0x55b2271ed3c0 msgr2=0x55b2275a1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 -- 172.21.15.22:0/1133488080 shutdown_connections 2024-09-21T05:44:11.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.129+0000 7fe398ccd740 1 -- 172.21.15.22:0/1133488080 wait complete. 2024-09-21T05:44:11.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.130+0000 7fe398ccd740 1 -- start start 2024-09-21T05:44:11.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.130+0000 7fe398ccd740 1 Processor -- start 2024-09-21T05:44:11.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d03b0 0x55b2275f5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275e79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b227273620 con 0x55b2271d8590 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2274a85f0 con 0x55b2271f9270 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b2275cba20 con 0x55b2271d03b0 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe398ccd740 10 client.0 _ll_register_callbacks cb 0x55b2274ac0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275b93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36404/0 (socket says 172.21.15.22:36404) 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 -- 172.21.15.22:0/770941624 learned_addr learned my addr 172.21.15.22:0/770941624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d03b0 0x55b2275f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38bfff640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275e79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 -- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d03b0 msgr2=0x55b2275f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d03b0 0x55b2275f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 -- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 msgr2=0x55b2275e79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275e79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 -- 172.21.15.22:0/770941624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2272db6b0 con 0x55b2271d8590 2024-09-21T05:44:11.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38bfff640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe38affd640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275b93e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe38002ab00 tx=0x7fe380009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8178fd7b56b186e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe36a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe36a7fc640 1 -- 172.21.15.22:0/770941624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380003b40 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.132+0000 7fe36a7fc640 1 -- 172.21.15.22:0/770941624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe380003ce0 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe38406cbb0 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe36a7fc640 1 -- 172.21.15.22:0/770941624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fe380005020 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 10 client.6846 fetch_fsmap learned FSMap version 62 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55b2275dd5f0 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe36a7fc640 1 -- 172.21.15.22:0/770941624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe38406cbb0 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe36a7fc640 1 -- 172.21.15.22:0/770941624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 62) ==== 3502+0+0 (secure 0 0 0) 0x7fe38004e050 con 0x55b2271d8590 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 10 client.6846 fetch_fsmap finished waiting for FSMap version 62 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 10 client.6846 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 20 client.6846 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:11.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2271b7e40 0x55b2275e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 4 client.6846 mds_command: new command op to 6720 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 --> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b227515770 con 0x55b2271b7e40 2024-09-21T05:44:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.133+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2271b7e40 0x55b2275e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.134+0000 7fe36a7fc640 10 client.6846 ms_handle_connect on v2:172.21.15.134:6835/2104741994 2024-09-21T05:44:11.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.134+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2271b7e40 0x55b2275e0660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccfef28337ce85a1 server_cookie=4e557138dda43c6d in_seq=0 out_seq=0 2024-09-21T05:44:11.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe36a7fc640 1 -- 172.21.15.22:0/770941624 <== mds.0 v2:172.21.15.134:6835/2104741994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b227515770 con 0x55b2271b7e40 2024-09-21T05:44:11.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe36a7fc640 10 client.6846 handle_command_reply: tid=0 2024-09-21T05:44:11.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 client.6846 shutdown 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 msgr2=0x55b2275b93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275b93e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fe38002ab00 tx=0x7fe380009550 comp rx=0 tx=0).stop 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 shutdown_connections 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275e79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2271b7e40 0x55b2275e0660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b2275b93e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 --2- 172.21.15.22:0/770941624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d03b0 0x55b2275f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe36a7fc640 0 client.6846 ms_handle_reset on v2:172.21.15.134:6835/2104741994 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 >> 172.21.15.22:0/770941624 conn(0x55b2271ed3c0 msgr2=0x55b22759a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 shutdown_connections 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 1 -- 172.21.15.22:0/770941624 wait complete. 2024-09-21T05:44:11.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.135+0000 7fe398ccd740 20 client.6846 trim_cache size 0 max 16384 2024-09-21T05:44:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.153+0000 7fe398ccd740 1 Processor -- start 2024-09-21T05:44:11.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.153+0000 7fe398ccd740 1 -- start start 2024-09-21T05:44:11.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 0x55b2275dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d8590 0x55b2275c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe398ccd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c0b89b0 con 0x55b2271d03b0 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c0b1df0 con 0x55b2271f9270 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe38c0b8360 con 0x55b2271d8590 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 0x55b2275dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 0x55b2275dfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36416/0 (socket says 172.21.15.22:36416) 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 -- 172.21.15.22:0/2997101755 learned_addr learned my addr 172.21.15.22:0/2997101755 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38bfff640 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 -- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d8590 msgr2=0x55b2275c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d8590 0x55b2275c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 -- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 msgr2=0x55b2275c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 -- 172.21.15.22:0/2997101755 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2274c1ae0 con 0x55b2271d03b0 2024-09-21T05:44:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.155+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 0x55b2275dfc90 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00f5a0 tx=0x7fe37c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc809a038c1b329 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe370ff9640 1 -- 172.21.15.22:0/2997101755 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c02f070 con 0x55b2271d03b0 2024-09-21T05:44:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe370ff9640 1 -- 172.21.15.22:0/2997101755 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe37c01b040 con 0x55b2271d03b0 2024-09-21T05:44:11.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe370ff9640 1 -- 172.21.15.22:0/2997101755 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00fc90 con 0x55b2271d03b0 2024-09-21T05:44:11.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 -- 172.21.15.22:0/2997101755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 msgr2=0x55b2275dfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 0x55b2275dfc90 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00f5a0 tx=0x7fe37c014000 comp rx=0 tx=0).stop 2024-09-21T05:44:11.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 -- 172.21.15.22:0/2997101755 shutdown_connections 2024-09-21T05:44:11.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271f9270 0x55b2275c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271d8590 0x55b2275c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 --2- 172.21.15.22:0/2997101755 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d03b0 0x55b2275dfc90 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 -- 172.21.15.22:0/2997101755 >> 172.21.15.22:0/2997101755 conn(0x55b2271ed3c0 msgr2=0x55b2275e7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 -- 172.21.15.22:0/2997101755 shutdown_connections 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.156+0000 7fe398ccd740 1 -- 172.21.15.22:0/2997101755 wait complete. 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.157+0000 7fe398ccd740 1 -- start start 2024-09-21T05:44:11.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.157+0000 7fe398ccd740 1 Processor -- start 2024-09-21T05:44:11.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b22765c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b227666710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b2275e81f0 con 0x55b2271d8590 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b2275e1800 con 0x55b2271d03b0 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b2275f8630 con 0x55b2271f9270 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 10 client.0 _ll_register_callbacks cb 0x7fe3800096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b22765c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b22765c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36428/0 (socket says 172.21.15.22:36428) 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 -- 172.21.15.22:0/1560928326 learned_addr learned my addr 172.21.15.22:0/1560928326 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38bfff640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b227666710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 -- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 msgr2=0x55b227666710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b227666710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 -- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 msgr2=0x55b2275f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 -- 172.21.15.22:0/1560928326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2275e7fa0 con 0x55b2271d8590 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe38affd640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b22765c730 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe380009830 tx=0x7fe380004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b42a3b4fb308b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe36a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe36a7fc640 1 -- 172.21.15.22:0/1560928326 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe380003b50 con 0x55b2271d8590 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe36a7fc640 1 -- 172.21.15.22:0/1560928326 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe380053730 con 0x55b2271d8590 2024-09-21T05:44:11.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.159+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe37c01b040 con 0x55b2271d8590 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe36a7fc640 1 -- 172.21.15.22:0/1560928326 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fe380003cf0 con 0x55b2271d8590 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 10 client.6852 fetch_fsmap learned FSMap version 62 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) -- 0x55b2275dc750 con 0x55b2271d8590 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe36a7fc640 1 -- 172.21.15.22:0/1560928326 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe37c01b040 con 0x55b2271d8590 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe36a7fc640 1 -- 172.21.15.22:0/1560928326 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 62) ==== 3502+0+0 (secure 0 0 0) 0x7fe38004e040 con 0x55b2271d8590 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 10 client.6852 fetch_fsmap finished waiting for FSMap version 62 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 10 client.6852 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 20 client.6852 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2275c7fe0 0x55b22759a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 4 client.6852 mds_command: new command op to 6720 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 --> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] -- command(tid 0: {"prefix": "status"}) -- 0x55b227511220 con 0x55b2275c7fe0 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.160+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2275c7fe0 0x55b22759a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe36a7fc640 10 client.6852 ms_handle_connect on v2:172.21.15.134:6835/2104741994 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe38b7fe640 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2275c7fe0 0x55b22759a200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2799343cd804bae7 server_cookie=dd5b7e19cdfd3510 in_seq=0 out_seq=0 2024-09-21T05:44:11.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe36a7fc640 1 -- 172.21.15.22:0/1560928326 <== mds.0 v2:172.21.15.134:6835/2104741994 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b227511220 con 0x55b2275c7fe0 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe36a7fc640 10 client.6852 handle_command_reply: tid=0 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 client.6852 shutdown 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 msgr2=0x55b22765c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b22765c730 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe380009830 tx=0x7fe380004610 comp rx=0 tx=0).stop 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 shutdown_connections 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b2271f9270 0x55b227666710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:6835/2104741994,v1:172.21.15.134:6837/2104741994] conn(0x55b2275c7fe0 0x55b22759a200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b2271d8590 0x55b22765c730 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 --2- 172.21.15.22:0/1560928326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b2271d03b0 0x55b2275f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe36a7fc640 0 client.6852 ms_handle_reset on v2:172.21.15.134:6835/2104741994 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 >> 172.21.15.22:0/1560928326 conn(0x55b2271ed3c0 msgr2=0x55b2275e7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 shutdown_connections 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 1 -- 172.21.15.22:0/1560928326 wait complete. 2024-09-21T05:44:11.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.161+0000 7fe398ccd740 20 client.6852 trim_cache size 0 max 16384 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 6720, 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.8356210050000001, 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 62, 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 82, 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 82, 2024-09-21T05:44:11.169 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 10.456123354000001 2024-09-21T05:44:11.170 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe378078b90 msgr2=0x7fe37807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe378078b90 0x7fe37807b0a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe37c001740 tx=0x7fe37c009040 comp rx=0 tx=0).stop 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 msgr2=0x7fe38c160f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 0x7fe38c160f60 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe384009f80 tx=0x7fe38400c800 comp rx=0 tx=0).stop 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 shutdown_connections 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe38c0d2ed0 0x7fe38c160f60 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe378078b90 0x7fe37807b0a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe38c0d1a80 0x7fe38c160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 --2- 172.21.15.22:0/214828179 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe38c0cf5d0 0x7fe38c151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 >> 172.21.15.22:0/214828179 conn(0x7fe38c08bfc0 msgr2=0x7fe38c06ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 shutdown_connections 2024-09-21T05:44:11.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.162+0000 7fe391d5e640 1 -- 172.21.15.22:0/214828179 wait complete. 2024-09-21T05:44:11.175 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:44:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93eeed3640 1 -- 172.21.15.22:0/4057389195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93d8004020 msgr2=0x7f93d8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4057389195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93d8004020 0x7f93d8004800 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009b20 tx=0x7f93d8005830 comp rx=0 tx=0).stop 2024-09-21T05:44:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93ed470640 1 -- 172.21.15.22:0/4057389195 reap_dead start 2024-09-21T05:44:11.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93eeed3640 1 -- 172.21.15.22:0/4057389195 shutdown_connections 2024-09-21T05:44:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93eeed3640 1 -- 172.21.15.22:0/4057389195 >> 172.21.15.22:0/4057389195 conn(0x7f93e808bfc0 msgr2=0x7f93e809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93eeed3640 1 -- 172.21.15.22:0/4057389195 shutdown_connections 2024-09-21T05:44:11.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.290+0000 7f93eeed3640 1 -- 172.21.15.22:0/4057389195 wait complete. 2024-09-21T05:44:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.292+0000 7f93eeed3640 1 Processor -- start 2024-09-21T05:44:11.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.292+0000 7f93eeed3640 1 -- start start 2024-09-21T05:44:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93eeed3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d8004020 0x7f93e81541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93eeed3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81501d0 0x7f93e81505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93eeed3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 0x7f93e80c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93eeed3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80cf200 con 0x7f93e8150b20 2024-09-21T05:44:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93eeed3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80cf080 con 0x7f93e81501d0 2024-09-21T05:44:11.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93eeed3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f93e80cf380 con 0x7f93d8004020 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93ed470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 0x7f93e80c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93ed470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 0x7f93e80c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36450/0 (socket says 172.21.15.22:36450) 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.293+0000 7f93ed470640 1 -- 172.21.15.22:0/4061659601 learned_addr learned my addr 172.21.15.22:0/4061659601 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ecc6f640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d8004020 0x7f93e81541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93e7fff640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81501d0 0x7f93e81505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ed470640 1 -- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d8004020 msgr2=0x7f93e81541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ed470640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d8004020 0x7f93e81541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ed470640 1 -- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81501d0 msgr2=0x7f93e81505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ed470640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81501d0 0x7f93e81505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ed470640 1 -- 172.21.15.22:0/4061659601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e81f8c30 con 0x7f93e8150b20 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93ed470640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 0x7f93e80c87e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005fd0 tx=0x7f93e000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0004b20 con 0x7f93e8150b20 2024-09-21T05:44:11.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93e0004cc0 con 0x7f93e8150b20 2024-09-21T05:44:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81f8f10 con 0x7f93e8150b20 2024-09-21T05:44:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0003bd0 con 0x7f93e8150b20 2024-09-21T05:44:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.294+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81f94d0 con 0x7f93e8150b20 2024-09-21T05:44:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.295+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e806e170 con 0x7f93e8150b20 2024-09-21T05:44:11.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.296+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93e0004030 con 0x7f93e8150b20 2024-09-21T05:44:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.296+0000 7f93e57fa640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93cc078b90 0x7f93cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.296+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f93e009b320 con 0x7f93e8150b20 2024-09-21T05:44:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.297+0000 7f93ecc6f640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93cc078b90 0x7f93cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:11.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.297+0000 7f93ecc6f640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93cc078b90 0x7f93cc07b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f93e80d2780 tx=0x7f93dc001590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:11.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.301+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93e00636c0 con 0x7f93e8150b20 2024-09-21T05:44:11.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.530+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93e80d1180 con 0x7f93e8150b20 2024-09-21T05:44:11.533 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.531+0000 7f93e57fa640 1 -- 172.21.15.22:0/4061659601 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) ==== 53+0+73 (secure 0 0 0) 0x7f93e006ae20 con 0x7f93e8150b20 2024-09-21T05:44:11.533 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:44:11.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93cc078b90 msgr2=0x7f93cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93cc078b90 0x7f93cc07b0a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f93e80d2780 tx=0x7f93dc001590 comp rx=0 tx=0).stop 2024-09-21T05:44:11.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 msgr2=0x7f93e80c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:11.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 0x7f93e80c87e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005fd0 tx=0x7f93e000f3c0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 shutdown_connections 2024-09-21T05:44:11.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f93e8150b20 0x7f93e80c87e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f93e81501d0 0x7f93e81505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f93cc078b90 0x7f93cc07b0a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.537 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 --2- 172.21.15.22:0/4061659601 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f93d8004020 0x7f93e81541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:11.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 >> 172.21.15.22:0/4061659601 conn(0x7f93e808bfc0 msgr2=0x7f93e8161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:11.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 shutdown_connections 2024-09-21T05:44:11.538 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11.533+0000 7f93eeed3640 1 -- 172.21.15.22:0/4061659601 wait complete. 2024-09-21T05:44:11.547 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:44:11.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:44:11.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:44:11.548 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:44:11.548 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:11.548 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:44:11.548 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:44:11.548 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:44:11.548 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:44:11.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:11.549 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:44:11.565 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42492sec preferred_lft 42492sec 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:44:11.566 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:44:11.567 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:11.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:11.568 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:44:11.629 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:44:11.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:11.631 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:44:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:44:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:44:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:44:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:44:11.687 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:44:11.688 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:44:11.688 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:44:11.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:44:11.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:44:11.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:44:11.688 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:44:11.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:44:11.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:44:11.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:11.815 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:44:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:44:11.831 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:44:11.831 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:44:11.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:11.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:44:11.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:44:11.897 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:11.952 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:44:11.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:44:11.953 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:12.008 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:44:12.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:12.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:12.103 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:12.103 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:44:12.103 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:44:12.114 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:12.114 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:44:12.114 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:44:12.115 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:44:12.115 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:44:12.121 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:44:12.122 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:44:12.122 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:44:12.122 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:44:12.122 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:44:12.122 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:44:12.123 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:44:12.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:12.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:12.150 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:44:12.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:44:12.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:44:12.254 INFO:teuthology.orchestra.run.smithi022.stdout:client6861 2024-09-21T05:44:12.254 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:44:12.254 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:12.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:12.270 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-09-21T05:44:12.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:12.333 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-09-21T05:44:12.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:12.391 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-21T05:44:12.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:12.449 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-09-21T05:44:12.510 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:44:12.510 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:44:12.510 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:44:12.511 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:44:12.511 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-09-21T05:44:12.511 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:44:12.617 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:44:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:44:12.626 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:44:12.626 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:12.682 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:44:12.682 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:12.739 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:44:12.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 -- 172.21.15.22:0/2734556693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0b97e0 msgr2=0x7f3b7c06f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:12.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 --2- 172.21.15.22:0/2734556693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0b97e0 0x7f3b7c06f6e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005fd0 tx=0x7f3b7002a4b0 comp rx=0 tx=0).stop 2024-09-21T05:44:12.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 -- 172.21.15.22:0/2734556693 shutdown_connections 2024-09-21T05:44:12.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 --2- 172.21.15.22:0/2734556693 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0b97e0 0x7f3b7c06f6e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:12.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 -- 172.21.15.22:0/2734556693 >> 172.21.15.22:0/2734556693 conn(0x7f3b7c09d840 msgr2=0x7f3b7c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:12.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 -- 172.21.15.22:0/2734556693 shutdown_connections 2024-09-21T05:44:12.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.894+0000 7f3b82a94640 1 -- 172.21.15.22:0/2734556693 wait complete. 2024-09-21T05:44:12.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.896+0000 7f3b82a94640 1 Processor -- start 2024-09-21T05:44:12.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.896+0000 7f3b82a94640 1 -- start start 2024-09-21T05:44:12.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b82a94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b7c06e0a0 0x7f3b7c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:12.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b82a94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b7c0b97e0 0x7f3b7c1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:12.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b82a94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 0x7f3b7c1f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:12.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b82a94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0cff00 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b82a94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c0cfd80 con 0x7f3b7c06e0a0 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b82a94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7c0d0080 con 0x7f3b7c0b97e0 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b81031640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 0x7f3b7c1f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b81031640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 0x7f3b7c1f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36468/0 (socket says 172.21.15.22:36468) 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b81031640 1 -- 172.21.15.22:0/3834573242 learned_addr learned my addr 172.21.15.22:0/3834573242 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b7bfff640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b7c0b97e0 0x7f3b7c1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b80830640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b7c06e0a0 0x7f3b7c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b81031640 1 -- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b7c0b97e0 msgr2=0x7f3b7c1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.897+0000 7f3b81031640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b7c0b97e0 0x7f3b7c1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b81031640 1 -- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b7c06e0a0 msgr2=0x7f3b7c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b81031640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b7c06e0a0 0x7f3b7c15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b81031640 1 -- 172.21.15.22:0/3834573242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c1f9a30 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b80830640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b7c06e0a0 0x7f3b7c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b81031640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 0x7f3b7c1f8bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3b74002230 tx=0x7f3b7400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:12.901 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74003b70 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b74003d10 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7c0b5d80 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b74005330 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.898+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b7c0b6340 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.899+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7c0d39a0 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.900+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3b740056e0 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.900+0000 7f3b797fa640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3b68078bc0 0x7f3b6807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:12.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.901+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f3b7409abd0 con 0x7f3b7c0d0e50 2024-09-21T05:44:12.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.901+0000 7f3b80830640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3b68078bc0 0x7f3b6807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:12.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.901+0000 7f3b80830640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3b68078bc0 0x7f3b6807b0d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009890 tx=0x7f3b6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:12.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:12.905+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b7c0d39a0 con 0x7f3b7c0d0e50 2024-09-21T05:44:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.107+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b7c0d0450 con 0x7f3b7c0d0e50 2024-09-21T05:44:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.107+0000 7f3b797fa640 1 -- 172.21.15.22:0/3834573242 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) ==== 80+0+0 (secure 0 0 0) 0x7f3b7c1f9a30 con 0x7f3b7c0d0e50 2024-09-21T05:44:13.109 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3b68078bc0 msgr2=0x7f3b6807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3b68078bc0 0x7f3b6807b0d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009890 tx=0x7f3b6c008040 comp rx=0 tx=0).stop 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 msgr2=0x7f3b7c1f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 0x7f3b7c1f8bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3b74002230 tx=0x7f3b7400cb90 comp rx=0 tx=0).stop 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 shutdown_connections 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3b7c0d0e50 0x7f3b7c1f8bd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3b68078bc0 0x7f3b6807b0d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3b7c0b97e0 0x7f3b7c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 --2- 172.21.15.22:0/3834573242 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3b7c06e0a0 0x7f3b7c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.109+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 >> 172.21.15.22:0/3834573242 conn(0x7f3b7c09d840 msgr2=0x7f3b7c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.110+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 shutdown_connections 2024-09-21T05:44:13.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.110+0000 7f3b82a94640 1 -- 172.21.15.22:0/3834573242 wait complete. 2024-09-21T05:44:13.123 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:44:13.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:13.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:13.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:44:13.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:13.175 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:44:13.201 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.358+0000 7f6aa8877640 1 -- 172.21.15.22:0/3539975032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa406cde0 msgr2=0x7f6aa406d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.358+0000 7f6aa8877640 1 --2- 172.21.15.22:0/3539975032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa406cde0 0x7f6aa406d1f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6a98005fd0 tx=0x7f6a9802a540 comp rx=0 tx=0).stop 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.359+0000 7f6aa8877640 1 -- 172.21.15.22:0/3539975032 shutdown_connections 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.359+0000 7f6aa8877640 1 --2- 172.21.15.22:0/3539975032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa406cde0 0x7f6aa406d1f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.359+0000 7f6aa8877640 1 --2- 172.21.15.22:0/3539975032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6aa409aa30 0x7f6aa415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.359+0000 7f6aa8877640 1 -- 172.21.15.22:0/3539975032 >> 172.21.15.22:0/3539975032 conn(0x7f6aa4074170 msgr2=0x7f6aa40b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.359+0000 7f6aa8877640 1 -- 172.21.15.22:0/3539975032 shutdown_connections 2024-09-21T05:44:13.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.359+0000 7f6aa8877640 1 -- 172.21.15.22:0/3539975032 wait complete. 2024-09-21T05:44:13.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.361+0000 7f6aa8877640 1 Processor -- start 2024-09-21T05:44:13.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.361+0000 7f6aa8877640 1 -- start start 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa8877640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6aa406cde0 0x7f6aa40b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa8877640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aa409aa30 0x7f6aa41f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa8877640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 0x7f6aa41e92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa8877640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa40cd6e0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa8877640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa40b4cf0 con 0x7f6aa409aa30 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa8877640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa409cd70 con 0x7f6aa406cde0 2024-09-21T05:44:13.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 0x7f6aa41e92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa259c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6aa406cde0 0x7f6aa40b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 0x7f6aa41e92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36478/0 (socket says 172.21.15.22:36478) 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 -- 172.21.15.22:0/2754476834 learned_addr learned my addr 172.21.15.22:0/2754476834 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa1d9b640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aa409aa30 0x7f6aa41f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 -- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6aa406cde0 msgr2=0x7f6aa40b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6aa406cde0 0x7f6aa40b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 -- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aa409aa30 msgr2=0x7f6aa41f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aa409aa30 0x7f6aa41f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.362+0000 7f6aa2d9d640 1 -- 172.21.15.22:0/2754476834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa41f67c0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.363+0000 7f6aa2d9d640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 0x7f6aa41e92e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c810 tx=0x7f6a9c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.363+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00ce60 con 0x7f6aa40b41d0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.363+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a9c003b90 con 0x7f6aa40b41d0 2024-09-21T05:44:13.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.363+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa409e9c0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.363+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9c00a850 con 0x7f6aa40b41d0 2024-09-21T05:44:13.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.363+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa409ef80 con 0x7f6aa40b41d0 2024-09-21T05:44:13.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.364+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa40b5280 con 0x7f6aa40b41d0 2024-09-21T05:44:13.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.365+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a9c00a9f0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.365+0000 7f6a92ffd640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a84078bc0 0x7f6a8407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.365+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f6a9c09a9d0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.365+0000 7f6aa259c640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a84078bc0 0x7f6a8407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.366+0000 7f6aa259c640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a84078bc0 0x7f6a8407b0d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009b30 tx=0x7f6a94005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:13.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.370+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6aa41f67c0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.595+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aa415b2c0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.595+0000 7f6a92ffd640 1 -- 172.21.15.22:0/2754476834 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5225 (secure 0 0 0) 0x7f6aa415cda0 con 0x7f6aa40b41d0 2024-09-21T05:44:13.598 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:13.598 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":62,"btime":"2024-09-21T05:44:08:349505+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":6708,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:07.731699+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6720},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6720":{"gid":6720,"name":"d","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/2104741994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2104741994},{"type":"v1","addr":"172.21.15.134:6837","nonce":2104741994}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6720,"qdb_cluster":[6720]},"id":6}]} 2024-09-21T05:44:13.598 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 62 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a84078bc0 msgr2=0x7f6a8407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a84078bc0 0x7f6a8407b0d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009b30 tx=0x7f6a94005f90 comp rx=0 tx=0).stop 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 msgr2=0x7f6aa41e92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 0x7f6aa41e92e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00c810 tx=0x7f6a9c00cc10 comp rx=0 tx=0).stop 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 shutdown_connections 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6aa40b41d0 0x7f6aa41e92e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6aa409aa30 0x7f6aa41f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a84078bc0 0x7f6a8407b0d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.597+0000 7f6aa8877640 1 --2- 172.21.15.22:0/2754476834 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6aa406cde0 0x7f6aa40b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.598+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 >> 172.21.15.22:0/2754476834 conn(0x7f6aa4074170 msgr2=0x7f6aa4070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:13.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.598+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 shutdown_connections 2024-09-21T05:44:13.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.598+0000 7f6aa8877640 1 -- 172.21.15.22:0/2754476834 wait complete. 2024-09-21T05:44:13.611 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-09-21T05:44:13.744 INFO:teuthology.orchestra.run.smithi134.stdout:{"message":"","return_code":0} 2024-09-21T05:44:13.744 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:44:13.745 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.860+0000 7f116d913640 1 -- 172.21.15.22:0/2471801112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 msgr2=0x7f11680ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.860+0000 7f116d913640 1 --2- 172.21.15.22:0/2471801112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 0x7f11680ceb50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1150009860 tx=0x7f115002a530 comp rx=0 tx=0).stop 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.860+0000 7f116d913640 1 -- 172.21.15.22:0/2471801112 shutdown_connections 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.860+0000 7f116d913640 1 --2- 172.21.15.22:0/2471801112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 0x7f11680ceb50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.860+0000 7f116d913640 1 -- 172.21.15.22:0/2471801112 >> 172.21.15.22:0/2471801112 conn(0x7f11680b3f10 msgr2=0x7f11680b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.861+0000 7f116d913640 1 -- 172.21.15.22:0/2471801112 shutdown_connections 2024-09-21T05:44:13.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.861+0000 7f116d913640 1 -- 172.21.15.22:0/2471801112 wait complete. 2024-09-21T05:44:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.862+0000 7f116d913640 1 Processor -- start 2024-09-21T05:44:13.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.862+0000 7f116d913640 1 -- start start 2024-09-21T05:44:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f116d913640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 0x7f116815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f116d913640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 0x7f1168149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f116d913640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11680d1d40 0x7f1168149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f116d913640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f11680d1520 con 0x7f11680ccba0 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f116d913640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f11680d13a0 con 0x7f11680d1d40 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f116d913640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f11680d16a0 con 0x7f11680b5210 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 0x7f1168149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 0x7f1168149900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36482/0 (socket says 172.21.15.22:36482) 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 -- 172.21.15.22:0/2793278322 learned_addr learned my addr 172.21.15.22:0/2793278322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f1166ffd640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 0x7f116815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11677fe640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11680d1d40 0x7f1168149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 -- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 msgr2=0x7f116815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 0x7f116815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 -- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11680d1d40 msgr2=0x7f1168149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:13.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11680d1d40 0x7f1168149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.864+0000 7f11667fc640 1 -- 172.21.15.22:0/2793278322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116815f6d0 con 0x7f11680ccba0 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.865+0000 7f11667fc640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 0x7f1168149900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f115002ab20 tx=0x7f1150003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.865+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150005c90 con 0x7f11680ccba0 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.865+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1150005020 con 0x7f11680ccba0 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.865+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11681f6d40 con 0x7f11680ccba0 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.865+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150005410 con 0x7f11680ccba0 2024-09-21T05:44:13.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.865+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11681f7300 con 0x7f11680ccba0 2024-09-21T05:44:13.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.866+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11680ceb50 con 0x7f11680ccba0 2024-09-21T05:44:13.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.866+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11500317c0 con 0x7f11680ccba0 2024-09-21T05:44:13.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.867+0000 7f115ffff640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1148078bc0 0x7f114807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:13.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.867+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f11500bbbf0 con 0x7f11680ccba0 2024-09-21T05:44:13.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.867+0000 7f1166ffd640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1148078bc0 0x7f114807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:13.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.868+0000 7f1166ffd640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1148078bc0 0x7f114807b0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1158001740 tx=0x7f1158001aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:13.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:13.871+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f115003f030 con 0x7f11680ccba0 2024-09-21T05:44:14.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.099+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f11680b2360 con 0x7f11680ccba0 2024-09-21T05:44:14.102 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:44:14.100+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:44:14.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.101+0000 7f115ffff640 1 -- 172.21.15.22:0/2793278322 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7f11500839e0 con 0x7f11680ccba0 2024-09-21T05:44:14.103 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:44:14.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1148078bc0 msgr2=0x7f114807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1148078bc0 0x7f114807b0d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1158001740 tx=0x7f1158001aa0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 msgr2=0x7f1168149900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 0x7f1168149900 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f115002ab20 tx=0x7f1150003c10 comp rx=0 tx=0).stop 2024-09-21T05:44:14.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 shutdown_connections 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f11680d1d40 0x7f1168149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11680ccba0 0x7f1168149900 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1148078bc0 0x7f114807b0d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 --2- 172.21.15.22:0/2793278322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11680b5210 0x7f116815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 >> 172.21.15.22:0/2793278322 conn(0x7f11680b3f10 msgr2=0x7f1168068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 shutdown_connections 2024-09-21T05:44:14.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.104+0000 7f116d913640 1 -- 172.21.15.22:0/2793278322 wait complete. 2024-09-21T05:44:14.109 INFO:tasks.ceph.mds.d.smithi134.stderr: -5638> 2024-09-21T05:44:07.306+0000 7fefef5d7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:44:14.117 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-21T05:44:14.164 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:44:14.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.170+0000 7f8c81e16f40 1 Processor -- start 2024-09-21T05:44:14.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.170+0000 7f8c81e16f40 1 -- start start 2024-09-21T05:44:14.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.171+0000 7f8c81e16f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf7fbd90 0x55b1bf8c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.171+0000 7f8c81e16f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf8d21d0 0x55b1bf829b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.171+0000 7f8c81e16f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 0x55b1bf8287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.171+0000 7f8c81e16f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1bf7ab430 con 0x55b1bf8d4c10 2024-09-21T05:44:14.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.171+0000 7f8c81e16f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1bf7b81f0 con 0x55b1bf8d21d0 2024-09-21T05:44:14.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c81e16f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b1bf74cd60 con 0x55b1bf7fbd90 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 0x55b1bf8287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 0x55b1bf8287b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36490/0 (socket says 172.21.15.22:36490) 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 -- 172.21.15.22:0/1400344570 learned_addr learned my addr 172.21.15.22:0/1400344570 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c73fff640 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf8d21d0 0x55b1bf829b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7b8d0640 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf7fbd90 0x55b1bf8c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 -- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf7fbd90 msgr2=0x55b1bf8c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf7fbd90 0x55b1bf8c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 -- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf8d21d0 msgr2=0x55b1bf829b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf8d21d0 0x55b1bf829b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 -- 172.21.15.22:0/1400344570 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1bf8b6dd0 con 0x55b1bf8d4c10 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7c0d1640 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 0x55b1bf8287b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8c74001620 tx=0x7f8c74001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2829858b5b5767ac server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7a8ce640 1 -- 172.21.15.22:0/1400344570 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400f710 con 0x55b1bf8d4c10 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7a8ce640 1 -- 172.21.15.22:0/1400344570 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c7400f9a0 con 0x55b1bf8d4c10 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.172+0000 7f8c7a8ce640 1 -- 172.21.15.22:0/1400344570 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7400fdf0 con 0x55b1bf8d4c10 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 -- 172.21.15.22:0/1400344570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 msgr2=0x55b1bf8287b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 0x55b1bf8287b0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8c74001620 tx=0x7f8c74001ae0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 -- 172.21.15.22:0/1400344570 shutdown_connections 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d4c10 0x55b1bf8287b0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf8d21d0 0x55b1bf829b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/1400344570 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf7fbd90 0x55b1bf8c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 -- 172.21.15.22:0/1400344570 >> 172.21.15.22:0/1400344570 conn(0x55b1bf71dac0 msgr2=0x55b1bf8d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 -- 172.21.15.22:0/1400344570 shutdown_connections 2024-09-21T05:44:14.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.173+0000 7f8c81e16f40 1 -- 172.21.15.22:0/1400344570 wait complete. 2024-09-21T05:44:14.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.174+0000 7f8c81e16f40 1 Processor -- start 2024-09-21T05:44:14.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.174+0000 7f8c81e16f40 1 -- start start 2024-09-21T05:44:14.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c81e16f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf7fbd90 0x55b1bf8ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c81e16f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d21d0 0x55b1bf8b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c81e16f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf8d4c10 0x55b1bf8b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c81e16f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b1bf8b7370 con 0x55b1bf8d21d0 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c81e16f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b1bf8b7090 con 0x55b1bf7fbd90 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c81e16f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b1bf8c9c40 con 0x55b1bf8d4c10 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c73fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d21d0 0x55b1bf8b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c73fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d21d0 0x55b1bf8b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:36500/0 (socket says 172.21.15.22:36500) 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c73fff640 1 -- 172.21.15.22:0/999188342 learned_addr learned my addr 172.21.15.22:0/999188342 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c7b8d0640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf7fbd90 0x55b1bf8ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.175+0000 7f8c7c0d1640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf8d4c10 0x55b1bf8b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c73fff640 1 -- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf8d4c10 msgr2=0x55b1bf8b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c73fff640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b1bf8d4c10 0x55b1bf8b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c73fff640 1 -- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf7fbd90 msgr2=0x55b1bf8ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c73fff640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b1bf7fbd90 0x55b1bf8ca670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c73fff640 1 -- 172.21.15.22:0/999188342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1bf8cabb0 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c73fff640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d21d0 0x55b1bf8b3660 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8c64004b20 tx=0x7f8c64004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c798cc640 1 -- 172.21.15.22:0/999188342 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6404d780 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b1bf829210 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c798cc640 1 -- 172.21.15.22:0/999188342 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c64067070 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c798cc640 1 -- 172.21.15.22:0/999188342 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c640566d0 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.176+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1bf82c7e0 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.178+0000 7f8c798cc640 1 -- 172.21.15.22:0/999188342 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c64056870 con 0x55b1bf8d21d0 2024-09-21T05:44:14.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.178+0000 7f8c798cc640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8c5c0775a0 0x7f8c5c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.178+0000 7f8c798cc640 1 -- 172.21.15.22:0/999188342 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f8c640ddb10 con 0x55b1bf8d21d0 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.178+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b1bf8c9540 0x55b1bf82b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.178+0000 7f8c7b8d0640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8c5c0775a0 0x7f8c5c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.178+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e83) -- 0x55b1bf82b760 con 0x55b1bf8c9540 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.179+0000 7f8c7c0d1640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b1bf8c9540 0x55b1bf82b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.179+0000 7f8c7c0d1640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b1bf8c9540 0x55b1bf82b220 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.179+0000 7f8c7b8d0640 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8c5c0775a0 0x7f8c5c079ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c001120 tx=0x7f8c6c0017b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c7c0d1640 1 -- 172.21.15.22:0/999188342 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v83'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f8c7400f070 con 0x55b1bf8c9540 2024-09-21T05:44:14.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b1bf8c9540 msgr2=0x55b1bf82b220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b1bf8c9540 0x55b1bf82b220 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8c5c0775a0 msgr2=0x7f8c5c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8c5c0775a0 0x7f8c5c079ab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c001120 tx=0x7f8c6c0017b0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d21d0 msgr2=0x55b1bf8b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c81e16f40 1 --2- 172.21.15.22:0/999188342 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b1bf8d21d0 0x55b1bf8b3660 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8c64004b20 tx=0x7f8c64004e70 comp rx=0 tx=0).stop 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.182+0000 7f8c7c0d1640 1 -- 172.21.15.22:0/999188342 reap_dead start 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.183+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 shutdown_connections 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.183+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 >> 172.21.15.22:0/999188342 conn(0x55b1bf71dac0 msgr2=0x55b1bf8d4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.183+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 shutdown_connections 2024-09-21T05:44:14.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.183+0000 7f8c81e16f40 1 -- 172.21.15.22:0/999188342 wait complete. 2024-09-21T05:44:14.188 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-21T05:44:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.239+0000 7f8b45e0b280 1 -- start start 2024-09-21T05:44:14.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.240+0000 7f8b45e0b280 1 Processor -- start 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 0x55dca197e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c65b00 0x55dca197e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66400 0x55dca197eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d32600 con 0x55dca0c65680 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d32480 con 0x55dca0c65b00 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dca0d32300 con 0x55dca0c66400 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 0x55dca197e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 0x55dca197e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34838/0 (socket says 172.21.15.22:34838) 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 -- 172.21.15.22:0/677156785 learned_addr learned my addr 172.21.15.22:0/677156785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43162640 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c65b00 0x55dca197e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b44164640 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66400 0x55dca197eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 -- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66400 msgr2=0x55dca197eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66400 0x55dca197eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 -- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c65b00 msgr2=0x55dca197e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c65b00 0x55dca197e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 -- 172.21.15.22:0/677156785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca0bee780 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b43963640 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 0x55dca197e000 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x55dca19c7200 tx=0x55dca0c8c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37c2a86abaa87cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.241+0000 7f8b4115e640 1 -- 172.21.15.22:0/677156785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dc9cba6340 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b4115e640 1 -- 172.21.15.22:0/677156785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55dc9cbace00 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b4115e640 1 -- 172.21.15.22:0/677156785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dc9cba6ea0 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca0bef4a0 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55dca0bee000 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b45e0b280 4 waiting for MDS map... 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b4115e640 1 -- 172.21.15.22:0/677156785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x55dca1aa2600 con 0x55dca0c65680 2024-09-21T05:44:14.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b4115e640 1 -- 172.21.15.22:0/677156785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 63) ==== 2893+0+0 (secure 0 0 0) 0x55dca0c0b180 con 0x55dca0c65680 2024-09-21T05:44:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b45e0b280 4 Got MDS map 63 2024-09-21T05:44:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b45e0b280 10 main: JournalTool::main 2024-09-21T05:44:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.242+0000 7f8b45e0b280 4 main: JournalTool: connecting to RADOS... 2024-09-21T05:44:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.243+0000 7f8b45e0b280 1 Processor -- start 2024-09-21T05:44:14.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.243+0000 7f8b45e0b280 1 -- start start 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d32180 con 0x55dca0c64900 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dca0c13380 con 0x55dca0c64d80 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dca0c13800 con 0x55dca0c64480 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34840/0 (socket says 172.21.15.22:34840) 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 -- 172.21.15.22:0/4242396992 learned_addr learned my addr 172.21.15.22:0/4242396992 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b44164640 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43162640 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 -- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 msgr2=0x55dca197fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 -- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64d80 msgr2=0x55dca197f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.244+0000 7f8b43963640 1 -- 172.21.15.22:0/4242396992 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca0bee3c0 con 0x55dca0c64900 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b43963640 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 0x55dca197f600 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x55dca19aaf00 tx=0x55dca0c8df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cf8495d4d57cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b3f15a640 1 -- 172.21.15.22:0/4242396992 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dc9cba7380 con 0x55dca0c64900 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b3f15a640 1 -- 172.21.15.22:0/4242396992 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55dc9cbac700 con 0x55dca0c64900 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b3f15a640 1 -- 172.21.15.22:0/4242396992 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dc9cba71e0 con 0x55dca0c64900 2024-09-21T05:44:14.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4242396992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 msgr2=0x55dca197f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 0x55dca197f600 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x55dca19aaf00 tx=0x55dca0c8df20 comp rx=0 tx=0).stop 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4242396992 shutdown_connections 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4242396992 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4242396992 >> 172.21.15.22:0/4242396992 conn(0x55dca0c65200 msgr2=0x55dca0cd6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.245+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4242396992 shutdown_connections 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.246+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4242396992 wait complete. 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.246+0000 7f8b45e0b280 1 Processor -- start 2024-09-21T05:44:14.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.246+0000 7f8b45e0b280 1 -- start start 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.247+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.247+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d33380 con 0x55dca0c64d80 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d33200 con 0x55dca0c64900 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dca0d33500 con 0x55dca0c64480 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34846/0 (socket says 172.21.15.22:34846) 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 -- 172.21.15.22:0/1568012136 learned_addr learned my addr 172.21.15.22:0/1568012136 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b44164640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43963640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 -- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 msgr2=0x55dca197fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 -- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64900 msgr2=0x55dca197f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 -- 172.21.15.22:0/1568012136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca0bee960 con 0x55dca0c64d80 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b43162640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 0x55dca197f080 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55dca19abd40 tx=0x55dca1aee6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b3d957640 1 -- 172.21.15.22:0/1568012136 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dc9cba7380 con 0x55dca0c64d80 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b3d957640 1 -- 172.21.15.22:0/1568012136 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55dc9cbac540 con 0x55dca0c64d80 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dca0bef680 con 0x55dca0c64d80 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b3d957640 1 -- 172.21.15.22:0/1568012136 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dc9cba71e0 con 0x55dca0c64d80 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca0bef860 con 0x55dca0c64d80 2024-09-21T05:44:14.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.248+0000 7f8b45e0b280 4 main: JournalTool: resolving pool 12 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.250+0000 7f8b3d957640 1 -- 172.21.15.22:0/1568012136 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55dca1af0c00 con 0x55dca0c64d80 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.250+0000 7f8b3d957640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55dca0c67180 0x55dca1980100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.250+0000 7f8b3d957640 1 -- 172.21.15.22:0/1568012136 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x55dca1ae7000 con 0x55dca0c64d80 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.250+0000 7f8b45e0b280 4 main: JournalTool: creating IoCtx.. 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.250+0000 7f8b45e0b280 4 main: Executing for rank 0 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.250+0000 7f8b44164640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55dca0c67180 0x55dca1980100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.251+0000 7f8b44164640 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55dca0c67180 0x55dca1980100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55dca19c7c50 tx=0x55dca1aeef00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.251+0000 7f8b45e0b280 1 -- start start 2024-09-21T05:44:14.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.251+0000 7f8b45e0b280 1 Processor -- start 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c66880 0x55dca1980680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66d00 0x55dca1980c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 0x55dca1981180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d33800 con 0x55dca0c67600 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55dca0d33980 con 0x55dca0c66880 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55dca0d33b00 con 0x55dca0c66d00 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 0x55dca1981180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b44164640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c66880 0x55dca1980680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 0x55dca1981180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34854/0 (socket says 172.21.15.22:34854) 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b44164640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c66880 0x55dca1980680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54528/0 (socket says 172.21.15.22:54528) 2024-09-21T05:44:14.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 -- 172.21.15.22:0/4115316802 learned_addr learned my addr 172.21.15.22:0/4115316802 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43963640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66d00 0x55dca1980c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66d00 msgr2=0x55dca1980c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66d00 0x55dca1980c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c66880 msgr2=0x55dca1980680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c66880 0x55dca1980680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dca0befa40 con 0x55dca0c67600 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b43162640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 0x55dca1981180 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x55dca10140f0 tx=0x55dca1aef140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b785cdf30513aaae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b3a951640 1 -- 172.21.15.22:0/4115316802 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dca0f3b1e0 con 0x55dca0c67600 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b3a951640 1 -- 172.21.15.22:0/4115316802 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55dc9cbada40 con 0x55dca0c67600 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b3a951640 1 -- 172.21.15.22:0/4115316802 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55dca0f3b380 con 0x55dca0c67600 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.253+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dca0bee960 con 0x55dca0c67600 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55dca0bef680 con 0x55dca0c67600 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b45e0b280 4 waiting for MDS map... 2024-09-21T05:44:14.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b3a951640 1 -- 172.21.15.22:0/4115316802 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x55dca1016600 con 0x55dca0c67600 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b3a951640 1 -- 172.21.15.22:0/4115316802 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 63) ==== 2893+0+0 (secure 0 0 0) 0x55dca0c0b500 con 0x55dca0c67600 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b45e0b280 4 Got MDS map 63 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55dca0c67a80 0x55dca1981700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) -- 0x55dca1026400 con 0x55dca0c67a80 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b44164640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55dca0c67a80 0x55dca1981700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.254+0000 7f8b44164640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55dca0c67a80 0x55dca1981700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=3adfd4fb0eb0c13e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.255+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca0c67a80 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.255+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55dca102c000 0x55dca1034000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.255+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83) -- 0x55dca1026800 con 0x55dca102c000 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.255+0000 7f8b43963640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55dca102c000 0x55dca1034000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.256+0000 7f8b43963640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55dca102c000 0x55dca1034000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=537d05a81c09f6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.257+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55dca0cf4500 con 0x55dca102c000 2024-09-21T05:44:14.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b3914e640 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) -- 0x55dca1026c00 con 0x55dca102c000 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b3914e640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55dca102f600 0x55dca1035b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b3914e640 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83) -- 0x55dca1027000 con 0x55dca102f600 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b44164640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55dca102f600 0x55dca1035b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55dca0cf4500 con 0x55dca102c000 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b44164640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55dca102f600 0x55dca1035b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=fa84132e0001e866 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca102f600 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stdout:old journal was 4212139~2605 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stdout:new journal start will be 8388608 (4173864 bytes past old end) 2024-09-21T05:44:14.263 INFO:teuthology.orchestra.run.smithi022.stdout:writing journal head 2024-09-21T05:44:14.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.258+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1027800 con 0x55dca102c000 2024-09-21T05:44:14.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v83'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4500 con 0x55dca102c000 2024-09-21T05:44:14.264 INFO:teuthology.orchestra.run.smithi022.stdout:writing EResetJournal entry 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1027c00 con 0x55dca102f600 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55dca102c480 0x55dca1034580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1046000 con 0x55dca102c480 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1046400 con 0x55dca0c67a80 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1046800 con 0x55dca102f600 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1046c00 con 0x55dca102f600 2024-09-21T05:44:14.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55dca102c900 0x55dca1034b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1047000 con 0x55dca102c900 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b43963640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55dca102c480 0x55dca1034580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.260+0000 7f8b43162640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55dca102c900 0x55dca1034b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.261+0000 7f8b43162640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55dca102c900 0x55dca1034b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=1d0a3df7a2ac3121 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.262+0000 7f8b43963640 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55dca102c480 0x55dca1034580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=506396d332f8eb0c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.262+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(6 200.00000002 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca102f600 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.262+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(10 200.00000006 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca102f600 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.262+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(8 200.00000004 [delete] v83'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca0c67a80 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.262+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(9 200.00000005 [delete] v83'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca102f600 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.263+0000 7f8b43162640 1 -- 172.21.15.22:0/4115316802 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(11 200.00000007 [delete] v83'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4780 con 0x55dca102c900 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.264+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(7 200.00000003 [delete] v83'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4500 con 0x55dca102c480 2024-09-21T05:44:14.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.264+0000 7f8b3914e640 1 -- 172.21.15.22:0/4115316802 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) -- 0x55dca1027400 con 0x55dca102f600 2024-09-21T05:44:14.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b44164640 1 -- 172.21.15.22:0/4115316802 <== osd.4 v2:172.21.15.134:6826/3950009634 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v83'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55dca0cf4000 con 0x55dca102f600 2024-09-21T05:44:14.268 INFO:teuthology.orchestra.run.smithi022.stdout:done 2024-09-21T05:44:14.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55dca102c900 msgr2=0x55dca1034b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55dca102c900 0x55dca1034b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55dca102c480 msgr2=0x55dca1034580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55dca102c480 0x55dca1034580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55dca102f600 msgr2=0x55dca1035b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55dca102f600 0x55dca1035b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55dca102c000 msgr2=0x55dca1034000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55dca102c000 0x55dca1034000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55dca0c67a80 msgr2=0x55dca1981700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55dca0c67a80 0x55dca1981700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 reap_dead start 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 reap_dead start 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b43963640 1 -- 172.21.15.22:0/4115316802 reap_dead start 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 msgr2=0x55dca1981180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 0x55dca1981180 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x55dca10140f0 tx=0x55dca1aef140 comp rx=0 tx=0).stop 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 shutdown_connections 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/4115316802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c67600 0x55dca1981180 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 >> 172.21.15.22:0/4115316802 conn(0x55dca0c64000 msgr2=0x55dca0fb8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 shutdown_connections 2024-09-21T05:44:14.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.265+0000 7f8b45e0b280 1 -- 172.21.15.22:0/4115316802 wait complete. 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55dca0c67180 msgr2=0x55dca1980100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55dca0c67180 0x55dca1980100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55dca19c7c50 tx=0x55dca1aeef00 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 msgr2=0x55dca197f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 0x55dca197f080 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55dca19abd40 tx=0x55dca1aee6c0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 shutdown_connections 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c64d80 0x55dca197f080 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c64900 0x55dca197f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55dca0c67180 0x55dca1980100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/1568012136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c64480 0x55dca197fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 >> 172.21.15.22:0/1568012136 conn(0x55dca0c65200 msgr2=0x55dca0cd6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 shutdown_connections 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/1568012136 wait complete. 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 msgr2=0x55dca197e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 0x55dca197e000 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x55dca19c7200 tx=0x55dca0c8c3c0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 shutdown_connections 2024-09-21T05:44:14.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55dca0c66400 0x55dca197eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55dca0c65b00 0x55dca197e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 --2- 172.21.15.22:0/677156785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55dca0c65680 0x55dca197e000 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 >> 172.21.15.22:0/677156785 conn(0x55dca0c65f80 msgr2=0x55dca0cd7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 shutdown_connections 2024-09-21T05:44:14.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.266+0000 7f8b45e0b280 1 -- 172.21.15.22:0/677156785 wait complete. 2024-09-21T05:44:14.274 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.086008, result: old journal was 4212139~2605 new journal start will be 8388608 (4173864 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-21T05:44:14.274 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T05:44:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 -- 172.21.15.22:0/2959950212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0b39a0 msgr2=0x7f850c0b3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 --2- 172.21.15.22:0/2959950212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0b39a0 0x7f850c0b3db0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f850c08d800 tx=0x7f84fc02a540 comp rx=0 tx=0).stop 2024-09-21T05:44:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 -- 172.21.15.22:0/2959950212 shutdown_connections 2024-09-21T05:44:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 --2- 172.21.15.22:0/2959950212 >> v2:172.21.15.134:3300/0 conn(0x7f850c0b4850 0x7f850c0b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 --2- 172.21.15.22:0/2959950212 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f850c0d43c0 0x7f850c0d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 --2- 172.21.15.22:0/2959950212 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0b39a0 0x7f850c0b3db0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 -- 172.21.15.22:0/2959950212 >> 172.21.15.22:0/2959950212 conn(0x7f850c08bfc0 msgr2=0x7f850c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:14.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 -- 172.21.15.22:0/2959950212 shutdown_connections 2024-09-21T05:44:14.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.392+0000 7f851245c640 1 -- 172.21.15.22:0/2959950212 wait complete. 2024-09-21T05:44:14.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.394+0000 7f851245c640 1 Processor -- start 2024-09-21T05:44:14.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.394+0000 7f851245c640 1 -- start start 2024-09-21T05:44:14.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f851245c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c0b39a0 0x7f850c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f851245c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f850c0b4850 0x7f850c1f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f851245c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 0x7f850c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f851245c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0b5530 con 0x7f850c0d43c0 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f851245c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0b53b0 con 0x7f850c0b39a0 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f851245c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f850c0b56b0 con 0x7f850c0b4850 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 0x7f850c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f850bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c0b39a0 0x7f850c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f8503fff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f850c0b4850 0x7f850c1f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f850bfff640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c0b39a0 0x7f850c0ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.134:3300/0 says I am v2:172.21.15.22:54536/0 (socket says 172.21.15.22:54536) 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f850bfff640 1 -- 172.21.15.22:0/2904903158 learned_addr learned my addr 172.21.15.22:0/2904903158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 0x7f850c0b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34868/0 (socket says 172.21.15.22:34868) 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 -- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f850c0b4850 msgr2=0x7f850c1f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f850c0b4850 0x7f850c1f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 -- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c0b39a0 msgr2=0x7f850c0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c0b39a0 0x7f850c0ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.395+0000 7f85109f9640 1 -- 172.21.15.22:0/2904903158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c0cd0f0 con 0x7f850c0d43c0 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.396+0000 7f85109f9640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 0x7f850c0b2670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8504005fd0 tx=0x7f850400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.396+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504016070 con 0x7f850c0d43c0 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.396+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f850400bd80 con 0x7f850c0d43c0 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.396+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040106c0 con 0x7f850c0d43c0 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.396+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c1fa150 con 0x7f850c0d43c0 2024-09-21T05:44:14.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.396+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1d0fa0 con 0x7f850c0d43c0 2024-09-21T05:44:14.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.397+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c1d0a30 con 0x7f850c0d43c0 2024-09-21T05:44:14.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.398+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8504019070 con 0x7f850c0d43c0 2024-09-21T05:44:14.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.398+0000 7f85097fa640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84ec078b90 0x7f84ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:14.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.398+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f850409b370 con 0x7f850c0d43c0 2024-09-21T05:44:14.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.398+0000 7f850bfff640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84ec078b90 0x7f84ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:14.402 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.399+0000 7f850bfff640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84ec078b90 0x7f84ec07b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009660 tx=0x7f84fc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:14.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.403+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8504063100 con 0x7f850c0d43c0 2024-09-21T05:44:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:14.628+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f850c0b4c60 con 0x7f850c0d43c0 2024-09-21T05:44:15.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.103+0000 7f85097fa640 1 -- 172.21.15.22:0/2904903158 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v64) ==== 162+0+0 (secure 0 0 0) 0x7f8504067b30 con 0x7f850c0d43c0 2024-09-21T05:44:15.105 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84ec078b90 msgr2=0x7f84ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84ec078b90 0x7f84ec07b0a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f84fc009660 tx=0x7f84fc004730 comp rx=0 tx=0).stop 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 msgr2=0x7f850c0b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 0x7f850c0b2670 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8504005fd0 tx=0x7f850400dd90 comp rx=0 tx=0).stop 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 shutdown_connections 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f850c0d43c0 0x7f850c0b2670 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84ec078b90 0x7f84ec07b0a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f850c0b4850 0x7f850c1f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 --2- 172.21.15.22:0/2904903158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f850c0b39a0 0x7f850c0ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.108 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.105+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 >> 172.21.15.22:0/2904903158 conn(0x7f850c08bfc0 msgr2=0x7f850c0d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:15.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.106+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 shutdown_connections 2024-09-21T05:44:15.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.106+0000 7f851245c640 1 -- 172.21.15.22:0/2904903158 wait complete. 2024-09-21T05:44:15.123 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 -- 172.21.15.22:0/401012744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c0cef60 msgr2=0x7f2f7c0cf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 --2- 172.21.15.22:0/401012744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c0cef60 0x7f2f7c0cf370 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c06f700 tx=0x7f2f6c02a820 comp rx=0 tx=0).stop 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 -- 172.21.15.22:0/401012744 shutdown_connections 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 --2- 172.21.15.22:0/401012744 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c0cef60 0x7f2f7c0cf370 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 --2- 172.21.15.22:0/401012744 >> v2:172.21.15.134:3300/0 conn(0x7f2f7c0b0500 0x7f2f7c0d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 -- 172.21.15.22:0/401012744 >> 172.21.15.22:0/401012744 conn(0x7f2f7c0d1060 msgr2=0x7f2f7c0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 -- 172.21.15.22:0/401012744 shutdown_connections 2024-09-21T05:44:15.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.242+0000 7f2f8259d640 1 -- 172.21.15.22:0/401012744 wait complete. 2024-09-21T05:44:15.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.244+0000 7f2f8259d640 1 Processor -- start 2024-09-21T05:44:15.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.244+0000 7f2f8259d640 1 -- start start 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.245+0000 7f2f8259d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 0x7f2f7c09e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.245+0000 7f2f8259d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f7c0b0500 0x7f2f7c1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.245+0000 7f2f8259d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f7c0cef60 0x7f2f7c1f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.245+0000 7f2f8259d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c0cf8b0 con 0x7f2f7c070b80 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.245+0000 7f2f8259d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7c15b940 con 0x7f2f7c0cef60 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.245+0000 7f2f8259d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2f7c0cfa30 con 0x7f2f7c0b0500 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 0x7f2f7c09e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 0x7f2f7c09e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34892/0 (socket says 172.21.15.22:34892) 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 -- 172.21.15.22:0/3890306630 learned_addr learned my addr 172.21.15.22:0/3890306630 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:15.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 -- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f7c0b0500 msgr2=0x7f2f7c1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f7c0b0500 0x7f2f7c1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 -- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f7c0cef60 msgr2=0x7f2f7c1f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f81d9c640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f7c0cef60 0x7f2f7c1f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f7c0cef60 0x7f2f7c1f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 -- 172.21.15.22:0/3890306630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f7c09d400 con 0x7f2f7c070b80 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8159b640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 0x7f2f7c09e4a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e7c0 tx=0x7f2f7400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400ccc0 con 0x7f2f7c070b80 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f74010430 con 0x7f2f7c070b80 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f7c09d6e0 con 0x7f2f7c070b80 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f74010860 con 0x7f2f7c070b80 2024-09-21T05:44:15.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.246+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f7c09dca0 con 0x7f2f7c070b80 2024-09-21T05:44:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.248+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f7c0cfe90 con 0x7f2f7c070b80 2024-09-21T05:44:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.248+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2f74010a00 con 0x7f2f7c070b80 2024-09-21T05:44:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.249+0000 7f2f79ffb640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2f68078b90 0x7f2f6807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.249+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f2f7409b850 con 0x7f2f7c070b80 2024-09-21T05:44:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.250+0000 7f2f80d9a640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2f68078b90 0x7f2f6807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:15.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.250+0000 7f2f80d9a640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2f68078b90 0x7f2f6807b0a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c158500 tx=0x7f2f700079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:15.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.254+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f74063b90 con 0x7f2f7c070b80 2024-09-21T05:44:15.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.479+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f7c074c90 con 0x7f2f7c070b80 2024-09-21T05:44:15.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.479+0000 7f2f79ffb640 1 -- 172.21.15.22:0/3890306630 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+5219 (secure 0 0 0) 0x7f2f74063490 con 0x7f2f7c070b80 2024-09-21T05:44:15.481 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:15.482 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":65,"btime":"2024-09-21T05:44:15:106835+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:15.106811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":65,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T05:44:15.482 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 65 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2f68078b90 msgr2=0x7f2f6807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2f68078b90 0x7f2f6807b0a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c158500 tx=0x7f2f700079f0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 msgr2=0x7f2f7c09e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 0x7f2f7c09e4a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e7c0 tx=0x7f2f7400ec80 comp rx=0 tx=0).stop 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 shutdown_connections 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2f7c0cef60 0x7f2f7c1f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2f68078b90 0x7f2f6807b0a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2f7c0b0500 0x7f2f7c1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 --2- 172.21.15.22:0/3890306630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2f7c070b80 0x7f2f7c09e4a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:15.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 >> 172.21.15.22:0/3890306630 conn(0x7f2f7c0d1060 msgr2=0x7f2f7c0cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:15.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.481+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 shutdown_connections 2024-09-21T05:44:15.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:15.482+0000 7f2f8259d640 1 -- 172.21.15.22:0/3890306630 wait complete. 2024-09-21T05:44:15.898 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:44:15.106811+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.22:6834/3207626530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3207626530}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3207626530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:44:15.898 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6708:up:replay 2024-09-21T05:44:16.899 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:17.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aafeb6640 1 -- 172.21.15.22:0/1317874215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a94003a00 msgr2=0x7f4a94003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/1317874215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4a94003a00 0x7f4a94003e70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009b40 tx=0x7f4a94005830 comp rx=0 tx=0).stop 2024-09-21T05:44:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aae453640 1 -- 172.21.15.22:0/1317874215 reap_dead start 2024-09-21T05:44:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aafeb6640 1 -- 172.21.15.22:0/1317874215 shutdown_connections 2024-09-21T05:44:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aafeb6640 1 -- 172.21.15.22:0/1317874215 >> 172.21.15.22:0/1317874215 conn(0x7f4aa809d7f0 msgr2=0x7f4aa80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aafeb6640 1 -- 172.21.15.22:0/1317874215 shutdown_connections 2024-09-21T05:44:17.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.028+0000 7f4aafeb6640 1 -- 172.21.15.22:0/1317874215 wait complete. 2024-09-21T05:44:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.030+0000 7f4aafeb6640 1 Processor -- start 2024-09-21T05:44:17.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.030+0000 7f4aafeb6640 1 -- start start 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.031+0000 7f4aafeb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a94003a00 0x7f4aa815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.031+0000 7f4aafeb6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa814f720 0x7f4aa80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.031+0000 7f4aafeb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 0x7f4aa80c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.031+0000 7f4aafeb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa806f270 con 0x7f4aa80b2590 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.031+0000 7f4aafeb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa806f0f0 con 0x7f4a94003a00 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.031+0000 7f4aafeb6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa806f3f0 con 0x7f4aa814f720 2024-09-21T05:44:17.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 0x7f4aa80c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 0x7f4aa80c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34906/0 (socket says 172.21.15.22:34906) 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 -- 172.21.15.22:0/700919012 learned_addr learned my addr 172.21.15.22:0/700919012 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aadc52640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a94003a00 0x7f4aa815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aad451640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa814f720 0x7f4aa80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 -- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa814f720 msgr2=0x7f4aa80b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa814f720 0x7f4aa80b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 -- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a94003a00 msgr2=0x7f4aa815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a94003a00 0x7f4aa815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 -- 172.21.15.22:0/700919012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa80b29a0 con 0x7f4aa80b2590 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aad451640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa814f720 0x7f4aa80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aae453640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 0x7f4aa80c8020 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0001330 tx=0x7f4aa0006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000fa20 con 0x7f4aa80b2590 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4aa0019070 con 0x7f4aa80b2590 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa80ca7e0 con 0x7f4aa80b2590 2024-09-21T05:44:17.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000fea0 con 0x7f4aa80b2590 2024-09-21T05:44:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.032+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa80caac0 con 0x7f4aa80b2590 2024-09-21T05:44:17.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.033+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa80b63b0 con 0x7f4aa80b2590 2024-09-21T05:44:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.034+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4aa001e460 con 0x7f4aa80b2590 2024-09-21T05:44:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.035+0000 7f4a9e7fc640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4a8c078b90 0x7f4a8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.035+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f4aa009b490 con 0x7f4aa80b2590 2024-09-21T05:44:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.035+0000 7f4aadc52640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4a8c078b90 0x7f4a8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:17.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.036+0000 7f4aadc52640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4a8c078b90 0x7f4a8c07b0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009b40 tx=0x7f4aa402d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:17.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.039+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4aa80b29a0 con 0x7f4aa80b2590 2024-09-21T05:44:17.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.266+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4aa80ca550 con 0x7f4aa80b2590 2024-09-21T05:44:17.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.267+0000 7f4a9e7fc640 1 -- 172.21.15.22:0/700919012 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5219 (secure 0 0 0) 0x7f4aa80b68f0 con 0x7f4aa80b2590 2024-09-21T05:44:17.270 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:17.270 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":67,"btime":"2024-09-21T05:44:17:110657+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:16.113393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":65,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T05:44:17.271 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 67 2024-09-21T05:44:17.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4a8c078b90 msgr2=0x7f4a8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:17.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4a8c078b90 0x7f4a8c07b0a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009b40 tx=0x7f4aa402d040 comp rx=0 tx=0).stop 2024-09-21T05:44:17.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 msgr2=0x7f4aa80c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 0x7f4aa80c8020 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0001330 tx=0x7f4aa0006660 comp rx=0 tx=0).stop 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 shutdown_connections 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4aa80b2590 0x7f4aa80c8020 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4a8c078b90 0x7f4a8c07b0a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4aa814f720 0x7f4aa80b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 --2- 172.21.15.22:0/700919012 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4a94003a00 0x7f4aa815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 >> 172.21.15.22:0/700919012 conn(0x7f4aa809d7f0 msgr2=0x7f4aa80b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 shutdown_connections 2024-09-21T05:44:17.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:17.269+0000 7f4aafeb6640 1 -- 172.21.15.22:0/700919012 wait complete. 2024-09-21T05:44:17.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:44:16.113393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.22:6834/3207626530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3207626530}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3207626530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:44:17.686 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6708:up:rejoin 2024-09-21T05:44:18.687 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 -- 172.21.15.22:0/1568755933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cfae0 msgr2=0x7fabb80cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 --2- 172.21.15.22:0/1568755933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cfae0 0x7fabb80cc970 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7faba8005fd0 tx=0x7faba802a4b0 comp rx=0 tx=0).stop 2024-09-21T05:44:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 -- 172.21.15.22:0/1568755933 shutdown_connections 2024-09-21T05:44:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 --2- 172.21.15.22:0/1568755933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cceb0 0x7fabb806ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 --2- 172.21.15.22:0/1568755933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cfae0 0x7fabb80cc970 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 -- 172.21.15.22:0/1568755933 >> 172.21.15.22:0/1568755933 conn(0x7fabb806d7d0 msgr2=0x7fabb80b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 -- 172.21.15.22:0/1568755933 shutdown_connections 2024-09-21T05:44:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.815+0000 7fabbff13640 1 -- 172.21.15.22:0/1568755933 wait complete. 2024-09-21T05:44:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.817+0000 7fabbff13640 1 Processor -- start 2024-09-21T05:44:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.817+0000 7fabbff13640 1 -- start start 2024-09-21T05:44:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbff13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 0x7fabb8156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbff13640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cfae0 0x7fabb8074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbff13640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb815a9b0 0x7fabb80745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbff13640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fabb80cc760 con 0x7fabb80cceb0 2024-09-21T05:44:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbff13640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fabb80cc5e0 con 0x7fabb815a9b0 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbff13640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fabb8070f20 con 0x7fabb80cfae0 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbdcaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 0x7fabb8156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbdcaf640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 0x7fabb8156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34924/0 (socket says 172.21.15.22:34924) 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbdcaf640 1 -- 172.21.15.22:0/1144234170 learned_addr learned my addr 172.21.15.22:0/1144234170 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbe4b0640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb815a9b0 0x7fabb80745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.818+0000 7fabbd4ae640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cfae0 0x7fabb8074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbdcaf640 1 -- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cfae0 msgr2=0x7fabb8074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbdcaf640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cfae0 0x7fabb8074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbdcaf640 1 -- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb815a9b0 msgr2=0x7fabb80745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbdcaf640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb815a9b0 0x7fabb80745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbdcaf640 1 -- 172.21.15.22:0/1144234170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb81f6c50 con 0x7fabb80cceb0 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbd4ae640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cfae0 0x7fabb8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:18.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbdcaf640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 0x7fabb8156ea0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c3d0 tx=0x7fabb400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:18.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4017070 con 0x7fabb80cceb0 2024-09-21T05:44:18.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb40053c0 con 0x7fabb80cceb0 2024-09-21T05:44:18.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb81f6ec0 con 0x7fabb80cceb0 2024-09-21T05:44:18.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4003850 con 0x7fabb80cceb0 2024-09-21T05:44:18.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.819+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81f7480 con 0x7fabb80cceb0 2024-09-21T05:44:18.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.820+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb80add20 con 0x7fabb80cceb0 2024-09-21T05:44:18.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.821+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fabb401a070 con 0x7fabb80cceb0 2024-09-21T05:44:18.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.821+0000 7fabae7fc640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab9c078b90 0x7fab9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:18.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.822+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fabb409a570 con 0x7fabb80cceb0 2024-09-21T05:44:18.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.822+0000 7fabbd4ae640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab9c078b90 0x7fab9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:18.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.823+0000 7fabbd4ae640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab9c078b90 0x7fab9c07b0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faba8033000 tx=0x7faba8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:18.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:18.825+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fabb4021020 con 0x7fabb80cceb0 2024-09-21T05:44:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.052+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabb80ada40 con 0x7fabb80cceb0 2024-09-21T05:44:19.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.052+0000 7fabae7fc640 1 -- 172.21.15.22:0/1144234170 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5226 (secure 0 0 0) 0x7fabb80b4780 con 0x7fabb80cceb0 2024-09-21T05:44:19.056 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:19.056 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":68,"btime":"2024-09-21T05:44:18:112132+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:18.112117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":7,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":6}]} 2024-09-21T05:44:19.056 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 68 2024-09-21T05:44:19.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab9c078b90 msgr2=0x7fab9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab9c078b90 0x7fab9c07b0a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7faba8033000 tx=0x7faba8004730 comp rx=0 tx=0).stop 2024-09-21T05:44:19.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 msgr2=0x7fabb8156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 0x7fabb8156ea0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c3d0 tx=0x7fabb400c890 comp rx=0 tx=0).stop 2024-09-21T05:44:19.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 shutdown_connections 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fabb815a9b0 0x7fabb80745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab9c078b90 0x7fab9c07b0a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fabb80cfae0 0x7fabb8074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 --2- 172.21.15.22:0/1144234170 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fabb80cceb0 0x7fabb8156ea0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 >> 172.21.15.22:0/1144234170 conn(0x7fabb806d7d0 msgr2=0x7fabb80ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 shutdown_connections 2024-09-21T05:44:19.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.055+0000 7fabbff13640 1 -- 172.21.15.22:0/1144234170 wait complete. 2024-09-21T05:44:19.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:44:18.112117+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'c', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.22:6834/3207626530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3207626530}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3207626530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6708, 'qdb_cluster': [6708]} 2024-09-21T05:44:19.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:44:19.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 -- 172.21.15.22:0/3341084667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d806eee0 msgr2=0x7f06d806f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 --2- 172.21.15.22:0/3341084667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d806eee0 0x7f06d806f2f0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f06d806e1d0 tx=0x7f06cc02a530 comp rx=0 tx=0).stop 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 -- 172.21.15.22:0/3341084667 shutdown_connections 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 --2- 172.21.15.22:0/3341084667 >> v2:172.21.15.134:3300/0 conn(0x7f06d80d0c20 0x7f06d80d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 --2- 172.21.15.22:0/3341084667 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d806eee0 0x7f06d806f2f0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 -- 172.21.15.22:0/3341084667 >> 172.21.15.22:0/3341084667 conn(0x7f06d809d8c0 msgr2=0x7f06d808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 -- 172.21.15.22:0/3341084667 shutdown_connections 2024-09-21T05:44:19.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.592+0000 7f06df6a9640 1 -- 172.21.15.22:0/3341084667 wait complete. 2024-09-21T05:44:19.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.594+0000 7f06df6a9640 1 Processor -- start 2024-09-21T05:44:19.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.594+0000 7f06df6a9640 1 -- start start 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06df6a9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06d806eee0 0x7f06d815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06df6a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 0x7f06d81f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06df6a9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06d80d0c20 0x7f06d81f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06df6a9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06d80b3d80 con 0x7f06d80b96c0 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06df6a9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06d80b3c00 con 0x7f06d806eee0 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06df6a9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06d80d1570 con 0x7f06d80d0c20 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06dcc44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 0x7f06d81f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06dcc44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 0x7f06d81f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34936/0 (socket says 172.21.15.22:34936) 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.595+0000 7f06dcc44640 1 -- 172.21.15.22:0/2196026092 learned_addr learned my addr 172.21.15.22:0/2196026092 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dd445640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06d806eee0 0x7f06d815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dcc44640 1 -- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06d80d0c20 msgr2=0x7f06d81f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06ddc46640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06d80d0c20 0x7f06d81f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dcc44640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06d80d0c20 0x7f06d81f5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dcc44640 1 -- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06d806eee0 msgr2=0x7f06d815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dcc44640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06d806eee0 0x7f06d815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dcc44640 1 -- 172.21.15.22:0/2196026092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d80a20e0 con 0x7f06d80b96c0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dd445640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06d806eee0 0x7f06d815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06dcc44640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 0x7f06d81f9430 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009920 tx=0x7f06cc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06d5ffb640 1 -- 172.21.15.22:0/2196026092 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc031d60 con 0x7f06d80b96c0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06d5ffb640 1 -- 172.21.15.22:0/2196026092 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06cc033e10 con 0x7f06d80b96c0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06d5ffb640 1 -- 172.21.15.22:0/2196026092 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc005260 con 0x7f06d80b96c0 2024-09-21T05:44:19.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d80a23a0 con 0x7f06d80b96c0 2024-09-21T05:44:19.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.596+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d815f0a0 con 0x7f06d80b96c0 2024-09-21T05:44:19.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.598+0000 7f06d5ffb640 1 -- 172.21.15.22:0/2196026092 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06cc005400 con 0x7f06d80b96c0 2024-09-21T05:44:19.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.599+0000 7f06d5ffb640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f06c4078bc0 0x7f06c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.599+0000 7f06d5ffb640 1 -- 172.21.15.22:0/2196026092 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f06cc0bbfc0 con 0x7f06d80b96c0 2024-09-21T05:44:19.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.599+0000 7f06dd445640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f06c4078bc0 0x7f06c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.600+0000 7f06e6619740 1 Processor -- start 2024-09-21T05:44:19.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.600+0000 7f06dd445640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f06c4078bc0 0x7f06c407b0d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f06c8007fd0 tx=0x7f06c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.600+0000 7f06e6619740 1 -- start start 2024-09-21T05:44:19.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.601+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144dedfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.601+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144dee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.601+0000 7f06e6619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144e0b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.601+0000 7f06e6619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562144d36820 con 0x5621449fb590 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.601+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562144d25c50 con 0x5621449f33b0 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.601+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562144d29a60 con 0x562144a1c270 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144e0b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144e0b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34946/0 (socket says 172.21.15.22:34946) 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 -- 172.21.15.22:0/3929007291 learned_addr learned my addr 172.21.15.22:0/3929007291 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06dd445640 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144dedfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06dcc44640 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144dee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 -- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 msgr2=0x562144dedfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144dedfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 -- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 msgr2=0x562144dee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144dee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 -- 172.21.15.22:0/3929007291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562144a87850 con 0x5621449fb590 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06ddc46640 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144e0b3e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f06d00099c0 tx=0x7f06d0009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c20641674b1685 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06beffd640 1 -- 172.21.15.22:0/3929007291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0016070 con 0x5621449fb590 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06beffd640 1 -- 172.21.15.22:0/3929007291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06d0004310 con 0x5621449fb590 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.602+0000 7f06beffd640 1 -- 172.21.15.22:0/3929007291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d000ec50 con 0x5621449fb590 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 -- 172.21.15.22:0/3929007291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 msgr2=0x562144e0b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144e0b3e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f06d00099c0 tx=0x7f06d0009e80 comp rx=0 tx=0).stop 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 -- 172.21.15.22:0/3929007291 shutdown_connections 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144e0b3e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144dee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 --2- 172.21.15.22:0/3929007291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144dedfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 -- 172.21.15.22:0/3929007291 >> 172.21.15.22:0/3929007291 conn(0x562144a103c0 msgr2=0x562144dc4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 -- 172.21.15.22:0/3929007291 shutdown_connections 2024-09-21T05:44:19.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.603+0000 7f06e6619740 1 -- 172.21.15.22:0/3929007291 wait complete. 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.604+0000 7f06e6619740 1 -- start start 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.604+0000 7f06e6619740 1 Processor -- start 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e00820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449fb590 0x562144dff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e01340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562144a96620 con 0x562144a1c270 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562144ccb5f0 con 0x5621449f33b0 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562144deea20 con 0x5621449fb590 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06e6619740 10 client.0 _ll_register_callbacks cb 0x562144ccf0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:19.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e01340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e01340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34958/0 (socket says 172.21.15.22:34958) 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 -- 172.21.15.22:0/2010480516 learned_addr learned my addr 172.21.15.22:0/2010480516 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06dd445640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e00820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06dcc44640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449fb590 0x562144dff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 -- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449fb590 msgr2=0x562144dff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449fb590 0x562144dff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 -- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 msgr2=0x562144e00820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e00820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 -- 172.21.15.22:0/2010480516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562144afe6b0 con 0x562144a1c270 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06dcc44640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449fb590 0x562144dff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06ddc46640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e01340 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x562144dc5e30 tx=0x7f06d0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8db421f823afc98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.606+0000 7f06b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06b3fff640 1 -- 172.21.15.22:0/2010480516 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0016040 con 0x562144a1c270 2024-09-21T05:44:19.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06b3fff640 1 -- 172.21.15.22:0/2010480516 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f06d0011680 con 0x562144a1c270 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d0004310 con 0x562144a1c270 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06b3fff640 1 -- 172.21.15.22:0/2010480516 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f06d0011880 con 0x562144a1c270 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 10 client.6963 fetch_fsmap learned FSMap version 68 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x562144e00300 con 0x562144a1c270 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06b3fff640 1 -- 172.21.15.22:0/2010480516 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06d0004310 con 0x562144a1c270 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06b3fff640 1 -- 172.21.15.22:0/2010480516 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f06d002f080 con 0x562144a1c270 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 10 client.6963 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 10 client.6963 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 20 client.6963 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5621449dae40 0x562144e03370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 4 client.6963 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562144d38770 con 0x5621449dae40 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.607+0000 7f06dd445640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5621449dae40 0x562144e03370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06dd445640 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5621449dae40 0x562144e03370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ab77c237d97c94 server_cookie=cf4e207f673a087d in_seq=0 out_seq=0 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06b3fff640 10 client.6963 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06b3fff640 1 -- 172.21.15.22:0/2010480516 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562144d38770 con 0x5621449dae40 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06b3fff640 10 client.6963 handle_command_reply: tid=0 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 client.6963 shutdown 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 msgr2=0x562144e01340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e01340 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x562144dc5e30 tx=0x7f06d0009680 comp rx=0 tx=0).stop 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 shutdown_connections 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5621449dae40 0x562144e03370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e01340 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449fb590 0x562144dff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 --2- 172.21.15.22:0/2010480516 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e00820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06b3fff640 0 client.6963 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.608+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 >> 172.21.15.22:0/2010480516 conn(0x562144a103c0 msgr2=0x562144dbe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.609+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 shutdown_connections 2024-09-21T05:44:19.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.609+0000 7f06e6619740 1 -- 172.21.15.22:0/2010480516 wait complete. 2024-09-21T05:44:19.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.609+0000 7f06e6619740 20 client.6963 trim_cache size 0 max 16384 2024-09-21T05:44:19.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.627+0000 7f06e6619740 1 Processor -- start 2024-09-21T05:44:19.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.627+0000 7f06e6619740 1 -- start start 2024-09-21T05:44:19.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06e6619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144dff0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06e6619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8201200 con 0x5621449fb590 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8161530 con 0x5621449f33b0 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06d80b83c0 con 0x562144a1c270 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144dff0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144dff0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34962/0 (socket says 172.21.15.22:34962) 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 -- 172.21.15.22:0/1604939325 learned_addr learned my addr 172.21.15.22:0/1604939325 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dd445640 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06ddc46640 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 -- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 msgr2=0x562144e0c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 -- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 msgr2=0x562144e048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 -- 172.21.15.22:0/1604939325 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562144dfeec0 con 0x5621449fb590 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06dcc44640 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144dff0b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f06cc02ab20 tx=0x7f06cc083430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c04765421d832ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06be7fc640 1 -- 172.21.15.22:0/1604939325 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc048040 con 0x5621449fb590 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06be7fc640 1 -- 172.21.15.22:0/1604939325 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06cc06f170 con 0x5621449fb590 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.629+0000 7f06be7fc640 1 -- 172.21.15.22:0/1604939325 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0840c0 con 0x5621449fb590 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 -- 172.21.15.22:0/1604939325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 msgr2=0x562144dff0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144dff0b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f06cc02ab20 tx=0x7f06cc083430 comp rx=0 tx=0).stop 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 -- 172.21.15.22:0/1604939325 shutdown_connections 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5621449fb590 0x562144dff0b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 --2- 172.21.15.22:0/1604939325 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449f33b0 0x562144e048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 -- 172.21.15.22:0/1604939325 >> 172.21.15.22:0/1604939325 conn(0x562144a103c0 msgr2=0x562144e00800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 -- 172.21.15.22:0/1604939325 shutdown_connections 2024-09-21T05:44:19.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.630+0000 7f06e6619740 1 -- 172.21.15.22:0/1604939325 wait complete. 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.631+0000 7f06e6619740 1 -- start start 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.631+0000 7f06e6619740 1 Processor -- start 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449f33b0 0x562144e03d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449fb590 0x562144e82190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562144dcd400 con 0x562144a1c270 2024-09-21T05:44:19.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562144e01da0 con 0x5621449fb590 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562144dcd9c0 con 0x5621449f33b0 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 10 client.0 _ll_register_callbacks cb 0x7f06cc005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34966/0 (socket says 172.21.15.22:34966) 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 -- 172.21.15.22:0/1080803137 learned_addr learned my addr 172.21.15.22:0/1080803137 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06dcc44640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449fb590 0x562144e82190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06dd445640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449f33b0 0x562144e03d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 -- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449f33b0 msgr2=0x562144e03d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449f33b0 0x562144e03d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 -- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449fb590 msgr2=0x562144e82190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449fb590 0x562144e82190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 -- 172.21.15.22:0/1080803137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562144e08240 con 0x562144a1c270 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06ddc46640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e0c2f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f06d000ed80 tx=0x7f06d0024db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360be89837780052 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06b3fff640 1 -- 172.21.15.22:0/1080803137 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0016030 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06b3fff640 1 -- 172.21.15.22:0/1080803137 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f06d0020040 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.633+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562144d34220 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06b3fff640 1 -- 172.21.15.22:0/1080803137 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f06d0039050 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 10 client.6969 fetch_fsmap learned FSMap version 68 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x562144e8bd90 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06b3fff640 1 -- 172.21.15.22:0/1080803137 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562144d34220 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06b3fff640 1 -- 172.21.15.22:0/1080803137 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f06d003c020 con 0x562144a1c270 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 10 client.6969 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 10 client.6969 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 20 client.6969 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x562144e8b870 0x562144dc2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 4 client.6969 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "status"}) -- 0x562144d31650 con 0x562144e8b870 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06dd445640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x562144e8b870 0x562144dc2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06b3fff640 10 client.6969 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.634+0000 7f06dd445640 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x562144e8b870 0x562144dc2a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510a3be6299d5c35 server_cookie=c26e2851927cb3fc in_seq=0 out_seq=0 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06b3fff640 1 -- 172.21.15.22:0/1080803137 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562144d31650 con 0x562144e8b870 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06b3fff640 10 client.6969 handle_command_reply: tid=0 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 client.6969 shutdown 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 msgr2=0x562144e0c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e0c2f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f06d000ed80 tx=0x7f06d0024db0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 shutdown_connections 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x562144e8b870 0x562144dc2a30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562144a1c270 0x562144e0c2f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5621449fb590 0x562144e82190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 --2- 172.21.15.22:0/1080803137 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5621449f33b0 0x562144e03d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 >> 172.21.15.22:0/1080803137 conn(0x562144a103c0 msgr2=0x562144e05050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06b3fff640 0 client.6969 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 shutdown_connections 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 1 -- 172.21.15.22:0/1080803137 wait complete. 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.635+0000 7f06e6619740 20 client.6969 trim_cache size 0 max 16384 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 6708, 2024-09-21T05:44:19.643 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 4.5273963180000001, 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 68, 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 83, 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 83, 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 18.939182199000001 2024-09-21T05:44:19.644 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f06c4078bc0 msgr2=0x7f06c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f06c4078bc0 0x7f06c407b0d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f06c8007fd0 tx=0x7f06c8009040 comp rx=0 tx=0).stop 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 msgr2=0x7f06d81f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 0x7f06d81f9430 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009920 tx=0x7f06cc033c20 comp rx=0 tx=0).stop 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 shutdown_connections 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f06c4078bc0 0x7f06c407b0d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06d80d0c20 0x7f06d81f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06d80b96c0 0x7f06d81f9430 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 --2- 172.21.15.22:0/2196026092 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06d806eee0 0x7f06d815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 >> 172.21.15.22:0/2196026092 conn(0x7f06d809d8c0 msgr2=0x7f06d8161cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 shutdown_connections 2024-09-21T05:44:19.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.636+0000 7f06df6a9640 1 -- 172.21.15.22:0/2196026092 wait complete. 2024-09-21T05:44:19.650 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 -- 172.21.15.22:0/2493927038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06441600b0 msgr2=0x7f06440ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 --2- 172.21.15.22:0/2493927038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06441600b0 0x7f06440ccbc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0634009830 tx=0x7f063402a510 comp rx=0 tx=0).stop 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 -- 172.21.15.22:0/2493927038 shutdown_connections 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 --2- 172.21.15.22:0/2493927038 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06440b7cc0 0x7f06440cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 --2- 172.21.15.22:0/2493927038 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06441600b0 0x7f06440ccbc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 -- 172.21.15.22:0/2493927038 >> 172.21.15.22:0/2493927038 conn(0x7f064406d4f0 msgr2=0x7f0644071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 -- 172.21.15.22:0/2493927038 shutdown_connections 2024-09-21T05:44:19.766 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.762+0000 7f06498dd640 1 -- 172.21.15.22:0/2493927038 wait complete. 2024-09-21T05:44:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.764+0000 7f06498dd640 1 Processor -- start 2024-09-21T05:44:19.767 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.764+0000 7f06498dd640 1 -- start start 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 0x7f06440b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06440cf150 0x7f0644152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06441600b0 0x7f064406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f06440b4460 con 0x7f06440b7cc0 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f06440d1ba0 con 0x7f06440cf150 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f06440b45e0 con 0x7f06441600b0 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 0x7f06440b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 0x7f06440b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34990/0 (socket says 172.21.15.22:34990) 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 -- 172.21.15.22:0/3270277549 learned_addr learned my addr 172.21.15.22:0/3270277549 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06437fe640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06441600b0 0x7f064406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06427fc640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06440cf150 0x7f0644152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 -- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06441600b0 msgr2=0x7f064406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06441600b0 0x7f064406e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 -- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06440cf150 msgr2=0x7f0644152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06440cf150 0x7f0644152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 -- 172.21.15.22:0/3270277549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06441533f0 con 0x7f06440b7cc0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f0642ffd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 0x7f06440b0f80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0634009950 tx=0x7f0634033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634005c90 con 0x7f06440b7cc0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0634031e50 con 0x7f06440b7cc0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0644154360 con 0x7f06440b7cc0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634033290 con 0x7f06440b7cc0 2024-09-21T05:44:19.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.766+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0644152f50 con 0x7f06440b7cc0 2024-09-21T05:44:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.768+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0644154150 con 0x7f06440b7cc0 2024-09-21T05:44:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.768+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0634033430 con 0x7f06440b7cc0 2024-09-21T05:44:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.769+0000 7f062bfff640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0630078b90 0x7f063007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:19.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.769+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f06340bc020 con 0x7f06440b7cc0 2024-09-21T05:44:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.769+0000 7f06427fc640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0630078b90 0x7f063007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:19.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.770+0000 7f06427fc640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0630078b90 0x7f063007b0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f064415f8e0 tx=0x7f0638005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:19.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:19.773+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f063403f030 con 0x7f06440b7cc0 2024-09-21T05:44:20.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.001+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f06440cec10 con 0x7f06440b7cc0 2024-09-21T05:44:20.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.001+0000 7f062bfff640 1 -- 172.21.15.22:0/3270277549 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v68) ==== 53+0+73 (secure 0 0 0) 0x7f0634084520 con 0x7f06440b7cc0 2024-09-21T05:44:20.004 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0630078b90 msgr2=0x7f063007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0630078b90 0x7f063007b0a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f064415f8e0 tx=0x7f0638005f90 comp rx=0 tx=0).stop 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 msgr2=0x7f06440b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 0x7f06440b0f80 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0634009950 tx=0x7f0634033c20 comp rx=0 tx=0).stop 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 shutdown_connections 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0630078b90 0x7f063007b0a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f06441600b0 0x7f064406e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f06440cf150 0x7f0644152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 --2- 172.21.15.22:0/3270277549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f06440b7cc0 0x7f06440b0f80 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 >> 172.21.15.22:0/3270277549 conn(0x7f064406d4f0 msgr2=0x7f06440b2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 shutdown_connections 2024-09-21T05:44:20.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.003+0000 7f06498dd640 1 -- 172.21.15.22:0/3270277549 wait complete. 2024-09-21T05:44:20.017 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:44:20.017 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:44:20.017 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:44:20.017 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:44:20.017 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:20.018 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:44:20.018 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:44:20.018 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:44:20.018 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:44:20.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.018 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:44:20.034 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:44:20.034 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:44:20.034 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:44:20.035 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42484sec preferred_lft 42484sec 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:44:20.036 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:44:20.037 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.037 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:44:20.037 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:44:20.037 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:44:20.037 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:44:20.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.037 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:44:20.099 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:44:20.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.101 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:44:20.158 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:44:20.158 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:44:20.158 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:44:20.158 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:44:20.158 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:44:20.159 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:44:20.159 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:44:20.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:44:20.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:44:20.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:44:20.159 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:44:20.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:44:20.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:44:20.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.288 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:44:20.303 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:44:20.303 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:44:20.303 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:44:20.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:44:20.369 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:44:20.370 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:20.426 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:44:20.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:44:20.426 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:20.481 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:44:20.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:20.578 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:20.578 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:44:20.578 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:44:20.589 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:20.589 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:44:20.589 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:44:20.589 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:44:20.589 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:44:20.595 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:44:20.596 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:44:20.597 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:44:20.598 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:44:20.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.598 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:44:20.623 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:44:20.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:44:20.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:44:20.722 INFO:teuthology.orchestra.run.smithi022.stdout:client6981 2024-09-21T05:44:20.723 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:44:20.723 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:44:20.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.739 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-09-21T05:44:20.797 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:44:20.794+0000 7f64756ba640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-09-21T05:44:20.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:44:20.798 INFO:teuthology.orchestra.run.smithi022.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-09-21T05:44:20.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:20.799 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-09-21T05:44:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 -- 172.21.15.22:0/2506984171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 msgr2=0x7fc468004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 --2- 172.21.15.22:0/2506984171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc468004c10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc48008cae0 tx=0x7fc468005830 comp rx=0 tx=0).stop 2024-09-21T05:44:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 -- 172.21.15.22:0/2506984171 shutdown_connections 2024-09-21T05:44:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 --2- 172.21.15.22:0/2506984171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc468004c10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.960 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 -- 172.21.15.22:0/2506984171 >> 172.21.15.22:0/2506984171 conn(0x7fc4800d45c0 msgr2=0x7fc4800d49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 -- 172.21.15.22:0/2506984171 shutdown_connections 2024-09-21T05:44:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.957+0000 7fc486a2e640 1 -- 172.21.15.22:0/2506984171 wait complete. 2024-09-21T05:44:20.961 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.959+0000 7fc486a2e640 1 Processor -- start 2024-09-21T05:44:20.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.959+0000 7fc486a2e640 1 -- start start 2024-09-21T05:44:20.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc486a2e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc4800b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc486a2e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc48015ff60 0x7fc480153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc486a2e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc480153d90 0x7fc480160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc486a2e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4800cf280 con 0x7fc468004800 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc486a2e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4800d2df0 con 0x7fc480153d90 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc486a2e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc4800cf400 con 0x7fc48015ff60 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc4800b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc4800b5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35010/0 (socket says 172.21.15.22:35010) 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.960+0000 7fc47ffff640 1 -- 172.21.15.22:0/3720883383 learned_addr learned my addr 172.21.15.22:0/3720883383 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc48015ff60 0x7fc480153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc484fcb640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc480153d90 0x7fc480160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47ffff640 1 -- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc48015ff60 msgr2=0x7fc480153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47ffff640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc48015ff60 0x7fc480153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47ffff640 1 -- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc480153d90 msgr2=0x7fc480160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47ffff640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc480153d90 0x7fc480160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47ffff640 1 -- 172.21.15.22:0/3720883383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4800b6100 con 0x7fc468004800 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc484fcb640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc480153d90 0x7fc480160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47ffff640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc4800b5870 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc470004f50 tx=0x7fc4700016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47cff9640 1 -- 172.21.15.22:0/3720883383 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470015020 con 0x7fc468004800 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48015f480 con 0x7fc468004800 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47cff9640 1 -- 172.21.15.22:0/3720883383 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc470011960 con 0x7fc468004800 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc47cff9640 1 -- 172.21.15.22:0/3720883383 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470011d90 con 0x7fc468004800 2024-09-21T05:44:20.966 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.961+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc480153ab0 con 0x7fc468004800 2024-09-21T05:44:20.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.963+0000 7fc47cff9640 1 -- 172.21.15.22:0/3720883383 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc470018020 con 0x7fc468004800 2024-09-21T05:44:20.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.964+0000 7fc47cff9640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc46c078bc0 0x7fc46c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.964+0000 7fc47cff9640 1 -- 172.21.15.22:0/3720883383 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fc47009b890 con 0x7fc468004800 2024-09-21T05:44:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.964+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc46c078bc0 0x7fc46c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.964+0000 7fc48da70740 1 Processor -- start 2024-09-21T05:44:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.964+0000 7fc48da70740 1 -- start start 2024-09-21T05:44:20.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.965+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc46c078bc0 0x7fc46c07b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc474033000 tx=0x7fc4740019f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc48da70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc48da70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe10724890 con 0x55fe1040a340 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe10713cc0 con 0x55fe103e9660 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fe10717ad0 con 0x55fe103e1480 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107f8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35012/0 (socket says 172.21.15.22:35012) 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 -- 172.21.15.22:0/886391771 learned_addr learned my addr 172.21.15.22:0/886391771 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc484fcb640 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 -- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 msgr2=0x55fe107dbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107dbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 -- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 msgr2=0x55fe107dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 -- 172.21.15.22:0/886391771 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe104ee7b0 con 0x55fe1040a340 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc47ffff640 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107f8e90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc470062f40 tx=0x7fc470062ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b58b37e4bcc75 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc465ffb640 1 -- 172.21.15.22:0/886391771 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470089070 con 0x55fe1040a340 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.966+0000 7fc465ffb640 1 -- 172.21.15.22:0/886391771 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc470068050 con 0x55fe1040a340 2024-09-21T05:44:20.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc465ffb640 1 -- 172.21.15.22:0/886391771 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47006cf00 con 0x55fe1040a340 2024-09-21T05:44:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 -- 172.21.15.22:0/886391771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 msgr2=0x55fe107f8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107f8e90 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc470062f40 tx=0x7fc470062ce0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 -- 172.21.15.22:0/886391771 shutdown_connections 2024-09-21T05:44:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107dbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 --2- 172.21.15.22:0/886391771 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107f8e90 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 -- 172.21.15.22:0/886391771 >> 172.21.15.22:0/886391771 conn(0x55fe103fe490 msgr2=0x55fe107b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 -- 172.21.15.22:0/886391771 shutdown_connections 2024-09-21T05:44:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.967+0000 7fc48da70740 1 -- 172.21.15.22:0/886391771 wait complete. 2024-09-21T05:44:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.968+0000 7fc48da70740 1 -- start start 2024-09-21T05:44:20.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.969+0000 7fc48da70740 1 Processor -- start 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e1480 0x55fe107d8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 0x55fe107b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe1040a340 0x55fe107b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe104846f0 con 0x55fe103e9660 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fe1069a480 con 0x55fe103e1480 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fe10805a90 con 0x55fe1040a340 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc48da70740 10 client.0 _ll_register_callbacks cb 0x55fe1070d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 0x55fe107b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 0x55fe107b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35022/0 (socket says 172.21.15.22:35022) 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 -- 172.21.15.22:0/4068648124 learned_addr learned my addr 172.21.15.22:0/4068648124 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47ffff640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e1480 0x55fe107d8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc484fcb640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe1040a340 0x55fe107b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 -- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe1040a340 msgr2=0x55fe107b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe1040a340 0x55fe107b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 -- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e1480 msgr2=0x55fe107d8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e1480 0x55fe107d8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 -- 172.21.15.22:0/4068648124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe1054ef40 con 0x55fe103e9660 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47ffff640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e1480 0x55fe107d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 0x55fe107b12f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c3d0 tx=0x7fc468005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba8f7fcb5c0f3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.970+0000 7fc45effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc45effd640 1 -- 172.21.15.22:0/4068648124 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474031040 con 0x55fe103e9660 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc45effd640 1 -- 172.21.15.22:0/4068648124 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fc474003cd0 con 0x55fe103e9660 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc470068050 con 0x55fe103e9660 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc45effd640 1 -- 172.21.15.22:0/4068648124 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc474040450 con 0x55fe103e9660 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 10 client.6993 fetch_fsmap learned FSMap version 68 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55fe10453650 con 0x55fe103e9660 2024-09-21T05:44:20.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc45effd640 1 -- 172.21.15.22:0/4068648124 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc470068050 con 0x55fe103e9660 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc45effd640 1 -- 172.21.15.22:0/4068648124 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7fc47404e020 con 0x55fe103e9660 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 10 client.6993 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 10 client.6993 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 20 client.6993 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe103c9120 0x55fe107ed600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 4 client.6993 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe107267e0 con 0x55fe103c9120 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.971+0000 7fc47ffff640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe103c9120 0x55fe107ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.972+0000 7fc45effd640 10 client.6993 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.972+0000 7fc47ffff640 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe103c9120 0x55fe107ed600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=332f25a7ebea298 server_cookie=19b0f22cf93db5 in_seq=0 out_seq=0 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.972+0000 7fc45effd640 1 -- 172.21.15.22:0/4068648124 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe107267e0 con 0x55fe103c9120 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.972+0000 7fc45effd640 10 client.6993 handle_command_reply: tid=0 2024-09-21T05:44:20.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.972+0000 7fc48da70740 1 client.6993 shutdown 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 msgr2=0x55fe107b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 0x55fe107b12f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc47400c3d0 tx=0x7fc468005830 comp rx=0 tx=0).stop 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 shutdown_connections 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe103c9120 0x55fe107ed600 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe1040a340 0x55fe107b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe103e9660 0x55fe107b12f0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 --2- 172.21.15.22:0/4068648124 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e1480 0x55fe107d8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc45effd640 0 client.6993 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 >> 172.21.15.22:0/4068648124 conn(0x55fe103fe490 msgr2=0x55fe107b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 shutdown_connections 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 1 -- 172.21.15.22:0/4068648124 wait complete. 2024-09-21T05:44:20.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.973+0000 7fc48da70740 20 client.6993 trim_cache size 0 max 16384 2024-09-21T05:44:20.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.991+0000 7fc48da70740 1 Processor -- start 2024-09-21T05:44:20.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.991+0000 7fc48da70740 1 -- start start 2024-09-21T05:44:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.992+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.992+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.992+0000 7fc48da70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.992+0000 7fc48da70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc480202610 con 0x55fe1040a340 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.992+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc480201660 con 0x55fe103e9660 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.992+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc48006cde0 con 0x55fe103e1480 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35032/0 (socket says 172.21.15.22:35032) 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 -- 172.21.15.22:0/213901343 learned_addr learned my addr 172.21.15.22:0/213901343 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc47ffff640 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 -- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 msgr2=0x55fe107cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 -- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 msgr2=0x55fe107c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 -- 172.21.15.22:0/213901343 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe107fa650 con 0x55fe1040a340 2024-09-21T05:44:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc47ffff640 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc484fcb640 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107c9e10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc478005fd0 tx=0x7fc478013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c239629b821f931 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc4657fa640 1 -- 172.21.15.22:0/213901343 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478004b20 con 0x55fe1040a340 2024-09-21T05:44:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc4657fa640 1 -- 172.21.15.22:0/213901343 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc478004cc0 con 0x55fe1040a340 2024-09-21T05:44:20.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.993+0000 7fc4657fa640 1 -- 172.21.15.22:0/213901343 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478005260 con 0x55fe1040a340 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 -- 172.21.15.22:0/213901343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 msgr2=0x55fe107c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107c9e10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc478005fd0 tx=0x7fc478013830 comp rx=0 tx=0).stop 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 -- 172.21.15.22:0/213901343 shutdown_connections 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107c9e10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 --2- 172.21.15.22:0/213901343 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe107cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 -- 172.21.15.22:0/213901343 >> 172.21.15.22:0/213901343 conn(0x55fe103fe490 msgr2=0x55fe107ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 -- 172.21.15.22:0/213901343 shutdown_connections 2024-09-21T05:44:20.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.994+0000 7fc48da70740 1 -- 172.21.15.22:0/213901343 wait complete. 2024-09-21T05:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.995+0000 7fc48da70740 1 -- start start 2024-09-21T05:44:21.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.995+0000 7fc48da70740 1 Processor -- start 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.996+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe10873a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.996+0000 7fc48da70740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.996+0000 7fc48da70740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107d46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.996+0000 7fc48da70740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fe107d5260 con 0x55fe1040a340 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.996+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4800cc290 con 0x55fe103e9660 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.996+0000 7fc48da70740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fe107dbe80 con 0x55fe103e1480 2024-09-21T05:44:21.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc48da70740 10 client.0 _ll_register_callbacks cb 0x7fc46800c0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107d46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107d46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35034/0 (socket says 172.21.15.22:35034) 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 -- 172.21.15.22:0/3077009065 learned_addr learned my addr 172.21.15.22:0/3077009065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc47ffff640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe10873a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 -- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 msgr2=0x55fe10873a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe10873a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 -- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 msgr2=0x55fe107d4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107d4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 -- 172.21.15.22:0/3077009065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe108763b0 con 0x55fe1040a340 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc484fcb640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107d46b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc478013d30 tx=0x7fc4780056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615f39a16005b7a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc45effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc45effd640 1 -- 172.21.15.22:0/3077009065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478004b20 con 0x55fe1040a340 2024-09-21T05:44:21.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc45effd640 1 -- 172.21.15.22:0/3077009065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fc478019050 con 0x55fe1040a340 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.997+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc478004cc0 con 0x55fe1040a340 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc45effd640 1 -- 172.21.15.22:0/3077009065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc47802f050 con 0x55fe1040a340 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 10 client.6999 fetch_fsmap learned FSMap version 68 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55fe10877900 con 0x55fe1040a340 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc45effd640 1 -- 172.21.15.22:0/3077009065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc478004cc0 con 0x55fe1040a340 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc45effd640 1 -- 172.21.15.22:0/3077009065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7fc478032020 con 0x55fe1040a340 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 10 client.6999 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 10 client.6999 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 20 client.6999 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe107b17f0 0x55fe10876860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 4 client.6999 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x55fe10722290 con 0x55fe107b17f0 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc47ffff640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe107b17f0 0x55fe10876860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.998+0000 7fc45effd640 10 client.6999 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc47ffff640 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe107b17f0 0x55fe10876860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850425f292a06395 server_cookie=3301649ef4e52a94 in_seq=0 out_seq=0 2024-09-21T05:44:21.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc45effd640 1 -- 172.21.15.22:0/3077009065 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe10722290 con 0x55fe107b17f0 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc45effd640 10 client.6999 handle_command_reply: tid=0 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 client.6999 shutdown 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 msgr2=0x55fe107d46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107d46b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc478013d30 tx=0x7fc4780056c0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 shutdown_connections 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55fe107b17f0 0x55fe10876860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc45effd640 0 client.6999 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fe1040a340 0x55fe107d46b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fe103e9660 0x55fe107d4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 --2- 172.21.15.22:0/3077009065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fe103e1480 0x55fe10873a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 >> 172.21.15.22:0/3077009065 conn(0x55fe103fe490 msgr2=0x55fe107dc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.007 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:44:20.999+0000 7f64756ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0) n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x56308c75c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-21T05:44:21.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 shutdown_connections 2024-09-21T05:44:21.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:20.999+0000 7fc48da70740 1 -- 172.21.15.22:0/3077009065 wait complete. 2024-09-21T05:44:21.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc48da70740 20 client.6999 trim_cache size 0 max 16384 2024-09-21T05:44:21.007 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:21.007 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:44:21.008 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "392e9f96-e85a-42b7-9ebe-58f3ae77d98b", 2024-09-21T05:44:21.008 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:44:21.008 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:21.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc46c078bc0 msgr2=0x7fc46c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc46c078bc0 0x7fc46c07b0d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc474033000 tx=0x7fc4740019f0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 msgr2=0x7fc4800b5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc4800b5870 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc470004f50 tx=0x7fc4700016e0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 shutdown_connections 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc480153d90 0x7fc480160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc46c078bc0 0x7fc46c07b0d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc48015ff60 0x7fc480153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 --2- 172.21.15.22:0/3720883383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc468004800 0x7fc4800b5870 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 >> 172.21.15.22:0/3720883383 conn(0x7fc4800d45c0 msgr2=0x7fc4800b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 shutdown_connections 2024-09-21T05:44:21.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.000+0000 7fc486a2e640 1 -- 172.21.15.22:0/3720883383 wait complete. 2024-09-21T05:44:21.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:21.014 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T05:44:21.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 -- 172.21.15.22:0/3430441660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 msgr2=0x7f2fd80b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 --2- 172.21.15.22:0/3430441660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd80b7490 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f2fd80d2fa0 tx=0x7f2fd002a560 comp rx=0 tx=0).stop 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 -- 172.21.15.22:0/3430441660 shutdown_connections 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 --2- 172.21.15.22:0/3430441660 >> v2:172.21.15.134:3300/0 conn(0x7f2fd80d2460 0x7f2fd815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 --2- 172.21.15.22:0/3430441660 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd80b7490 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 --2- 172.21.15.22:0/3430441660 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fd81570e0 0x7f2fd80d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 -- 172.21.15.22:0/3430441660 >> 172.21.15.22:0/3430441660 conn(0x7f2fd806d550 msgr2=0x7f2fd8071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 -- 172.21.15.22:0/3430441660 shutdown_connections 2024-09-21T05:44:21.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.130+0000 7f2fdead3640 1 -- 172.21.15.22:0/3430441660 wait complete. 2024-09-21T05:44:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.132+0000 7f2fdead3640 1 Processor -- start 2024-09-21T05:44:21.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.132+0000 7f2fdead3640 1 -- start start 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.133+0000 7f2fdead3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd80d2460 0x7f2fd80cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.133+0000 7f2fdead3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fd81570e0 0x7f2fd81f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.133+0000 7f2fdead3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.133+0000 7f2fdead3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd80b7b50 con 0x7f2fd815d670 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.133+0000 7f2fdead3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd80b79d0 con 0x7f2fd80d2460 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.133+0000 7f2fdead3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd80b7cd0 con 0x7f2fd81570e0 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35046/0 (socket says 172.21.15.22:35046) 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 -- 172.21.15.22:0/2715179518 learned_addr learned my addr 172.21.15.22:0/2715179518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd80d2460 0x7f2fd80cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fd81570e0 0x7f2fd81f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 -- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fd81570e0 msgr2=0x7f2fd81f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fd81570e0 0x7f2fd81f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 -- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd80d2460 msgr2=0x7f2fd80cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd80d2460 0x7f2fd80cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 -- 172.21.15.22:0/2715179518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd80c9390 con 0x7f2fd815d670 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdd070640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd815bc00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00099c0 tx=0x7f2fd0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fd57fa640 1 -- 172.21.15.22:0/2715179518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd003e030 con 0x7f2fd815d670 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fd57fa640 1 -- 172.21.15.22:0/2715179518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fd0003e40 con 0x7f2fd815d670 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd806e190 con 0x7f2fd815d670 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fd57fa640 1 -- 172.21.15.22:0/2715179518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0035040 con 0x7f2fd815d670 2024-09-21T05:44:21.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.134+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd80c9d40 con 0x7f2fd815d670 2024-09-21T05:44:21.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.137+0000 7f2fd57fa640 1 -- 172.21.15.22:0/2715179518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2fd0041030 con 0x7f2fd815d670 2024-09-21T05:44:21.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.137+0000 7f2fd57fa640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2fb8078b90 0x7f2fb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.137+0000 7f2fd57fa640 1 -- 172.21.15.22:0/2715179518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f2fd00bb570 con 0x7f2fd815d670 2024-09-21T05:44:21.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.138+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2fb8078b90 0x7f2fb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.138+0000 7f2fe5a45740 1 Processor -- start 2024-09-21T05:44:21.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.138+0000 7f2fe5a45740 1 -- start start 2024-09-21T05:44:21.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.138+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2fb8078b90 0x7f2fb807b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0004630 tx=0x7f2fc000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 0x55d75c436a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0214d0 0x55d75c40b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c40bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d75c364a60 con 0x55d75c04a390 2024-09-21T05:44:21.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d75c353e90 con 0x55d75c0296b0 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d75c357ca0 con 0x55d75c0214d0 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fdc86f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 0x55d75c436a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fdc86f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 0x55d75c436a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35052/0 (socket says 172.21.15.22:35052) 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2294731056 learned_addr learned my addr 172.21.15.22:0/2294731056 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0214d0 0x55d75c40b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.139+0000 7f2fdd070640 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c40bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0214d0 msgr2=0x55d75c40b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0214d0 0x55d75c40b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 msgr2=0x55d75c40bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c40bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2294731056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75c12c7d0 con 0x55d75c04a390 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 0x55d75c436a10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000bbf0 tx=0x7f2fc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239df02a9d8dd39e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fce7fc640 1 -- 172.21.15.22:0/2294731056 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0011d40 con 0x55d75c04a390 2024-09-21T05:44:21.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fce7fc640 1 -- 172.21.15.22:0/2294731056 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc001e420 con 0x55d75c04a390 2024-09-21T05:44:21.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.140+0000 7f2fce7fc640 1 -- 172.21.15.22:0/2294731056 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc001e830 con 0x55d75c04a390 2024-09-21T05:44:21.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2294731056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 msgr2=0x55d75c436a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 0x55d75c436a10 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000bbf0 tx=0x7f2fc0013000 comp rx=0 tx=0).stop 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2294731056 shutdown_connections 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c40bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0214d0 0x55d75c40b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2294731056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c04a390 0x55d75c436a10 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2294731056 >> 172.21.15.22:0/2294731056 conn(0x55d75c03e4e0 msgr2=0x55d75c3f1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2294731056 shutdown_connections 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.141+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2294731056 wait complete. 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.142+0000 7f2fe5a45740 1 -- start start 2024-09-21T05:44:21.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.142+0000 7f2fe5a45740 1 Processor -- start 2024-09-21T05:44:21.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0214d0 0x55d75c406340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 0x55d75c4493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c449900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d75c0c4740 con 0x55d75c0296b0 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d75c2f9970 con 0x55d75c0214d0 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d75c408e30 con 0x55d75c04a390 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fe5a45740 10 client.0 _ll_register_callbacks cb 0x55d75c308780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 0x55d75c4493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fd7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 0x55d75c4493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35058/0 (socket says 172.21.15.22:35058) 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.143+0000 7f2fd7fff640 1 -- 172.21.15.22:0/1402334463 learned_addr learned my addr 172.21.15.22:0/1402334463 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0214d0 0x55d75c406340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fdd070640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c449900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fd7fff640 1 -- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 msgr2=0x55d75c449900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.151 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c449900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fd7fff640 1 -- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0214d0 msgr2=0x55d75c406340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0214d0 0x55d75c406340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fd7fff640 1 -- 172.21.15.22:0/1402334463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75c0b5970 con 0x55d75c0296b0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 0x55d75c4493c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8011750 tx=0x7f2fc8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f719aafd8ff7ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1402334463 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8003c40 con 0x55d75c0296b0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1402334463 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f2fc8003de0 con 0x55d75c0296b0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc001e420 con 0x55d75c0296b0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1402334463 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2fc8023050 con 0x55d75c0296b0 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fe5a45740 10 client.7011 fetch_fsmap learned FSMap version 68 2024-09-21T05:44:21.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55d75c0936a0 con 0x55d75c0296b0 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.144+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1402334463 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc001e420 con 0x55d75c0296b0 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1402334463 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f2fc8026020 con 0x55d75c0296b0 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fe5a45740 10 client.7011 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fe5a45740 10 client.7011 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fe5a45740 20 client.7011 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c008f60 0x55d75c4303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fe5a45740 4 client.7011 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d75c3669b0 con 0x55d75c008f60 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c008f60 0x55d75c4303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fb77fe640 10 client.7011 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.145+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c008f60 0x55d75c4303b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cba21c9cd19ea93 server_cookie=c370065751a3b31b in_seq=0 out_seq=0 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1402334463 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d75c3669b0 con 0x55d75c008f60 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fb77fe640 10 client.7011 handle_command_reply: tid=0 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 client.7011 shutdown 2024-09-21T05:44:21.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 msgr2=0x55d75c4493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 0x55d75c4493c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8011750 tx=0x7f2fc8011c10 comp rx=0 tx=0).stop 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 shutdown_connections 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c008f60 0x55d75c4303b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c449900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0296b0 0x55d75c4493c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1402334463 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0214d0 0x55d75c406340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fb77fe640 0 client.7011 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 >> 172.21.15.22:0/1402334463 conn(0x55d75c03e4e0 msgr2=0x55d75c3eca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 shutdown_connections 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1402334463 wait complete. 2024-09-21T05:44:21.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.146+0000 7f2fe5a45740 20 client.7011 trim_cache size 0 max 16384 2024-09-21T05:44:21.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.164+0000 7f2fe5a45740 1 Processor -- start 2024-09-21T05:44:21.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.164+0000 7f2fe5a45740 1 -- start start 2024-09-21T05:44:21.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c3f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0296b0 0x55d75c3f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c04a390 0x55d75c433080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd80c7b30 con 0x55d75c0214d0 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd814ebc0 con 0x55d75c04a390 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2fd81fa240 con 0x55d75c0296b0 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c3f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c3f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35062/0 (socket says 172.21.15.22:35062) 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2591329797 learned_addr learned my addr 172.21.15.22:0/2591329797 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2591329797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0296b0 msgr2=0x55d75c3f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0296b0 0x55d75c3f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2591329797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c04a390 msgr2=0x55d75c433080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c04a390 0x55d75c433080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 -- 172.21.15.22:0/2591329797 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75c3f3520 con 0x55d75c0214d0 2024-09-21T05:44:21.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c3f1c90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000bb90 tx=0x7f2fc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64cf20b121d66f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fcdffb640 1 -- 172.21.15.22:0/2591329797 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0011d40 con 0x55d75c0214d0 2024-09-21T05:44:21.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fcdffb640 1 -- 172.21.15.22:0/2591329797 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc003f070 con 0x55d75c0214d0 2024-09-21T05:44:21.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.166+0000 7f2fcdffb640 1 -- 172.21.15.22:0/2591329797 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc001eb30 con 0x55d75c0214d0 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2591329797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 msgr2=0x55d75c3f1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c3f1c90 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000bb90 tx=0x7f2fc0004730 comp rx=0 tx=0).stop 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2591329797 shutdown_connections 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c04a390 0x55d75c433080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c0296b0 0x55d75c3f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/2591329797 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c3f1c90 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2591329797 >> 172.21.15.22:0/2591329797 conn(0x55d75c03e4e0 msgr2=0x55d75c405f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2591329797 shutdown_connections 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.167+0000 7f2fe5a45740 1 -- 172.21.15.22:0/2591329797 wait complete. 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.168+0000 7f2fe5a45740 1 -- start start 2024-09-21T05:44:21.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.168+0000 7f2fe5a45740 1 Processor -- start 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.169+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c4b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c3f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c3f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d75c3f1560 con 0x55d75c0214d0 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd814e550 con 0x55d75c0296b0 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d75c42e290 con 0x55d75c04a390 2024-09-21T05:44:21.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 10 client.0 _ll_register_callbacks cb 0x7f2fc8007a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c4b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c4b5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:35064/0 (socket says 172.21.15.22:35064) 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 -- 172.21.15.22:0/1427141192 learned_addr learned my addr 172.21.15.22:0/1427141192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c3f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdd070640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c3f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 -- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 msgr2=0x55d75c3f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c3f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 -- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 msgr2=0x55d75c3f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c3f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 -- 172.21.15.22:0/1427141192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75c408a50 con 0x55d75c0214d0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fdc86f640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c4b5940 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0004410 tx=0x7f2fc0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62902f6c45f3a690 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1427141192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0005440 con 0x55d75c0214d0 2024-09-21T05:44:21.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1427141192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f2fc00055e0 con 0x55d75c0214d0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.170+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc003f070 con 0x55d75c0214d0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1427141192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f2fc004d050 con 0x55d75c0214d0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 10 client.7017 fetch_fsmap learned FSMap version 68 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55d75c4084a0 con 0x55d75c0214d0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1427141192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fc003f070 con 0x55d75c0214d0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1427141192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 68) ==== 3502+0+0 (secure 0 0 0) 0x7f2fc0050020 con 0x55d75c0214d0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 10 client.7017 fetch_fsmap finished waiting for FSMap version 68 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 10 client.7017 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 20 client.7017 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c42d490 0x55d75c3eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 4 client.7017 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d75c362460 con 0x55d75c42d490 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c42d490 0x55d75c3eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:21.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fb77fe640 10 client.7017 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.171+0000 7f2fd7fff640 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c42d490 0x55d75c3eadd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639b548747293270 server_cookie=dabaad44c0a6e9a in_seq=0 out_seq=0 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fb77fe640 1 -- 172.21.15.22:0/1427141192 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55d75c362460 con 0x55d75c42d490 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fb77fe640 10 client.7017 handle_command_reply: tid=0 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 client.7017 shutdown 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 msgr2=0x55d75c4b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c4b5940 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0004410 tx=0x7f2fc0011bc0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 shutdown_connections 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d75c42d490 0x55d75c3eadd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d75c04a390 0x55d75c3f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d75c0296b0 0x55d75c3f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fb77fe640 0 client.7017 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 --2- 172.21.15.22:0/1427141192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d75c0214d0 0x55d75c4b5940 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 >> 172.21.15.22:0/1427141192 conn(0x55d75c03e4e0 msgr2=0x55d75c449b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 shutdown_connections 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 1 -- 172.21.15.22:0/1427141192 wait complete. 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fe5a45740 20 client.7017 trim_cache size 0 max 16384 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "392e9f96-e85a-42b7-9ebe-58f3ae77d98b": { 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "392e9f96-e85a-42b7-9ebe-58f3ae77d98b", 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive" 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:21.180 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "392e9f96-e85a-42b7-9ebe-58f3ae77d98b": { 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:44:20.999933+0000", 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:21.181 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.172+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2fb8078b90 msgr2=0x7f2fb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2fb8078b90 0x7f2fb807b0a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0004630 tx=0x7f2fc000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 msgr2=0x7f2fd815bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd815bc00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00099c0 tx=0x7f2fd0003910 comp rx=0 tx=0).stop 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 shutdown_connections 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2fd815d670 0x7f2fd815bc00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2fb8078b90 0x7f2fb807b0a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2fd81570e0 0x7f2fd81f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 --2- 172.21.15.22:0/2715179518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2fd80d2460 0x7f2fd80cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:21.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 >> 172.21.15.22:0/2715179518 conn(0x7f2fd806d550 msgr2=0x7f2fd80b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:21.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 shutdown_connections 2024-09-21T05:44:21.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:21.173+0000 7f2fdead3640 1 -- 172.21.15.22:0/2715179518 wait complete. 2024-09-21T05:44:21.187 INFO:tasks.cephfs.filesystem:scrub status for tag:392e9f96-e85a-42b7-9ebe-58f3ae77d98b - {'path': '/dir', 'tag': '392e9f96-e85a-42b7-9ebe-58f3ae77d98b', 'options': 'recursive'} 2024-09-21T05:44:21.998 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:44:22.001 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:44:22.307 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:44:22.304+0000 7f64796c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:44:29.416 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:44:29.414+0000 7fbf61087640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-21T05:44:51.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:51.188 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T05:44:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 -- 172.21.15.22:0/1004551659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00b9b60 msgr2=0x7efff00d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 --2- 172.21.15.22:0/1004551659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00b9b60 0x7efff00d4d80 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7efff00d00e0 tx=0x7effec02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 -- 172.21.15.22:0/1004551659 shutdown_connections 2024-09-21T05:44:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 --2- 172.21.15.22:0/1004551659 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00b9b60 0x7efff00d4d80 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 -- 172.21.15.22:0/1004551659 >> 172.21.15.22:0/1004551659 conn(0x7efff009d7f0 msgr2=0x7efff00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 -- 172.21.15.22:0/1004551659 shutdown_connections 2024-09-21T05:44:51.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.320+0000 7efff7382640 1 -- 172.21.15.22:0/1004551659 wait complete. 2024-09-21T05:44:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.322+0000 7efff7382640 1 Processor -- start 2024-09-21T05:44:51.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.322+0000 7efff7382640 1 -- start start 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff7382640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efff00b9b60 0x7efff014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff7382640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 0x7efff0160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff7382640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efff015ba40 0x7efff0161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff7382640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efff006f830 con 0x7efff00cf540 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff7382640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efff006f6b0 con 0x7efff00b9b60 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff7382640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efff006f9b0 con 0x7efff015ba40 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff491d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 0x7efff0160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff491d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 0x7efff0160bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53376/0 (socket says 172.21.15.22:53376) 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.323+0000 7efff491d640 1 -- 172.21.15.22:0/3111788644 learned_addr learned my addr 172.21.15.22:0/3111788644 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff511e640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efff00b9b60 0x7efff014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff591f640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efff015ba40 0x7efff0161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff491d640 1 -- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efff015ba40 msgr2=0x7efff0161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff491d640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efff015ba40 0x7efff0161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff491d640 1 -- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efff00b9b60 msgr2=0x7efff014f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff491d640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efff00b9b60 0x7efff014f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff491d640 1 -- 172.21.15.22:0/3111788644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff014fe90 con 0x7efff00cf540 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff511e640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efff00b9b60 0x7efff014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff591f640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efff015ba40 0x7efff0161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff491d640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 0x7efff0160bc0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7effe000c3a0 tx=0x7effe000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7effe5ffb640 1 -- 172.21.15.22:0/3111788644 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0011790 con 0x7efff00cf540 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7effe5ffb640 1 -- 172.21.15.22:0/3111788644 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effe0017070 con 0x7efff00cf540 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff00b5920 con 0x7efff00cf540 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7effe5ffb640 1 -- 172.21.15.22:0/3111788644 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe0011ce0 con 0x7efff00cf540 2024-09-21T05:44:51.328 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.324+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff00c82a0 con 0x7efff00cf540 2024-09-21T05:44:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.326+0000 7effe5ffb640 1 -- 172.21.15.22:0/3111788644 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7effe001c460 con 0x7efff00cf540 2024-09-21T05:44:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.327+0000 7effe5ffb640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7effd4078b90 0x7effd407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.327+0000 7effe5ffb640 1 -- 172.21.15.22:0/3111788644 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7effe009abf0 con 0x7efff00cf540 2024-09-21T05:44:51.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.327+0000 7efff511e640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7effd4078b90 0x7effd407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.327+0000 7efffe3ca740 1 Processor -- start 2024-09-21T05:44:51.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.327+0000 7efff511e640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7effd4078b90 0x7effd407b0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7effec034000 tx=0x7effec004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.328+0000 7efffe3ca740 1 -- start start 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f55740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f55c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f72b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efffe3ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556291e9ea60 con 0x556291b84390 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556291e8de90 con 0x556291b636b0 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556291e91ca0 con 0x556291b5b4d0 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f55740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f55740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53380/0 (socket says 172.21.15.22:53380) 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 -- 172.21.15.22:0/3858470397 learned_addr learned my addr 172.21.15.22:0/3858470397 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff491d640 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f55c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff591f640 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f72b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 -- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 msgr2=0x556291f55c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f55c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 -- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 msgr2=0x556291f72b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f72b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 -- 172.21.15.22:0/3858470397 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556291c667d0 con 0x556291b84390 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.329+0000 7efff511e640 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f55740 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7effec009910 tx=0x7effec03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f10d695f8c4708 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.330+0000 7effdaffd640 1 -- 172.21.15.22:0/3858470397 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec04d070 con 0x556291b84390 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.330+0000 7effdaffd640 1 -- 172.21.15.22:0/3858470397 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effec049540 con 0x556291b84390 2024-09-21T05:44:51.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.330+0000 7effdaffd640 1 -- 172.21.15.22:0/3858470397 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec049930 con 0x556291b84390 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.330+0000 7efffe3ca740 1 -- 172.21.15.22:0/3858470397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 msgr2=0x556291f55740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.330+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f55740 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7effec009910 tx=0x7effec03d000 comp rx=0 tx=0).stop 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 -- 172.21.15.22:0/3858470397 shutdown_connections 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f72b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f55c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3858470397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f55740 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 -- 172.21.15.22:0/3858470397 >> 172.21.15.22:0/3858470397 conn(0x556291b784e0 msgr2=0x556291f2ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 -- 172.21.15.22:0/3858470397 shutdown_connections 2024-09-21T05:44:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 -- 172.21.15.22:0/3858470397 wait complete. 2024-09-21T05:44:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.331+0000 7efffe3ca740 1 -- start start 2024-09-21T05:44:51.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.332+0000 7efffe3ca740 1 Processor -- start 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 0x556291f410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f80060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b84390 0x556291f805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556291bfe740 con 0x556291b5b4d0 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556291e33970 con 0x556291b636b0 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556291f561c0 con 0x556291b84390 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efffe3ca740 10 client.0 _ll_register_callbacks cb 0x556291e36be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efff511e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 0x556291f410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efff511e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 0x556291f410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53388/0 (socket says 172.21.15.22:53388) 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efff511e640 1 -- 172.21.15.22:0/1036722383 learned_addr learned my addr 172.21.15.22:0/1036722383 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efff491d640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f80060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.333+0000 7efff591f640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b84390 0x556291f805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff511e640 1 -- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b84390 msgr2=0x556291f805a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff511e640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b84390 0x556291f805a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff511e640 1 -- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 msgr2=0x556291f80060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff511e640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f80060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff511e640 1 -- 172.21.15.22:0/1036722383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556291bef970 con 0x556291b5b4d0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff591f640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b84390 0x556291f805a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff491d640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efff511e640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 0x556291f410a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x556291f2d5d0 tx=0x7effec04baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97ef2e4f500203f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7effcffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7effcffff640 1 -- 172.21.15.22:0/1036722383 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec04d040 con 0x556291b5b4d0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7effcffff640 1 -- 172.21.15.22:0/1036722383 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7effec04bce0 con 0x556291b5b4d0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec049540 con 0x556291b5b4d0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7effcffff640 1 -- 172.21.15.22:0/1036722383 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7effec064050 con 0x556291b5b4d0 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efffe3ca740 10 client.7032 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:51.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x556291bcd6a0 con 0x556291b5b4d0 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7effcffff640 1 -- 172.21.15.22:0/1036722383 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effec049540 con 0x556291b5b4d0 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7effcffff640 1 -- 172.21.15.22:0/1036722383 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7effec067020 con 0x556291b5b4d0 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efffe3ca740 10 client.7032 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efffe3ca740 10 client.7032 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.334+0000 7efffe3ca740 20 client.7032 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.335+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291b42f60 0x556291f29580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.335+0000 7efffe3ca740 4 client.7032 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.335+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556291ea09b0 con 0x556291b42f60 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.335+0000 7efff491d640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291b42f60 0x556291f29580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.335+0000 7effcffff640 10 client.7032 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.335+0000 7efff491d640 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291b42f60 0x556291f29580 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5537d379b35bed47 server_cookie=b5e2488b5e8b2295 in_seq=0 out_seq=0 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7effcffff640 1 -- 172.21.15.22:0/1036722383 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556291ea09b0 con 0x556291b42f60 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7effcffff640 10 client.7032 handle_command_reply: tid=0 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 client.7032 shutdown 2024-09-21T05:44:51.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 msgr2=0x556291f410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 0x556291f410a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x556291f2d5d0 tx=0x7effec04baf0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 shutdown_connections 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291b42f60 0x556291f29580 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b84390 0x556291f805a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f80060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7effcffff640 0 client.7032 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1036722383 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b5b4d0 0x556291f410a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 >> 172.21.15.22:0/1036722383 conn(0x556291b784e0 msgr2=0x556291f72150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 shutdown_connections 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 1 -- 172.21.15.22:0/1036722383 wait complete. 2024-09-21T05:44:51.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.336+0000 7efffe3ca740 20 client.7032 trim_cache size 0 max 16384 2024-09-21T05:44:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.354+0000 7efffe3ca740 1 Processor -- start 2024-09-21T05:44:51.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.354+0000 7efffe3ca740 1 -- start start 2024-09-21T05:44:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.355+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f29770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.355+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f3e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f3ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efffe3ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efff02023c0 con 0x556291b84390 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efff0201400 con 0x556291b636b0 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efff00b4d80 con 0x556291b5b4d0 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f3ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f3ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53396/0 (socket says 172.21.15.22:53396) 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 -- 172.21.15.22:0/3698626119 learned_addr learned my addr 172.21.15.22:0/3698626119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 -- 172.21.15.22:0/3698626119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 msgr2=0x556291f29770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f29770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 -- 172.21.15.22:0/3698626119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 msgr2=0x556291f3e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f3e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 -- 172.21.15.22:0/3698626119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556291f80fb0 con 0x556291b84390 2024-09-21T05:44:51.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7efff591f640 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f3ebd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7effe8001e40 tx=0x7effe8012690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfd5a79668bb661 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7effda7fc640 1 -- 172.21.15.22:0/3698626119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8019030 con 0x556291b84390 2024-09-21T05:44:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7effda7fc640 1 -- 172.21.15.22:0/3698626119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effe8006b50 con 0x556291b84390 2024-09-21T05:44:51.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.356+0000 7effda7fc640 1 -- 172.21.15.22:0/3698626119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8007020 con 0x556291b84390 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 -- 172.21.15.22:0/3698626119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 msgr2=0x556291f3ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f3ebd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7effe8001e40 tx=0x7effe8012690 comp rx=0 tx=0).stop 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 -- 172.21.15.22:0/3698626119 shutdown_connections 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f3ebd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f3e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 --2- 172.21.15.22:0/3698626119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291f29770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 -- 172.21.15.22:0/3698626119 >> 172.21.15.22:0/3698626119 conn(0x556291b784e0 msgr2=0x556291f40e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 -- 172.21.15.22:0/3698626119 shutdown_connections 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.357+0000 7efffe3ca740 1 -- 172.21.15.22:0/3698626119 wait complete. 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.358+0000 7efffe3ca740 1 -- start start 2024-09-21T05:44:51.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.358+0000 7efffe3ca740 1 Processor -- start 2024-09-21T05:44:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.359+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291fd5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.359+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f2b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efffe3ca740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f2bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efffe3ca740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efff00b53d0 con 0x556291b84390 2024-09-21T05:44:51.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556291ff0430 con 0x556291b636b0 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efffe3ca740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x556291ff0680 con 0x556291b5b4d0 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efffe3ca740 10 client.0 _ll_register_callbacks cb 0x7effe80015c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f2bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f2bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53400/0 (socket says 172.21.15.22:53400) 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 -- 172.21.15.22:0/1392889008 learned_addr learned my addr 172.21.15.22:0/1392889008 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff491d640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f2b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff511e640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291fd5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 -- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 msgr2=0x556291fd5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291fd5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 -- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 msgr2=0x556291f2b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f2b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 -- 172.21.15.22:0/1392889008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556291ff0140 con 0x556291b84390 2024-09-21T05:44:51.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efff591f640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f2bbc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7effe8001e10 tx=0x7effe8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d7d1dd6ac851cb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7effcffff640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7effcffff640 1 -- 172.21.15.22:0/1392889008 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8001a80 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7effcffff640 1 -- 172.21.15.22:0/1392889008 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7effe8006d00 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.360+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe8006b50 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7effcffff640 1 -- 172.21.15.22:0/1392889008 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7effe8003970 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 10 client.7038 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x556291f94870 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7effcffff640 1 -- 172.21.15.22:0/1392889008 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effe8006b50 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7effcffff640 1 -- 172.21.15.22:0/1392889008 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7effe8030050 con 0x556291b84390 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 10 client.7038 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 10 client.7038 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 20 client.7038 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:51.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291fd8990 0x556291ff2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 4 client.7038 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556291e9c460 con 0x556291fd8990 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7efff511e640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291fd8990 0x556291ff2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.361+0000 7effcffff640 10 client.7038 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efff511e640 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291fd8990 0x556291ff2cd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589b08d0f254e0e6 server_cookie=4125c94baaef1bbf in_seq=0 out_seq=0 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7effcffff640 1 -- 172.21.15.22:0/1392889008 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+377 (crc 0 0 0) 0x556291e9c460 con 0x556291fd8990 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7effcffff640 10 client.7038 handle_command_reply: tid=0 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 client.7038 shutdown 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 msgr2=0x556291f2bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f2bbc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7effe8001e10 tx=0x7effe8007750 comp rx=0 tx=0).stop 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 shutdown_connections 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x556291fd8990 0x556291ff2cd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556291b84390 0x556291f2bbc0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556291b636b0 0x556291f2b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7effcffff640 0 client.7038 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 --2- 172.21.15.22:0/1392889008 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556291b5b4d0 0x556291fd5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 >> 172.21.15.22:0/1392889008 conn(0x556291b784e0 msgr2=0x556291f80240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 shutdown_connections 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 1 -- 172.21.15.22:0/1392889008 wait complete. 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.362+0000 7efffe3ca740 20 client.7038 trim_cache size 0 max 16384 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:44:51.369 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "392e9f96-e85a-42b7-9ebe-58f3ae77d98b": { 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:44:20.999933+0000", 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:51.370 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7effd4078b90 msgr2=0x7effd407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7effd4078b90 0x7effd407b0a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7effec034000 tx=0x7effec004730 comp rx=0 tx=0).stop 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 msgr2=0x7efff0160bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 0x7efff0160bc0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7effe000c3a0 tx=0x7effe000c860 comp rx=0 tx=0).stop 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 shutdown_connections 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7effd4078b90 0x7effd407b0a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efff015ba40 0x7efff0161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efff00cf540 0x7efff0160bc0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 --2- 172.21.15.22:0/3111788644 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efff00b9b60 0x7efff014f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 >> 172.21.15.22:0/3111788644 conn(0x7efff009d7f0 msgr2=0x7efff01ea530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 shutdown_connections 2024-09-21T05:44:51.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.363+0000 7efff7382640 1 -- 172.21.15.22:0/3111788644 wait complete. 2024-09-21T05:44:51.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:44:51.376 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 -- 172.21.15.22:0/86832913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0706c0 msgr2=0x7f6f7c072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 --2- 172.21.15.22:0/86832913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0706c0 0x7f6f7c072ff0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6f74009830 tx=0x7f6f7402a480 comp rx=0 tx=0).stop 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 -- 172.21.15.22:0/86832913 shutdown_connections 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 --2- 172.21.15.22:0/86832913 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0706c0 0x7f6f7c072ff0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 --2- 172.21.15.22:0/86832913 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f7c0cd3e0 0x7f6f7c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 -- 172.21.15.22:0/86832913 >> 172.21.15.22:0/86832913 conn(0x7f6f7c06d7d0 msgr2=0x7f6f7c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 -- 172.21.15.22:0/86832913 shutdown_connections 2024-09-21T05:44:51.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.490+0000 7f6f83a84640 1 -- 172.21.15.22:0/86832913 wait complete. 2024-09-21T05:44:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.492+0000 7f6f83a84640 1 Processor -- start 2024-09-21T05:44:51.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.492+0000 7f6f83a84640 1 -- start start 2024-09-21T05:44:51.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.493+0000 7f6f83a84640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f7c0706c0 0x7f6f7c149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.493+0000 7f6f83a84640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 0x7f6f7c1466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.493+0000 7f6f83a84640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f7c0cd3e0 0x7f6f7c148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.493+0000 7f6f83a84640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c073b70 con 0x7f6f7c0af780 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.493+0000 7f6f83a84640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c0739f0 con 0x7f6f7c0706c0 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.493+0000 7f6f83a84640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6f7c073cf0 con 0x7f6f7c0cd3e0 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 0x7f6f7c1466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 0x7f6f7c1466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53412/0 (socket says 172.21.15.22:53412) 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 -- 172.21.15.22:0/1392525192 learned_addr learned my addr 172.21.15.22:0/1392525192 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f82021640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f7c0cd3e0 0x7f6f7c148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f81820640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f7c0706c0 0x7f6f7c149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 -- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f7c0cd3e0 msgr2=0x7f6f7c148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f7c0cd3e0 0x7f6f7c148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 -- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f7c0706c0 msgr2=0x7f6f7c149ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f7c0706c0 0x7f6f7c149ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 -- 172.21.15.22:0/1392525192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7c09d310 con 0x7f6f7c0af780 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f8101f640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 0x7f6f7c1466e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c970 tx=0x7f6f6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c007bb0 con 0x7f6f7c0af780 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f6c007e40 con 0x7f6f7c0af780 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00f690 con 0x7f6f7c0af780 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c147010 con 0x7f6f7c0af780 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.494+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c1475d0 con 0x7f6f7c0af780 2024-09-21T05:44:51.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.495+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c072ff0 con 0x7f6f7c0af780 2024-09-21T05:44:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.496+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6f6c00f830 con 0x7f6f7c0af780 2024-09-21T05:44:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.497+0000 7f6f727fc640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6f60078b90 0x7f6f6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.497+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f6f6c09b640 con 0x7f6f7c0af780 2024-09-21T05:44:51.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.497+0000 7f6f81820640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6f60078b90 0x7f6f6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.498+0000 7f6f81820640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6f60078b90 0x7f6f6007b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6f780020f0 tx=0x7f6f78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.504 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.502+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f6c014030 con 0x7f6f7c0af780 2024-09-21T05:44:51.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.727+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f7c09d0a0 con 0x7f6f7c0af780 2024-09-21T05:44:51.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.728+0000 7f6f727fc640 1 -- 172.21.15.22:0/1392525192 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5227 (secure 0 0 0) 0x7f6f6c063a20 con 0x7f6f7c0af780 2024-09-21T05:44:51.730 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:51.730 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":69,"btime":"2024-09-21T05:44:29:414897+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:29.112432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":6}]} 2024-09-21T05:44:51.731 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 69 2024-09-21T05:44:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6f60078b90 msgr2=0x7f6f6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6f60078b90 0x7f6f6007b0a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6f780020f0 tx=0x7f6f78009040 comp rx=0 tx=0).stop 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 msgr2=0x7f6f7c1466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 0x7f6f7c1466e0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00c970 tx=0x7f6f6c00ce30 comp rx=0 tx=0).stop 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 shutdown_connections 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6f60078b90 0x7f6f6007b0a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6f7c0cd3e0 0x7f6f7c148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6f7c0af780 0x7f6f7c1466e0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 --2- 172.21.15.22:0/1392525192 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6f7c0706c0 0x7f6f7c149ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 >> 172.21.15.22:0/1392525192 conn(0x7f6f7c06d7d0 msgr2=0x7f6f7c0b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 shutdown_connections 2024-09-21T05:44:51.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.730+0000 7f6f83a84640 1 -- 172.21.15.22:0/1392525192 wait complete. 2024-09-21T05:44:51.743 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T05:44:51.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3055228316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80b66b0 msgr2=0x7fc1a806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3055228316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80b66b0 0x7fc1a806eab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a002a780 comp rx=0 tx=0).stop 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3055228316 shutdown_connections 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3055228316 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80b66b0 0x7fc1a806eab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3055228316 >> v2:172.21.15.134:3300/0 conn(0x7fc1a80b9b60 0x7fc1a80d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3055228316 >> 172.21.15.22:0/3055228316 conn(0x7fc1a809d7f0 msgr2=0x7fc1a80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3055228316 shutdown_connections 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.860+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3055228316 wait complete. 2024-09-21T05:44:51.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.862+0000 7fc1adc9b640 1 Processor -- start 2024-09-21T05:44:51.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.862+0000 7fc1adc9b640 1 -- start start 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1adc9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 0x7fc1a80ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1adc9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80cf540 0x7fc1a81f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1adc9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a81f6c90 0x7fc1a81f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1adc9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a806f630 con 0x7fc1a80b9b60 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1adc9b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a806f4b0 con 0x7fc1a81f6c90 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1adc9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a806f7b0 con 0x7fc1a80cf540 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 0x7fc1a80ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1a77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 0x7fc1a80ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53416/0 (socket says 172.21.15.22:53416) 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.863+0000 7fc1a77fe640 1 -- 172.21.15.22:0/3868330017 learned_addr learned my addr 172.21.15.22:0/3868330017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a81f6c90 0x7fc1a81f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80cf540 0x7fc1a81f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a77fe640 1 -- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80cf540 msgr2=0x7fc1a81f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80cf540 0x7fc1a81f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a77fe640 1 -- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a81f6c90 msgr2=0x7fc1a81f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a81f6c90 0x7fc1a81f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a77fe640 1 -- 172.21.15.22:0/3868330017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a80cbc80 con 0x7fc1a80b9b60 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 0x7fc1a80ccd20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9f0 tx=0x7fc19800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc18ffff640 1 -- 172.21.15.22:0/3868330017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19800cd80 con 0x7fc1a80b9b60 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc18ffff640 1 -- 172.21.15.22:0/3868330017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc198004020 con 0x7fc1a80b9b60 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a80cbf60 con 0x7fc1a80b9b60 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc18ffff640 1 -- 172.21.15.22:0/3868330017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198004450 con 0x7fc1a80b9b60 2024-09-21T05:44:51.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.864+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a80cc520 con 0x7fc1a80b9b60 2024-09-21T05:44:51.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.867+0000 7fc18ffff640 1 -- 172.21.15.22:0/3868330017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc198005030 con 0x7fc1a80b9b60 2024-09-21T05:44:51.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.867+0000 7fc18ffff640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc194078b90 0x7fc19407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.867+0000 7fc18ffff640 1 -- 172.21.15.22:0/3868330017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fc19809b5f0 con 0x7fc1a80b9b60 2024-09-21T05:44:51.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.867+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc194078b90 0x7fc19407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.868+0000 7fc1b4c0c740 1 Processor -- start 2024-09-21T05:44:51.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.868+0000 7fc1b4c0c740 1 -- start start 2024-09-21T05:44:51.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.868+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc194078b90 0x7fc19407b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80ce8d0 tx=0x7fc19c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a972f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a972f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a973161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563a972419a0 con 0x563a96f06670 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a97230dd0 con 0x563a96efe490 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563a97234be0 con 0x563a96f27350 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a973161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a973161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53418/0 (socket says 172.21.15.22:53418) 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1a7fff640 1 -- 172.21.15.22:0/1475875015 learned_addr learned my addr 172.21.15.22:0/1475875015 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a972f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.869+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a972f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc1a7fff640 1 -- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 msgr2=0x563a972f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a972f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc1a7fff640 1 -- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 msgr2=0x563a972f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a972f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc1a7fff640 1 -- 172.21.15.22:0/1475875015 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a971c9700 con 0x563a96f06670 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a973161c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0006010 tx=0x7fc1a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e248a18693ce257 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc18d7fa640 1 -- 172.21.15.22:0/1475875015 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0041070 con 0x563a96f06670 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc18d7fa640 1 -- 172.21.15.22:0/1475875015 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1a0003c80 con 0x563a96f06670 2024-09-21T05:44:51.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.870+0000 7fc18d7fa640 1 -- 172.21.15.22:0/1475875015 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0032630 con 0x563a96f06670 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/1475875015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 msgr2=0x563a973161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a973161c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0006010 tx=0x7fc1a0004a40 comp rx=0 tx=0).stop 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/1475875015 shutdown_connections 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a973161c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a972f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/1475875015 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a972f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/1475875015 >> 172.21.15.22:0/1475875015 conn(0x563a96f1b4a0 msgr2=0x563a972cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/1475875015 shutdown_connections 2024-09-21T05:44:51.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.871+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/1475875015 wait complete. 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.872+0000 7fc1b4c0c740 1 -- start start 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.872+0000 7fc1b4c0c740 1 Processor -- start 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a9730eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a97310340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563a96fa1700 con 0x563a96f06670 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a971d6340 con 0x563a96efe490 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563a972f9800 con 0x563a96f27350 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1b4c0c740 10 client.0 _ll_register_callbacks cb 0x563a971d4ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53420/0 (socket says 172.21.15.22:53420) 2024-09-21T05:44:51.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/3067093274 learned_addr learned my addr 172.21.15.22:0/3067093274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a9730eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a97310340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 msgr2=0x563a97310340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a97310340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 msgr2=0x563a9730eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a9730eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/3067093274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a9700b7c0 con 0x563a96f06670 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730acc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0146d0 tx=0x7fc19c001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7e655b9e6ae5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.874+0000 7fc1867fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:51.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1867fc640 1 -- 172.21.15.22:0/3067093274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c025020 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1867fc640 1 -- 172.21.15.22:0/3067093274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fc19c021610 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1a0003c80 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1867fc640 1 -- 172.21.15.22:0/3067093274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc19c02f050 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 10 client.7056 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x563a9706bf50 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1867fc640 1 -- 172.21.15.22:0/3067093274 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1a0003c80 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1867fc640 1 -- 172.21.15.22:0/3067093274 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7fc19c032020 con 0x563a96f06670 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 10 client.7056 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 10 client.7056 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 20 client.7056 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a96ee6130 0x563a973262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 4 client.7056 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a972438f0 con 0x563a96ee6130 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.875+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a96ee6130 0x563a973262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.876+0000 7fc1867fc640 10 client.7056 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.876+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a96ee6130 0x563a973262a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f39b3dbedeb6e4 server_cookie=3e807aadcfebc71 in_seq=0 out_seq=0 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.876+0000 7fc1867fc640 1 -- 172.21.15.22:0/3067093274 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a972438f0 con 0x563a96ee6130 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.876+0000 7fc1867fc640 10 client.7056 handle_command_reply: tid=0 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.876+0000 7fc1b4c0c740 1 client.7056 shutdown 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 msgr2=0x563a9730acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730acc0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0146d0 tx=0x7fc19c001080 comp rx=0 tx=0).stop 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 shutdown_connections 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a96ee6130 0x563a973262a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a97310340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730acc0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1867fc640 0 client.7056 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/3067093274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a9730eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 >> 172.21.15.22:0/3067093274 conn(0x563a96f1b4a0 msgr2=0x563a972c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 shutdown_connections 2024-09-21T05:44:51.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/3067093274 wait complete. 2024-09-21T05:44:51.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.877+0000 7fc1b4c0c740 20 client.7056 trim_cache size 0 max 16384 2024-09-21T05:44:51.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.891+0000 7fc1b4c0c740 1 Processor -- start 2024-09-21T05:44:51.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.891+0000 7fc1b4c0c740 1 -- start start 2024-09-21T05:44:51.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a97309cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a9730e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8148680 con 0x563a96f06670 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81f5f30 con 0x563a96efe490 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a8148010 con 0x563a96f27350 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53422/0 (socket says 172.21.15.22:53422) 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/174415486 learned_addr learned my addr 172.21.15.22:0/174415486 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a97309cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.892+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a9730e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 msgr2=0x563a9730e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a9730e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 msgr2=0x563a97309cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a97309cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/174415486 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a96f70660 con 0x563a96f06670 2024-09-21T05:44:51.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730a200 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc19c021820 tx=0x7fc19c021850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59239812fb45950 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc18cff9640 1 -- 172.21.15.22:0/174415486 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c025050 con 0x563a96f06670 2024-09-21T05:44:51.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc18cff9640 1 -- 172.21.15.22:0/174415486 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c01b070 con 0x563a96f06670 2024-09-21T05:44:51.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.893+0000 7fc18cff9640 1 -- 172.21.15.22:0/174415486 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c006c70 con 0x563a96f06670 2024-09-21T05:44:51.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/174415486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 msgr2=0x563a9730a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730a200 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fc19c021820 tx=0x7fc19c021850 comp rx=0 tx=0).stop 2024-09-21T05:44:51.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/174415486 shutdown_connections 2024-09-21T05:44:51.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96f27350 0x563a9730e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730a200 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/174415486 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96efe490 0x563a97309cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/174415486 >> 172.21.15.22:0/174415486 conn(0x563a96f1b4a0 msgr2=0x563a972c4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/174415486 shutdown_connections 2024-09-21T05:44:51.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.894+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/174415486 wait complete. 2024-09-21T05:44:51.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.895+0000 7fc1b4c0c740 1 -- start start 2024-09-21T05:44:51.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.895+0000 7fc1b4c0c740 1 Processor -- start 2024-09-21T05:44:51.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96efe490 0x563a973928a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96f27350 0x563a9730ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563a97323450 con 0x563a96f06670 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563a97310880 con 0x563a96f27350 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563a97323ad0 con 0x563a96efe490 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 10 client.0 _ll_register_callbacks cb 0x7fc19c001d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53424/0 (socket says 172.21.15.22:53424) 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/4115349193 learned_addr learned my addr 172.21.15.22:0/4115349193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96efe490 0x563a973928a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96f27350 0x563a9730ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96efe490 msgr2=0x563a973928a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96efe490 0x563a973928a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96f27350 msgr2=0x563a9730ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96f27350 0x563a9730ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 -- 172.21.15.22:0/4115349193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a972cc710 con 0x563a96f06670 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a7fff640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96f27350 0x563a9730ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1a6ffd640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730e840 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x563a972ca560 tx=0x7fc19c00fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7a4220552f87d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1867fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1867fc640 1 -- 172.21.15.22:0/4115349193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c025040 con 0x563a96f06670 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1867fc640 1 -- 172.21.15.22:0/4115349193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7fc19c006be0 con 0x563a96f06670 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.897+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c01b070 con 0x563a96f06670 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1867fc640 1 -- 172.21.15.22:0/4115349193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc19c04c050 con 0x563a96f06670 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 10 client.7062 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:51.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x563a97392de0 con 0x563a96f06670 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1867fc640 1 -- 172.21.15.22:0/4115349193 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc19c01b070 con 0x563a96f06670 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1867fc640 1 -- 172.21.15.22:0/4115349193 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7fc19c04f020 con 0x563a96f06670 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 10 client.7062 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 10 client.7062 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 20 client.7062 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a97397110 0x563a97328010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 4 client.7062 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x563a973116d0 con 0x563a97397110 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.898+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a97397110 0x563a97328010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1867fc640 10 client.7062 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1a77fe640 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a97397110 0x563a97328010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941b6655d54bfa53 server_cookie=530604b21425cd36 in_seq=0 out_seq=0 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1867fc640 1 -- 172.21.15.22:0/4115349193 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x563a973116d0 con 0x563a97397110 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1867fc640 10 client.7062 handle_command_reply: tid=0 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 client.7062 shutdown 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 msgr2=0x563a9730e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730e840 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x563a972ca560 tx=0x7fc19c00fe90 comp rx=0 tx=0).stop 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 shutdown_connections 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x563a97397110 0x563a97328010 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563a96f27350 0x563a9730ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563a96f06670 0x563a9730e840 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 --2- 172.21.15.22:0/4115349193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563a96efe490 0x563a973928a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1867fc640 0 client.7062 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 >> 172.21.15.22:0/4115349193 conn(0x563a96f1b4a0 msgr2=0x563a97309cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 shutdown_connections 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 1 -- 172.21.15.22:0/4115349193 wait complete. 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.899+0000 7fc1b4c0c740 20 client.7062 trim_cache size 0 max 16384 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout: "damage_type": "backtrace", 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 1213209192, 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627777, 2024-09-21T05:44:51.908 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/testdir/hardlink" 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "damage_type": "backtrace", 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 1722659165, 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627776, 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir" 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "damage_type": "dir_frag", 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 2817771187, 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627776, 2024-09-21T05:44:51.909 INFO:teuthology.orchestra.run.smithi022.stdout: "frag": "*", 2024-09-21T05:44:51.910 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir" 2024-09-21T05:44:51.910 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:51.910 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc194078b90 msgr2=0x7fc19407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc194078b90 0x7fc19407b0a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80ce8d0 tx=0x7fc19c009040 comp rx=0 tx=0).stop 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 msgr2=0x7fc1a80ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 0x7fc1a80ccd20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e9f0 tx=0x7fc19800eeb0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 shutdown_connections 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc1a81f6c90 0x7fc1a81f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc194078b90 0x7fc19407b0a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc1a80cf540 0x7fc1a81f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 --2- 172.21.15.22:0/3868330017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc1a80b9b60 0x7fc1a80ccd20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 >> 172.21.15.22:0/3868330017 conn(0x7fc1a809d7f0 msgr2=0x7fc1a80b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:51.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 shutdown_connections 2024-09-21T05:44:51.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:51.900+0000 7fc1adc9b640 1 -- 172.21.15.22:0/3868330017 wait complete. 2024-09-21T05:44:51.913 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.026+0000 7f5fa195e640 1 -- 172.21.15.22:0/3967525035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0b7fe0 msgr2=0x7f5f9c0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.026+0000 7f5fa195e640 1 --2- 172.21.15.22:0/3967525035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0b7fe0 0x7f5f9c0d0190 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f94005fd0 tx=0x7f5f9402a820 comp rx=0 tx=0).stop 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.027+0000 7f5fa195e640 1 -- 172.21.15.22:0/3967525035 shutdown_connections 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.027+0000 7f5fa195e640 1 --2- 172.21.15.22:0/3967525035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0b7fe0 0x7f5f9c0d0190 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.027+0000 7f5fa195e640 1 --2- 172.21.15.22:0/3967525035 >> v2:172.21.15.134:3300/0 conn(0x7f5f9c0cf440 0x7f5f9c15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.027+0000 7f5fa195e640 1 -- 172.21.15.22:0/3967525035 >> 172.21.15.22:0/3967525035 conn(0x7f5f9c0710b0 msgr2=0x7f5f9c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.027+0000 7f5fa195e640 1 -- 172.21.15.22:0/3967525035 shutdown_connections 2024-09-21T05:44:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.027+0000 7f5fa195e640 1 -- 172.21.15.22:0/3967525035 wait complete. 2024-09-21T05:44:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.028+0000 7f5fa195e640 1 Processor -- start 2024-09-21T05:44:52.030 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.029+0000 7f5fa195e640 1 -- start start 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5fa195e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f9c0b7fe0 0x7f5f9c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5fa195e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0cd360 0x7f5f9c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5fa195e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 0x7f5f9c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5fa195e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c0d0c00 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5fa195e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9c0d0a80 con 0x7f5f9c0b7fe0 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5fa195e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9c0d0d80 con 0x7f5f9c0cd360 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 0x7f5f9c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 0x7f5f9c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53430/0 (socket says 172.21.15.22:53430) 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 -- 172.21.15.22:0/212167580 learned_addr learned my addr 172.21.15.22:0/212167580 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9affd640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f9c0b7fe0 0x7f5f9c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9a7fc640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0cd360 0x7f5f9c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 -- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0cd360 msgr2=0x7f5f9c15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0cd360 0x7f5f9c15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 -- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f9c0b7fe0 msgr2=0x7f5f9c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f9c0b7fe0 0x7f5f9c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.030+0000 7f5f9b7fe640 1 -- 172.21.15.22:0/212167580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9c0ca620 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.031+0000 7f5f9b7fe640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 0x7f5f9c160000 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5f94035040 tx=0x7f5f94032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.031+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94037040 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.031+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f94042070 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.031+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f94003e60 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.031+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9c0ca900 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.031+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9c15fae0 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.032+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f9c0caad0 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.033+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f94004260 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.033+0000 7f5f93fff640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5f7c078b90 0x7f5f7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.033+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f5f940bbae0 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.033+0000 7f5f9affd640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5f7c078b90 0x7f5f7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.034+0000 7f5f9affd640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5f7c078b90 0x7f5f7c07b0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c001610 tx=0x7f5f8c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.038+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f94083ec0 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:44:52.156 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:44:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.265+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5f9c0d12a0 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.265+0000 7f5f93fff640 1 -- 172.21.15.22:0/212167580 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f5f940837c0 con 0x7f5f9c0cbb00 2024-09-21T05:44:52.268 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:44:52.268 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5f7c078b90 msgr2=0x7f5f7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5f7c078b90 0x7f5f7c07b0a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c001610 tx=0x7f5f8c009040 comp rx=0 tx=0).stop 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 msgr2=0x7f5f9c160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 0x7f5f9c160000 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f5f94035040 tx=0x7f5f94032c60 comp rx=0 tx=0).stop 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 shutdown_connections 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5f9c0cbb00 0x7f5f9c160000 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5f7c078b90 0x7f5f7c07b0a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5f9c0cd360 0x7f5f9c15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 --2- 172.21.15.22:0/212167580 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5f9c0b7fe0 0x7f5f9c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 >> 172.21.15.22:0/212167580 conn(0x7f5f9c0710b0 msgr2=0x7f5f9c0c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 shutdown_connections 2024-09-21T05:44:52.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.267+0000 7f5fa195e640 1 -- 172.21.15.22:0/212167580 wait complete. 2024-09-21T05:44:52.281 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T05:44:52.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:52.281 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-09-21T05:44:52.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.402+0000 7f470cf4a640 1 -- 172.21.15.22:0/2734756172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 msgr2=0x7f46f4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.402+0000 7f470cf4a640 1 --2- 172.21.15.22:0/2734756172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f46f4004c10 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009b20 tx=0x7f46f4005830 comp rx=0 tx=0).stop 2024-09-21T05:44:52.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.403+0000 7f470cf4a640 1 -- 172.21.15.22:0/2734756172 shutdown_connections 2024-09-21T05:44:52.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.403+0000 7f470cf4a640 1 --2- 172.21.15.22:0/2734756172 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f46f4004c10 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.403+0000 7f470cf4a640 1 -- 172.21.15.22:0/2734756172 >> 172.21.15.22:0/2734756172 conn(0x7f4708072a90 msgr2=0x7f4708072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.403+0000 7f470cf4a640 1 -- 172.21.15.22:0/2734756172 shutdown_connections 2024-09-21T05:44:52.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.403+0000 7f470cf4a640 1 -- 172.21.15.22:0/2734756172 wait complete. 2024-09-21T05:44:52.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.405+0000 7f470cf4a640 1 Processor -- start 2024-09-21T05:44:52.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.405+0000 7f470cf4a640 1 -- start start 2024-09-21T05:44:52.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470cf4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f47080cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470cf4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47080b1130 0x7f47080b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470cf4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470806f1f0 0x7f47080b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470cf4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f470815c7e0 con 0x7f46f4004800 2024-09-21T05:44:52.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470cf4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f470815c660 con 0x7f470806f1f0 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470cf4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f470815c960 con 0x7f47080b1130 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f47080cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f47080cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53456/0 (socket says 172.21.15.22:53456) 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 -- 172.21.15.22:0/289557833 learned_addr learned my addr 172.21.15.22:0/289557833 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f4705d9b640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47080b1130 0x7f47080b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f4706d9d640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470806f1f0 0x7f47080b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 -- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47080b1130 msgr2=0x7f47080b2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47080b1130 0x7f47080b2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 -- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470806f1f0 msgr2=0x7f47080b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470806f1f0 0x7f47080b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.406+0000 7f470659c640 1 -- 172.21.15.22:0/289557833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47081f9040 con 0x7f46f4004800 2024-09-21T05:44:52.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.407+0000 7f470659c640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f47080cbd00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f47000066b0 tx=0x7f4700001860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.407+0000 7f46feffd640 1 -- 172.21.15.22:0/289557833 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700011700 con 0x7f46f4004800 2024-09-21T05:44:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.407+0000 7f46feffd640 1 -- 172.21.15.22:0/289557833 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4700016020 con 0x7f46f4004800 2024-09-21T05:44:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.407+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47081f9320 con 0x7f46f4004800 2024-09-21T05:44:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.407+0000 7f46feffd640 1 -- 172.21.15.22:0/289557833 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700011b60 con 0x7f46f4004800 2024-09-21T05:44:52.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.407+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47081f98e0 con 0x7f46f4004800 2024-09-21T05:44:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.409+0000 7f46feffd640 1 -- 172.21.15.22:0/289557833 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4700011d00 con 0x7f46f4004800 2024-09-21T05:44:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.409+0000 7f46feffd640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f46e8078bc0 0x7f46e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.409+0000 7f46feffd640 1 -- 172.21.15.22:0/289557833 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f470009b300 con 0x7f46f4004800 2024-09-21T05:44:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.409+0000 7f4705d9b640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f46e8078bc0 0x7f46e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.410+0000 7f4713eb3740 1 Processor -- start 2024-09-21T05:44:52.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.410+0000 7f4705d9b640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f46e8078bc0 0x7f46e807b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f46f0035000 tx=0x7f46f00046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.410+0000 7f4713eb3740 1 -- start start 2024-09-21T05:44:52.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 0x559dd7b2ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7b2f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4713eb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559dd7a77120 con 0x559dd775c410 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559dd7a66550 con 0x559dd773b730 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559dd7a6a360 con 0x559dd7733550 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 0x559dd7b2ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 0x559dd7b2ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53458/0 (socket says 172.21.15.22:53458) 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 -- 172.21.15.22:0/153529204 learned_addr learned my addr 172.21.15.22:0/153529204 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4705d9b640 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7b2f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4706d9d640 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 -- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 msgr2=0x559dd7b2f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7b2f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 -- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 msgr2=0x559dd7b4c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 -- 172.21.15.22:0/153529204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd7758470 con 0x559dd775c410 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f4706d9d640 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f470659c640 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 0x559dd7b2ed10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4700099880 tx=0x7f4700062cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfa6261cabc0a83 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f46e7fff640 1 -- 172.21.15.22:0/153529204 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700015070 con 0x559dd775c410 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f46e7fff640 1 -- 172.21.15.22:0/153529204 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f470006ca90 con 0x559dd775c410 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.412+0000 7f46e7fff640 1 -- 172.21.15.22:0/153529204 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47000891d0 con 0x559dd775c410 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 -- 172.21.15.22:0/153529204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 msgr2=0x559dd7b2ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 0x559dd7b2ed10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4700099880 tx=0x7f4700062cc0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 -- 172.21.15.22:0/153529204 shutdown_connections 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7b2f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 --2- 172.21.15.22:0/153529204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd775c410 0x559dd7b2ed10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 -- 172.21.15.22:0/153529204 >> 172.21.15.22:0/153529204 conn(0x559dd7750560 msgr2=0x559dd7b04fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 -- 172.21.15.22:0/153529204 shutdown_connections 2024-09-21T05:44:52.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.413+0000 7f4713eb3740 1 -- 172.21.15.22:0/153529204 wait complete. 2024-09-21T05:44:52.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.414+0000 7f4713eb3740 1 -- start start 2024-09-21T05:44:52.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.414+0000 7f4713eb3740 1 Processor -- start 2024-09-21T05:44:52.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 0x559dd7b17d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b40900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd775c410 0x559dd7b1c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559dd77d67c0 con 0x559dd7733550 2024-09-21T05:44:52.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559dd7a0b890 con 0x559dd773b730 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559dd7b2f790 con 0x559dd775c410 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4713eb3740 10 client.0 _ll_register_callbacks cb 0x559dd7a5fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 0x559dd7b17d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 0x559dd7b17d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53472/0 (socket says 172.21.15.22:53472) 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 -- 172.21.15.22:0/4030269269 learned_addr learned my addr 172.21.15.22:0/4030269269 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4705d9b640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b40900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f4706d9d640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd775c410 0x559dd7b1c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 -- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd775c410 msgr2=0x559dd7b1c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd775c410 0x559dd7b1c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 -- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 msgr2=0x559dd7b40900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b40900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.416+0000 7f470659c640 1 -- 172.21.15.22:0/4030269269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd783c7e0 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f470659c640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 0x559dd7b17d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f470005f970 tx=0x7f470006a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66d3f1f82e41582 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f46e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f46e57fa640 1 -- 172.21.15.22:0/4030269269 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700015030 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f46e57fa640 1 -- 172.21.15.22:0/4030269269 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f4700062410 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470006ca90 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f46e57fa640 1 -- 172.21.15.22:0/4030269269 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f4700062630 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 10 client.7080 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x559dd789cf70 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f46e57fa640 1 -- 172.21.15.22:0/4030269269 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f470006ca90 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f46e57fa640 1 -- 172.21.15.22:0/4030269269 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f470004aaf0 con 0x559dd7733550 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 10 client.7080 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:52.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 10 client.7080 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 20 client.7080 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd771b1f0 0x559dd7b5c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 4 client.7080 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dd7a79070 con 0x559dd771b1f0 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.417+0000 7f4705d9b640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd771b1f0 0x559dd7b5c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.418+0000 7f4705d9b640 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd771b1f0 0x559dd7b5c8d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5476071ed0004af server_cookie=d91a8135d1cb6acb in_seq=0 out_seq=0 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.418+0000 7f46e57fa640 10 client.7080 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.418+0000 7f46e57fa640 1 -- 172.21.15.22:0/4030269269 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dd7a79070 con 0x559dd771b1f0 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.418+0000 7f46e57fa640 10 client.7080 handle_command_reply: tid=0 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.418+0000 7f4713eb3740 1 client.7080 shutdown 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 msgr2=0x559dd7b17d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 0x559dd7b17d40 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f470005f970 tx=0x7f470006a030 comp rx=0 tx=0).stop 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 shutdown_connections 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd771b1f0 0x559dd7b5c8d0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd775c410 0x559dd7b1c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f46e57fa640 0 client.7080 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b40900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 --2- 172.21.15.22:0/4030269269 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd7733550 0x559dd7b17d40 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 >> 172.21.15.22:0/4030269269 conn(0x559dd7750560 msgr2=0x559dd7b4b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 shutdown_connections 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 1 -- 172.21.15.22:0/4030269269 wait complete. 2024-09-21T05:44:52.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.419+0000 7f4713eb3740 20 client.7080 trim_cache size 0 max 16384 2024-09-21T05:44:52.439 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.437+0000 7f4713eb3740 1 Processor -- start 2024-09-21T05:44:52.440 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.437+0000 7f4713eb3740 1 -- start start 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd771b1f0 0x559dd7b2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7afe420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 0x559dd7afe960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4713eb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4708202640 con 0x559dd773b730 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4708201750 con 0x559dd771b1f0 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47080b5f50 con 0x559dd7733550 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 0x559dd7afe960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f470659c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd771b1f0 0x559dd7b2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 0x559dd7afe960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53482/0 (socket says 172.21.15.22:53482) 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 -- 172.21.15.22:0/2598787645 learned_addr learned my addr 172.21.15.22:0/2598787645 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4705d9b640 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7afe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 -- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 msgr2=0x559dd7afe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7afe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 -- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd771b1f0 msgr2=0x559dd7b2eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd771b1f0 0x559dd7b2eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 -- 172.21.15.22:0/2598787645 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd7b17730 con 0x559dd773b730 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f4706d9d640 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 0x559dd7afe960 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f47080cfea0 tx=0x7f46f8006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766454879043875a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.439+0000 7f46e77fe640 1 -- 172.21.15.22:0/2598787645 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8011d60 con 0x559dd773b730 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f46e77fe640 1 -- 172.21.15.22:0/2598787645 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46f801d480 con 0x559dd773b730 2024-09-21T05:44:52.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f46e77fe640 1 -- 172.21.15.22:0/2598787645 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f801d890 con 0x559dd773b730 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 -- 172.21.15.22:0/2598787645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 msgr2=0x559dd7afe960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 0x559dd7afe960 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f47080cfea0 tx=0x7f46f8006940 comp rx=0 tx=0).stop 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 -- 172.21.15.22:0/2598787645 shutdown_connections 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd773b730 0x559dd7afe960 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7afe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 --2- 172.21.15.22:0/2598787645 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd771b1f0 0x559dd7b2eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 -- 172.21.15.22:0/2598787645 >> 172.21.15.22:0/2598787645 conn(0x559dd7750560 msgr2=0x559dd7afac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 -- 172.21.15.22:0/2598787645 shutdown_connections 2024-09-21T05:44:52.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.440+0000 7f4713eb3740 1 -- 172.21.15.22:0/2598787645 wait complete. 2024-09-21T05:44:52.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.441+0000 7f4713eb3740 1 -- start start 2024-09-21T05:44:52.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.441+0000 7f4713eb3740 1 Processor -- start 2024-09-21T05:44:52.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 0x559dd7bc5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7bc9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47080b65a0 con 0x559dd771b1f0 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559dd7b4cee0 con 0x559dd773b730 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559dd7bccf80 con 0x559dd7733550 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4713eb3740 10 client.0 _ll_register_callbacks cb 0x7f46f40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 0x559dd7bc5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 0x559dd7bc5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53494/0 (socket says 172.21.15.22:53494) 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 -- 172.21.15.22:0/3805027609 learned_addr learned my addr 172.21.15.22:0/3805027609 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4705d9b640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7bc9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4706d9d640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 -- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 msgr2=0x559dd7bc9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7bc9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 -- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 msgr2=0x559dd7b4c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 -- 172.21.15.22:0/3805027609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd77a5720 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f4706d9d640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f470659c640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 0x559dd7bc5f70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f470006a0a0 tx=0x7f4700000f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bac46b4481398c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f46e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.443+0000 7f46e57fa640 1 -- 172.21.15.22:0/3805027609 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700015040 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f46e57fa640 1 -- 172.21.15.22:0/3805027609 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f470005eb00 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f801d480 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f46e57fa640 1 -- 172.21.15.22:0/3805027609 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f4700075050 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 10 client.7086 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x559dd77a5720 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f46e57fa640 1 -- 172.21.15.22:0/3805027609 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46f801d480 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f46e57fa640 1 -- 172.21.15.22:0/3805027609 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f4700078020 con 0x559dd771b1f0 2024-09-21T05:44:52.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 10 client.7086 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 10 client.7086 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 20 client.7086 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd76f0630 0x559dd7b2e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 4 client.7086 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x559dd7b1e160 con 0x559dd76f0630 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f4705d9b640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd76f0630 0x559dd7b2e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.444+0000 7f46e57fa640 10 client.7086 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4705d9b640 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd76f0630 0x559dd7b2e9a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7dda8ab9386629 server_cookie=94696f22da5b6ce5 in_seq=0 out_seq=0 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f46e57fa640 1 -- 172.21.15.22:0/3805027609 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559dd7b1e160 con 0x559dd76f0630 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f46e57fa640 10 client.7086 handle_command_reply: tid=0 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 client.7086 shutdown 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 msgr2=0x559dd7bc5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 0x559dd7bc5f70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f470006a0a0 tx=0x7f4700000f90 comp rx=0 tx=0).stop 2024-09-21T05:44:52.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 shutdown_connections 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x559dd76f0630 0x559dd7b2e9a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559dd773b730 0x559dd7b4c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559dd7733550 0x559dd7bc9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f46e57fa640 0 client.7086 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 --2- 172.21.15.22:0/3805027609 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559dd771b1f0 0x559dd7bc5f70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 >> 172.21.15.22:0/3805027609 conn(0x559dd7750560 msgr2=0x559dd7b18560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.445+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 shutdown_connections 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f4713eb3740 1 -- 172.21.15.22:0/3805027609 wait complete. 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f4713eb3740 20 client.7086 trim_cache size 0 max 16384 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:52.452 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:44:52.453 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "7d0ab2bf-0671-4a7a-9187-e82cb7a5967d", 2024-09-21T05:44:52.453 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:44:52.453 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:52.453 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f64756ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0)->f(v1 m2024-09-21T05:44:12.387045+0000) n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1)->n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56308c75c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f46e8078bc0 msgr2=0x7f46e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f46e8078bc0 0x7f46e807b0d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f46f0035000 tx=0x7f46f00046c0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 msgr2=0x7f47080cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f47080cbd00 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f47000066b0 tx=0x7f4700001860 comp rx=0 tx=0).stop 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 shutdown_connections 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f470806f1f0 0x7f47080b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f46e8078bc0 0x7f46e807b0d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47080b1130 0x7f47080b2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 --2- 172.21.15.22:0/289557833 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f46f4004800 0x7f47080cbd00 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 >> 172.21.15.22:0/289557833 conn(0x7f4708072a90 msgr2=0x7f47080d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 shutdown_connections 2024-09-21T05:44:52.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.446+0000 7f470cf4a640 1 -- 172.21.15.22:0/289557833 wait complete. 2024-09-21T05:44:52.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:44:52.460 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 -- 172.21.15.22:0/2493725552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180d16c0 msgr2=0x7f29180cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 --2- 172.21.15.22:0/2493725552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180d16c0 0x7f29180cf310 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f290c005fd0 tx=0x7f290c02a540 comp rx=0 tx=0).stop 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 -- 172.21.15.22:0/2493725552 shutdown_connections 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 --2- 172.21.15.22:0/2493725552 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29180b3a10 0x7f29180cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 --2- 172.21.15.22:0/2493725552 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180d16c0 0x7f29180cf310 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 -- 172.21.15.22:0/2493725552 >> 172.21.15.22:0/2493725552 conn(0x7f291809d8f0 msgr2=0x7f29180d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 -- 172.21.15.22:0/2493725552 shutdown_connections 2024-09-21T05:44:52.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.576+0000 7f291e572640 1 -- 172.21.15.22:0/2493725552 wait complete. 2024-09-21T05:44:52.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.578+0000 7f291e572640 1 Processor -- start 2024-09-21T05:44:52.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.578+0000 7f291e572640 1 -- start start 2024-09-21T05:44:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.579+0000 7f291e572640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29180b3a10 0x7f291815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.583 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.579+0000 7f291e572640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 0x7f2918156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.579+0000 7f291e572640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29180c8a70 0x7f29180c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.579+0000 7f291e572640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f29180d0120 con 0x7f29180b9b60 2024-09-21T05:44:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.579+0000 7f291e572640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f29180cffa0 con 0x7f29180c8a70 2024-09-21T05:44:52.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.579+0000 7f291e572640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f29180d02a0 con 0x7f29180b3a10 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 0x7f2918156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 0x7f2918156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53520/0 (socket says 172.21.15.22:53520) 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 -- 172.21.15.22:0/4119155227 learned_addr learned my addr 172.21.15.22:0/4119155227 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f291cb0f640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29180c8a70 0x7f29180c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f2917fff640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29180b3a10 0x7f291815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 -- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29180b3a10 msgr2=0x7f291815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29180b3a10 0x7f291815c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 -- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29180c8a70 msgr2=0x7f29180c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29180c8a70 0x7f29180c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 -- 172.21.15.22:0/4119155227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29180c93c0 con 0x7f29180b9b60 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f29177fe640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 0x7f2918156ab0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f290c034040 tx=0x7f290c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f2914ff9640 1 -- 172.21.15.22:0/4119155227 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c03e030 con 0x7f29180b9b60 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f2914ff9640 1 -- 172.21.15.22:0/4119155227 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f290c003d90 con 0x7f29180b9b60 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2918156ff0 con 0x7f29180b9b60 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f2914ff9640 1 -- 172.21.15.22:0/4119155227 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c035040 con 0x7f29180b9b60 2024-09-21T05:44:52.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.580+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f291815ce30 con 0x7f29180b9b60 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.582+0000 7f2914ff9640 1 -- 172.21.15.22:0/4119155227 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f290c041030 con 0x7f29180b9b60 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.583+0000 7f2914ff9640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2904078b90 0x7f290407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.583+0000 7f2914ff9640 1 -- 172.21.15.22:0/4119155227 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f290c0bb5d0 con 0x7f29180b9b60 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.583+0000 7f2917fff640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2904078b90 0x7f290407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.584+0000 7f2917fff640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2904078b90 0x7f290407b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2908001720 tx=0x7f2908009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.584+0000 7f29255c0740 1 Processor -- start 2024-09-21T05:44:52.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.584+0000 7f29255c0740 1 -- start start 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f29255c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021270a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021245860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021245da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f29255c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55802119ea60 con 0x558020e84390 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55802118de90 con 0x558020e636b0 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558021191ca0 con 0x558020e5b4d0 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.585+0000 7f2917fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021270a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021270a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53534/0 (socket says 172.21.15.22:53534) 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 -- 172.21.15.22:0/1932370584 learned_addr learned my addr 172.21.15.22:0/1932370584 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f29177fe640 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021245860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f291cb0f640 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021245da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 -- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 msgr2=0x558021245860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021245860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 -- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 msgr2=0x558021245da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021245da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 -- 172.21.15.22:0/1932370584 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558020f667d0 con 0x558020e84390 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f29177fe640 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021245860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f2917fff640 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021270a10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f290800f970 tx=0x7f2908012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24c111ad6c13b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f28fdffb640 1 -- 172.21.15.22:0/1932370584 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908025070 con 0x558020e84390 2024-09-21T05:44:52.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f28fdffb640 1 -- 172.21.15.22:0/1932370584 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f290801b050 con 0x558020e84390 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.586+0000 7f28fdffb640 1 -- 172.21.15.22:0/1932370584 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908014e40 con 0x558020e84390 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 -- 172.21.15.22:0/1932370584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 msgr2=0x558021270a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021270a10 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f290800f970 tx=0x7f2908012000 comp rx=0 tx=0).stop 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 -- 172.21.15.22:0/1932370584 shutdown_connections 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021245da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021245860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 --2- 172.21.15.22:0/1932370584 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021270a10 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 -- 172.21.15.22:0/1932370584 >> 172.21.15.22:0/1932370584 conn(0x558020e784e0 msgr2=0x55802122bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 -- 172.21.15.22:0/1932370584 shutdown_connections 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.587+0000 7f29255c0740 1 -- 172.21.15.22:0/1932370584 wait complete. 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.588+0000 7f29255c0740 1 -- start start 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.588+0000 7f29255c0740 1 Processor -- start 2024-09-21T05:44:52.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x55802126d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021242db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x558020efe740 con 0x558020e5b4d0 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x558021133970 con 0x558020e636b0 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x558021243fe0 con 0x558020e84390 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29255c0740 10 client.0 _ll_register_callbacks cb 0x558021142780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x55802126d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x55802126d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53546/0 (socket says 172.21.15.22:53546) 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 -- 172.21.15.22:0/836326105 learned_addr learned my addr 172.21.15.22:0/836326105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f291cb0f640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29177fe640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021242db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 -- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 msgr2=0x5580212432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 -- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 msgr2=0x558021242db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021242db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 -- 172.21.15.22:0/836326105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558020eef970 con 0x558020e5b4d0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f29177fe640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021242db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f2917fff640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x55802126d600 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2908014820 tx=0x7f29080206a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2a579cb9131a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f28f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.590+0000 7f28f6ffd640 1 -- 172.21.15.22:0/836326105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908025040 con 0x558020e5b4d0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f28f6ffd640 1 -- 172.21.15.22:0/836326105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f2908020ac0 con 0x558020e5b4d0 2024-09-21T05:44:52.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290801b050 con 0x558020e5b4d0 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f28f6ffd640 1 -- 172.21.15.22:0/836326105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f2908038050 con 0x558020e5b4d0 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 10 client.7098 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x558020ecd6a0 con 0x558020e5b4d0 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f28f6ffd640 1 -- 172.21.15.22:0/836326105 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f290801b050 con 0x558020e5b4d0 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f28f6ffd640 1 -- 172.21.15.22:0/836326105 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f290803b020 con 0x558020e5b4d0 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 10 client.7098 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 10 client.7098 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 20 client.7098 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x558020e42f60 0x55802127ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 4 client.7098 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580211a09b0 con 0x558020e42f60 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.591+0000 7f29177fe640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x558020e42f60 0x55802127ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.592+0000 7f28f6ffd640 10 client.7098 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.592+0000 7f29177fe640 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x558020e42f60 0x55802127ec40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d14c67209c2212 server_cookie=38b6a024d8bdf72b in_seq=0 out_seq=0 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.592+0000 7f28f6ffd640 1 -- 172.21.15.22:0/836326105 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5580211a09b0 con 0x558020e42f60 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.592+0000 7f28f6ffd640 10 client.7098 handle_command_reply: tid=0 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.592+0000 7f29255c0740 1 client.7098 shutdown 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 msgr2=0x55802126d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x55802126d600 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2908014820 tx=0x7f29080206a0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 shutdown_connections 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x558020e42f60 0x55802127ec40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021242db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f28f6ffd640 0 client.7098 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 --2- 172.21.15.22:0/836326105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x55802126d600 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 >> 172.21.15.22:0/836326105 conn(0x558020e784e0 msgr2=0x558021270a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 shutdown_connections 2024-09-21T05:44:52.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 1 -- 172.21.15.22:0/836326105 wait complete. 2024-09-21T05:44:52.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.593+0000 7f29255c0740 20 client.7098 trim_cache size 0 max 16384 2024-09-21T05:44:52.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.611+0000 7f29255c0740 1 Processor -- start 2024-09-21T05:44:52.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.611+0000 7f29255c0740 1 -- start start 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021226710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021294970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f29255c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021294eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f29255c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2918202450 con 0x558020e84390 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2918201470 con 0x558020e636b0 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f29180b4ac0 con 0x558020e5b4d0 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f291cb0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021294eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f291cb0f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021294eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53548/0 (socket says 172.21.15.22:53548) 2024-09-21T05:44:52.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.612+0000 7f291cb0f640 1 -- 172.21.15.22:0/1014877089 learned_addr learned my addr 172.21.15.22:0/1014877089 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f29177fe640 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021294970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f2917fff640 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021226710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f291cb0f640 1 -- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 msgr2=0x558021226710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f291cb0f640 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021226710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f291cb0f640 1 -- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 msgr2=0x558021294970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f291cb0f640 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021294970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f291cb0f640 1 -- 172.21.15.22:0/1014877089 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558021229490 con 0x558020e84390 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f291cb0f640 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021294eb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2910009f60 tx=0x7f2910009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4daee3bff31a6bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f28fd7fa640 1 -- 172.21.15.22:0/1014877089 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910020070 con 0x558020e84390 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f28fd7fa640 1 -- 172.21.15.22:0/1014877089 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2910011790 con 0x558020e84390 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.613+0000 7f28fd7fa640 1 -- 172.21.15.22:0/1014877089 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2910011be0 con 0x558020e84390 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 -- 172.21.15.22:0/1014877089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 msgr2=0x558021294eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021294eb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2910009f60 tx=0x7f2910009f90 comp rx=0 tx=0).stop 2024-09-21T05:44:52.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 -- 172.21.15.22:0/1014877089 shutdown_connections 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e84390 0x558021294eb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021294970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 --2- 172.21.15.22:0/1014877089 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e5b4d0 0x558021226710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 -- 172.21.15.22:0/1014877089 >> 172.21.15.22:0/1014877089 conn(0x558020e784e0 msgr2=0x55802122bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 -- 172.21.15.22:0/1014877089 shutdown_connections 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.614+0000 7f29255c0740 1 -- 172.21.15.22:0/1014877089 wait complete. 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.615+0000 7f29255c0740 1 -- start start 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.615+0000 7f29255c0740 1 Processor -- start 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x5580212df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021224dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f291806ecb0 con 0x558020e5b4d0 2024-09-21T05:44:52.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5580212258c0 con 0x558020e636b0 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55802126dce0 con 0x558020e84390 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29255c0740 10 client.0 _ll_register_callbacks cb 0x7f2908023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f2917fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x5580212df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f2917fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x5580212df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53552/0 (socket says 172.21.15.22:53552) 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f2917fff640 1 -- 172.21.15.22:0/3633030846 learned_addr learned my addr 172.21.15.22:0/3633030846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f29177fe640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021224dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.616+0000 7f291cb0f640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f2917fff640 1 -- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 msgr2=0x5580212f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f2917fff640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f2917fff640 1 -- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 msgr2=0x558021224dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f2917fff640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021224dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f2917fff640 1 -- 172.21.15.22:0/3633030846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580212f3650 con 0x558020e5b4d0 2024-09-21T05:44:52.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f2917fff640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x5580212df340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2908014820 tx=0x7f2908020f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a43be8c22245a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f28f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f28f6ffd640 1 -- 172.21.15.22:0/3633030846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908025040 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f28f6ffd640 1 -- 172.21.15.22:0/3633030846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5795+0+0 (secure 0 0 0) 0x7f290802b850 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2910011790 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f28f6ffd640 1 -- 172.21.15.22:0/3633030846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f2908049050 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f29255c0740 10 client.7104 fetch_fsmap learned FSMap version 69 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x558021225310 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.617+0000 7f28f6ffd640 1 -- 172.21.15.22:0/3633030846 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2910011790 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f28f6ffd640 1 -- 172.21.15.22:0/3633030846 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f290804c020 con 0x558020e5b4d0 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 10 client.7104 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 10 client.7104 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:44:52.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 20 client.7104 populate_metadata read hostname 'smithi022' 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5580212e1fc0 0x5580212e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 4 client.7104 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55802119c460 con 0x5580212e1fc0 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29177fe640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5580212e1fc0 0x5580212e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f28f6ffd640 10 client.7104 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29177fe640 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5580212e1fc0 0x5580212e4f70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8fd62db22de1a2 server_cookie=2c6ba1ffeb0384dd in_seq=0 out_seq=0 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f28f6ffd640 1 -- 172.21.15.22:0/3633030846 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+864 (crc 0 0 0) 0x55802119c460 con 0x5580212e1fc0 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f28f6ffd640 10 client.7104 handle_command_reply: tid=0 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.618+0000 7f29255c0740 1 client.7104 shutdown 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 msgr2=0x5580212df340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x5580212df340 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2908014820 tx=0x7f2908020f40 comp rx=0 tx=0).stop 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 shutdown_connections 2024-09-21T05:44:52.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x5580212e1fc0 0x5580212e4f70 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x558020e84390 0x5580212f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f28f6ffd640 0 client.7104 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x558020e636b0 0x558021224dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 --2- 172.21.15.22:0/3633030846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x558020e5b4d0 0x5580212df340 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 >> 172.21.15.22:0/3633030846 conn(0x558020e784e0 msgr2=0x5580212948e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 shutdown_connections 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 1 -- 172.21.15.22:0/3633030846 wait complete. 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f29255c0740 20 client.7104 trim_cache size 0 max 16384 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:44:52.629 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "7d0ab2bf-0671-4a7a-9187-e82cb7a5967d": { 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "7d0ab2bf-0671-4a7a-9187-e82cb7a5967d", 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,repair" 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "7d0ab2bf-0671-4a7a-9187-e82cb7a5967d": { 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:44:52.445525+0000", 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:44:52.630 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "392e9f96-e85a-42b7-9ebe-58f3ae77d98b": { 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:44:20.999933+0000", 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:44:52.631 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:44:52.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2904078b90 msgr2=0x7f290407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2904078b90 0x7f290407b0a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2908001720 tx=0x7f2908009040 comp rx=0 tx=0).stop 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 msgr2=0x7f2918156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 0x7f2918156ab0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f290c034040 tx=0x7f290c0038a0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 shutdown_connections 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f29180c8a70 0x7f29180c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f29180b9b60 0x7f2918156ab0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2904078b90 0x7f290407b0a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 --2- 172.21.15.22:0/4119155227 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f29180b3a10 0x7f291815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.619+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 >> 172.21.15.22:0/4119155227 conn(0x7f291809d8f0 msgr2=0x7f29180b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:44:52.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.620+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 shutdown_connections 2024-09-21T05:44:52.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:44:52.620+0000 7f291e572640 1 -- 172.21.15.22:0/4119155227 wait complete. 2024-09-21T05:44:52.634 INFO:tasks.cephfs.filesystem:scrub status for tag:7d0ab2bf-0671-4a7a-9187-e82cb7a5967d - {'path': '/dir', 'tag': '7d0ab2bf-0671-4a7a-9187-e82cb7a5967d', 'options': 'recursive,repair'} 2024-09-21T05:45:22.300 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:45:22.303 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:45:22.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:22.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.767+0000 7f258d494640 1 -- 172.21.15.22:0/2803996066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880b64e0 msgr2=0x7f25880d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.767+0000 7f258d494640 1 --2- 172.21.15.22:0/2803996066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880b64e0 0x7f25880d25b0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f258808ccf0 tx=0x7f258002a8f0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 -- 172.21.15.22:0/2803996066 shutdown_connections 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 --2- 172.21.15.22:0/2803996066 >> v2:172.21.15.134:3300/0 conn(0x7f25880cf540 0x7f25880b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 --2- 172.21.15.22:0/2803996066 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880b64e0 0x7f25880d25b0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 --2- 172.21.15.22:0/2803996066 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25880b9b60 0x7f25880d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 -- 172.21.15.22:0/2803996066 >> 172.21.15.22:0/2803996066 conn(0x7f258809d7f0 msgr2=0x7f25880d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 -- 172.21.15.22:0/2803996066 shutdown_connections 2024-09-21T05:45:22.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.768+0000 7f258d494640 1 -- 172.21.15.22:0/2803996066 wait complete. 2024-09-21T05:45:22.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.770+0000 7f258d494640 1 Processor -- start 2024-09-21T05:45:22.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.770+0000 7f258d494640 1 -- start start 2024-09-21T05:45:22.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f258d494640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25880b64e0 0x7f258815c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f258d494640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25880b9b60 0x7f25880a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f258d494640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 0x7f25880a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f258d494640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f25880d2f90 con 0x7f25880cf540 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f258d494640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25880d2e10 con 0x7f25880b64e0 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f258d494640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f25880d3110 con 0x7f25880b9b60 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 0x7f25880a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 0x7f25880a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33334/0 (socket says 172.21.15.22:33334) 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f25877fe640 1 -- 172.21.15.22:0/3375580396 learned_addr learned my addr 172.21.15.22:0/3375580396 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25880b64e0 0x7f258815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.771+0000 7f25867fc640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25880b9b60 0x7f25880a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f25877fe640 1 -- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25880b9b60 msgr2=0x7f25880a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f25877fe640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25880b9b60 0x7f25880a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f25877fe640 1 -- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25880b64e0 msgr2=0x7f258815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f25877fe640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25880b64e0 0x7f258815c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f25877fe640 1 -- 172.21.15.22:0/3375580396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258815df60 con 0x7f25880cf540 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f25877fe640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 0x7f25880a1b50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2580033040 tx=0x7f2580003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f256f7fe640 1 -- 172.21.15.22:0/3375580396 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580004240 con 0x7f25880cf540 2024-09-21T05:45:22.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25881f9290 con 0x7f25880cf540 2024-09-21T05:45:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f256f7fe640 1 -- 172.21.15.22:0/3375580396 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f258002d070 con 0x7f25880cf540 2024-09-21T05:45:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f256f7fe640 1 -- 172.21.15.22:0/3375580396 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800046a0 con 0x7f25880cf540 2024-09-21T05:45:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.772+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25881f9850 con 0x7f25880cf540 2024-09-21T05:45:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.774+0000 7f256f7fe640 1 -- 172.21.15.22:0/3375580396 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2580031480 con 0x7f25880cf540 2024-09-21T05:45:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.774+0000 7f256f7fe640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2574078b90 0x7f257407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.774+0000 7f256f7fe640 1 -- 172.21.15.22:0/3375580396 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f25800bc880 con 0x7f25880cf540 2024-09-21T05:45:22.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.775+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2574078b90 0x7f257407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.775+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2574078b90 0x7f257407b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f257800bab0 tx=0x7f257800b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.776+0000 7f2594400740 1 Processor -- start 2024-09-21T05:45:22.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.776+0000 7f2594400740 1 -- start start 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3a7390 0x55d91c793a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c768860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2594400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2594400740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d91c6c1a60 con 0x55d91c3866b0 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d91c6b0e90 con 0x55d91c37e4d0 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d91c6b4ca0 con 0x55d91c3a7390 2024-09-21T05:45:22.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33336/0 (socket says 172.21.15.22:33336) 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f25877fe640 1 -- 172.21.15.22:0/3106098450 learned_addr learned my addr 172.21.15.22:0/3106098450 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3a7390 0x55d91c793a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.777+0000 7f25867fc640 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c768860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25877fe640 1 -- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3a7390 msgr2=0x55d91c793a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25877fe640 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3a7390 0x55d91c793a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25877fe640 1 -- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 msgr2=0x55d91c768860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25877fe640 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c768860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25877fe640 1 -- 172.21.15.22:0/3106098450 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91c4897d0 con 0x55d91c3866b0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25867fc640 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c768860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f25877fe640 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768da0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f25800bacb0 tx=0x7f2580083e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14851b760b886dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f256cff9640 1 -- 172.21.15.22:0/3106098450 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580084120 con 0x55d91c3866b0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f256cff9640 1 -- 172.21.15.22:0/3106098450 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2580036020 con 0x55d91c3866b0 2024-09-21T05:45:22.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.778+0000 7f256cff9640 1 -- 172.21.15.22:0/3106098450 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580084610 con 0x55d91c3866b0 2024-09-21T05:45:22.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 -- 172.21.15.22:0/3106098450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 msgr2=0x55d91c768da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768da0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f25800bacb0 tx=0x7f2580083e40 comp rx=0 tx=0).stop 2024-09-21T05:45:22.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 -- 172.21.15.22:0/3106098450 shutdown_connections 2024-09-21T05:45:22.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768da0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c768860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 --2- 172.21.15.22:0/3106098450 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3a7390 0x55d91c793a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 -- 172.21.15.22:0/3106098450 >> 172.21.15.22:0/3106098450 conn(0x55d91c39b4e0 msgr2=0x55d91c74ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 -- 172.21.15.22:0/3106098450 shutdown_connections 2024-09-21T05:45:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.779+0000 7f2594400740 1 -- 172.21.15.22:0/3106098450 wait complete. 2024-09-21T05:45:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.780+0000 7f2594400740 1 -- start start 2024-09-21T05:45:22.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.780+0000 7f2594400740 1 Processor -- start 2024-09-21T05:45:22.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c765420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3866b0 0x55d91c78c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c78cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d91c421740 con 0x55d91c3a7390 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d91c656970 con 0x55d91c37e4d0 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d91c6b0e90 con 0x55d91c3866b0 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f2594400740 10 client.0 _ll_register_callbacks cb 0x55d91c665780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c78cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c78cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33344/0 (socket says 172.21.15.22:33344) 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.781+0000 7f25877fe640 1 -- 172.21.15.22:0/3897526790 learned_addr learned my addr 172.21.15.22:0/3897526790 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c765420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25867fc640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3866b0 0x55d91c78c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25877fe640 1 -- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3866b0 msgr2=0x55d91c78c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25877fe640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3866b0 0x55d91c78c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25877fe640 1 -- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 msgr2=0x55d91c765420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25877fe640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c765420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25877fe640 1 -- 172.21.15.22:0/3897526790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91c412970 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f25877fe640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c78cbc0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2580005fd0 tx=0x7f25800bacb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa4e9f169638c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2565ffb640 1 -- 172.21.15.22:0/3897526790 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580089040 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2565ffb640 1 -- 172.21.15.22:0/3897526790 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f258008da60 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2580036020 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2565ffb640 1 -- 172.21.15.22:0/3897526790 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f2580096050 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2594400740 10 client.7116 fetch_fsmap learned FSMap version 69 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55d91c3f06a0 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.782+0000 7f2565ffb640 1 -- 172.21.15.22:0/3897526790 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2580036020 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2565ffb640 1 -- 172.21.15.22:0/3897526790 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f2580099020 con 0x55d91c3a7390 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2594400740 10 client.7116 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2594400740 10 client.7116 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2594400740 20 client.7116 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:22.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2594400740 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c365f60 0x55d91c7af470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2594400740 4 client.7116 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d91c6c39b0 con 0x55d91c365f60 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c365f60 0x55d91c7af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2565ffb640 10 client.7116 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.783+0000 7f2586ffd640 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c365f60 0x55d91c7af470 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e313cf9546f6fb21 server_cookie=dd526a6ea64d0499 in_seq=0 out_seq=0 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2565ffb640 1 -- 172.21.15.22:0/3897526790 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d91c6c39b0 con 0x55d91c365f60 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2565ffb640 10 client.7116 handle_command_reply: tid=0 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 client.7116 shutdown 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 msgr2=0x55d91c78cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c78cbc0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2580005fd0 tx=0x7f25800bacb0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 shutdown_connections 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c365f60 0x55d91c7af470 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c78cbc0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c3866b0 0x55d91c78c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 --2- 172.21.15.22:0/3897526790 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c37e4d0 0x55d91c765420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2565ffb640 0 client.7116 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 >> 172.21.15.22:0/3897526790 conn(0x55d91c39b4e0 msgr2=0x55d91c775c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 shutdown_connections 2024-09-21T05:45:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 1 -- 172.21.15.22:0/3897526790 wait complete. 2024-09-21T05:45:22.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.784+0000 7f2594400740 20 client.7116 trim_cache size 0 max 16384 2024-09-21T05:45:22.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.803+0000 7f2594400740 1 Processor -- start 2024-09-21T05:45:22.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.803+0000 7f2594400740 1 -- start start 2024-09-21T05:45:22.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c775af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f2594400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3a7390 0x55d91c768f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f2594400740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2588202150 con 0x55d91c3866b0 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f258815f900 con 0x55d91c3a7390 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f258815ff50 con 0x55d91c37e4d0 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33360/0 (socket says 172.21.15.22:33360) 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 -- 172.21.15.22:0/1310104762 learned_addr learned my addr 172.21.15.22:0/1310104762 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 -- 172.21.15.22:0/1310104762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 msgr2=0x55d91c775af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c775af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 -- 172.21.15.22:0/1310104762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3a7390 msgr2=0x55d91c768f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3a7390 0x55d91c768f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 -- 172.21.15.22:0/1310104762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91c765790 con 0x55d91c3866b0 2024-09-21T05:45:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.804+0000 7f25867fc640 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768a50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f257c0044e0 tx=0x7f257c00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6720c7c695b604e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2567fff640 1 -- 172.21.15.22:0/1310104762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c003a70 con 0x55d91c3866b0 2024-09-21T05:45:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2567fff640 1 -- 172.21.15.22:0/1310104762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f257c003d00 con 0x55d91c3866b0 2024-09-21T05:45:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2567fff640 1 -- 172.21.15.22:0/1310104762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0126b0 con 0x55d91c3866b0 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 -- 172.21.15.22:0/1310104762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 msgr2=0x55d91c768a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768a50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f257c0044e0 tx=0x7f257c00bab0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 -- 172.21.15.22:0/1310104762 shutdown_connections 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3a7390 0x55d91c768f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3866b0 0x55d91c768a50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 --2- 172.21.15.22:0/1310104762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c775af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 -- 172.21.15.22:0/1310104762 >> 172.21.15.22:0/1310104762 conn(0x55d91c39b4e0 msgr2=0x55d91c749500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 -- 172.21.15.22:0/1310104762 shutdown_connections 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.805+0000 7f2594400740 1 -- 172.21.15.22:0/1310104762 wait complete. 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.806+0000 7f2594400740 1 -- start start 2024-09-21T05:45:22.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.806+0000 7f2594400740 1 Processor -- start 2024-09-21T05:45:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c7f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3866b0 0x55d91c795c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c74e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55d91c74a4c0 con 0x55d91c3a7390 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55d91c6c1a60 con 0x55d91c3866b0 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55d91c795350 con 0x55d91c37e4d0 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2594400740 10 client.0 _ll_register_callbacks cb 0x7f257800b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c74e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c74e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33368/0 (socket says 172.21.15.22:33368) 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 -- 172.21.15.22:0/2623758625 learned_addr learned my addr 172.21.15.22:0/2623758625 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2586ffd640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c7f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25867fc640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3866b0 0x55d91c795c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 -- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 msgr2=0x55d91c7f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c7f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 -- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3866b0 msgr2=0x55d91c795c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3866b0 0x55d91c795c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 -- 172.21.15.22:0/2623758625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91c765000 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f25877fe640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c74e770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f258008d390 tx=0x7f258002aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae715c786f1bf6a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2565ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.808+0000 7f2565ffb640 1 -- 172.21.15.22:0/2623758625 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580089040 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2565ffb640 1 -- 172.21.15.22:0/2623758625 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f2580004b40 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f257c003d00 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2565ffb640 1 -- 172.21.15.22:0/2623758625 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f25800b2050 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 10 client.7122 fetch_fsmap learned FSMap version 69 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55d91c765000 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2565ffb640 1 -- 172.21.15.22:0/2623758625 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f257c003d00 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2565ffb640 1 -- 172.21.15.22:0/2623758625 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f25800b5020 con 0x55d91c3a7390 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 10 client.7122 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 10 client.7122 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:45:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 20 client.7122 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c7961c0 0x55d91c74ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 4 client.7122 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d91c6bf460 con 0x55d91c7961c0 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2586ffd640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c7961c0 0x55d91c74ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2565ffb640 10 client.7122 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.809+0000 7f2586ffd640 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c7961c0 0x55d91c74ecb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76508c6feb27a9fb server_cookie=ba8992d8d1ec4a42 in_seq=0 out_seq=0 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2565ffb640 1 -- 172.21.15.22:0/2623758625 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+664 (crc 0 0 0) 0x55d91c6bf460 con 0x55d91c7961c0 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2565ffb640 10 client.7122 handle_command_reply: tid=0 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 client.7122 shutdown 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 msgr2=0x55d91c74e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c74e770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f258008d390 tx=0x7f258002aea0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 shutdown_connections 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x55d91c7961c0 0x55d91c74ecb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55d91c3a7390 0x55d91c74e770 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55d91c3866b0 0x55d91c795c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 --2- 172.21.15.22:0/2623758625 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55d91c37e4d0 0x55d91c7f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 >> 172.21.15.22:0/2623758625 conn(0x55d91c39b4e0 msgr2=0x55d91c78f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2565ffb640 0 client.7122 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 shutdown_connections 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 1 -- 172.21.15.22:0/2623758625 wait complete. 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.810+0000 7f2594400740 20 client.7122 trim_cache size 0 max 16384 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: "7d0ab2bf-0671-4a7a-9187-e82cb7a5967d": { 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:44:52.445525+0000", 2024-09-21T05:45:22.816 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "392e9f96-e85a-42b7-9ebe-58f3ae77d98b": { 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:44:20.999933+0000", 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:45:22.817 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:45:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2574078b90 msgr2=0x7f257407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2574078b90 0x7f257407b0a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f257800bab0 tx=0x7f257800b260 comp rx=0 tx=0).stop 2024-09-21T05:45:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 msgr2=0x7f25880a1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 0x7f25880a1b50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2580033040 tx=0x7f2580003c20 comp rx=0 tx=0).stop 2024-09-21T05:45:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 shutdown_connections 2024-09-21T05:45:22.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25880cf540 0x7f25880a1b50 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2574078b90 0x7f257407b0a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f25880b9b60 0x7f25880a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 --2- 172.21.15.22:0/3375580396 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25880b64e0 0x7f258815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 >> 172.21.15.22:0/3375580396 conn(0x7f258809d7f0 msgr2=0x7f25880d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 shutdown_connections 2024-09-21T05:45:22.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.811+0000 7f258d494640 1 -- 172.21.15.22:0/3375580396 wait complete. 2024-09-21T05:45:22.824 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:45:22.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:45:22.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 -- 172.21.15.22:0/2251231159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 msgr2=0x7f82200d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 --2- 172.21.15.22:0/2251231159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f82200d0d80 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8214005fd0 tx=0x7f821402a530 comp rx=0 tx=0).stop 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 -- 172.21.15.22:0/2251231159 shutdown_connections 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 --2- 172.21.15.22:0/2251231159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 0x7f82200d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 --2- 172.21.15.22:0/2251231159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f82200d0d80 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 -- 172.21.15.22:0/2251231159 >> 172.21.15.22:0/2251231159 conn(0x7f8220072ae0 msgr2=0x7f8220072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 -- 172.21.15.22:0/2251231159 shutdown_connections 2024-09-21T05:45:22.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.938+0000 7f8225c27640 1 -- 172.21.15.22:0/2251231159 wait complete. 2024-09-21T05:45:22.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.940+0000 7f8225c27640 1 Processor -- start 2024-09-21T05:45:22.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.940+0000 7f8225c27640 1 -- start start 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.941+0000 7f8225c27640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82200b51c0 0x7f82200b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f8225c27640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f822006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f8225c27640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 0x7f822006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f8225c27640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f82200cf220 con 0x7f82200ccbd0 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f8225c27640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f82200cf0a0 con 0x7f82200b51c0 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f8225c27640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f82200cf3a0 con 0x7f82200d12c0 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f822006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f822006eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33380/0 (socket says 172.21.15.22:33380) 2024-09-21T05:45:22.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 -- 172.21.15.22:0/4219541464 learned_addr learned my addr 172.21.15.22:0/4219541464 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821f7fe640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82200b51c0 0x7f82200b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821ffff640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 0x7f822006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 -- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 msgr2=0x7f822006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 0x7f822006f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 -- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82200b51c0 msgr2=0x7f82200b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82200b51c0 0x7f82200b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 -- 172.21.15.22:0/4219541464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8220156320 con 0x7f82200ccbd0 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821f7fe640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82200b51c0 0x7f82200b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821ffff640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 0x7f822006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f821effd640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f822006eaf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8214034040 tx=0x7f8214003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.942+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821403e030 con 0x7f82200ccbd0 2024-09-21T05:45:22.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.943+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82140047a0 con 0x7f82200ccbd0 2024-09-21T05:45:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.943+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8220156600 con 0x7f82200ccbd0 2024-09-21T05:45:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.943+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214035040 con 0x7f82200ccbd0 2024-09-21T05:45:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.943+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8220156bc0 con 0x7f82200ccbd0 2024-09-21T05:45:22.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.944+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82200cf890 con 0x7f82200ccbd0 2024-09-21T05:45:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.944+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8214041030 con 0x7f82200ccbd0 2024-09-21T05:45:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.945+0000 7f8207fff640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f820c078b90 0x7f820c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.945+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f82140bb7d0 con 0x7f82200ccbd0 2024-09-21T05:45:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.945+0000 7f821f7fe640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f820c078b90 0x7f820c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:22.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.946+0000 7f821f7fe640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f820c078b90 0x7f820c07b0a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f821000a830 tx=0x7f8210008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:22.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:22.950+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8214083aa0 con 0x7f82200ccbd0 2024-09-21T05:45:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.179+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82200693e0 con 0x7f82200ccbd0 2024-09-21T05:45:23.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.179+0000 7f8207fff640 1 -- 172.21.15.22:0/4219541464 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5227 (secure 0 0 0) 0x7f82140833a0 con 0x7f82200ccbd0 2024-09-21T05:45:23.181 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:45:23.181 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":69,"btime":"2024-09-21T05:44:29:414897+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:29.112432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":6}]} 2024-09-21T05:45:23.182 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 69 2024-09-21T05:45:23.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f820c078b90 msgr2=0x7f820c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f820c078b90 0x7f820c07b0a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f821000a830 tx=0x7f8210008040 comp rx=0 tx=0).stop 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 msgr2=0x7f822006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f822006eaf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8214034040 tx=0x7f8214003970 comp rx=0 tx=0).stop 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 shutdown_connections 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f820c078b90 0x7f820c07b0a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f82200d12c0 0x7f822006f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f82200ccbd0 0x7f822006eaf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 --2- 172.21.15.22:0/4219541464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f82200b51c0 0x7f82200b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 >> 172.21.15.22:0/4219541464 conn(0x7f8220072ae0 msgr2=0x7f822015bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 shutdown_connections 2024-09-21T05:45:23.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.181+0000 7f8225c27640 1 -- 172.21.15.22:0/4219541464 wait complete. 2024-09-21T05:45:23.195 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 -- 172.21.15.22:0/3508802436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80d14f0 msgr2=0x7f71b8162b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 --2- 172.21.15.22:0/3508802436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80d14f0 0x7f71b8162b40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f71b80d1d20 tx=0x7f71b002a820 comp rx=0 tx=0).stop 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 -- 172.21.15.22:0/3508802436 shutdown_connections 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 --2- 172.21.15.22:0/3508802436 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80d14f0 0x7f71b8162b40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 --2- 172.21.15.22:0/3508802436 >> v2:172.21.15.134:3300/0 conn(0x7f71b80b9b60 0x7f71b80d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 -- 172.21.15.22:0/3508802436 >> 172.21.15.22:0/3508802436 conn(0x7f71b809d8f0 msgr2=0x7f71b80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 -- 172.21.15.22:0/3508802436 shutdown_connections 2024-09-21T05:45:23.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.311+0000 7f71be9a1640 1 -- 172.21.15.22:0/3508802436 wait complete. 2024-09-21T05:45:23.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.313+0000 7f71be9a1640 1 Processor -- start 2024-09-21T05:45:23.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.313+0000 7f71be9a1640 1 -- start start 2024-09-21T05:45:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71be9a1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 0x7f71b81576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71be9a1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b80b9b60 0x7f71b80b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71be9a1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b80d14f0 0x7f71b81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71be9a1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8163080 con 0x7f71b80b3a10 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71be9a1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71b80d1fa0 con 0x7f71b80b9b60 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71be9a1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71b8163200 con 0x7f71b80d14f0 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 0x7f71b81576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 0x7f71b81576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33392/0 (socket says 172.21.15.22:33392) 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71b7fff640 1 -- 172.21.15.22:0/225987423 learned_addr learned my addr 172.21.15.22:0/225987423 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71b77fe640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b80b9b60 0x7f71b80b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.314+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b80d14f0 0x7f71b81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b7fff640 1 -- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b80d14f0 msgr2=0x7f71b81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b7fff640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b80d14f0 0x7f71b81598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b7fff640 1 -- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b80b9b60 msgr2=0x7f71b80b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b7fff640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b80b9b60 0x7f71b80b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b7fff640 1 -- 172.21.15.22:0/225987423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b80b4f30 con 0x7f71b80b3a10 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b7fff640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 0x7f71b81576b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c970 tx=0x7f71a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b4ff9640 1 -- 172.21.15.22:0/225987423 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a8007bb0 con 0x7f71b80b3a10 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b4ff9640 1 -- 172.21.15.22:0/225987423 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a8007e40 con 0x7f71b80b3a10 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b80b67f0 con 0x7f71b80b3a10 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71b4ff9640 1 -- 172.21.15.22:0/225987423 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a800f690 con 0x7f71b80b3a10 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.315+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b80b6db0 con 0x7f71b80b3a10 2024-09-21T05:45:23.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.317+0000 7f71b4ff9640 1 -- 172.21.15.22:0/225987423 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71a800f830 con 0x7f71b80b3a10 2024-09-21T05:45:23.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.318+0000 7f71b4ff9640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f71a4078bc0 0x7f71a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.318+0000 7f71b4ff9640 1 -- 172.21.15.22:0/225987423 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f71a809b3c0 con 0x7f71b80b3a10 2024-09-21T05:45:23.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.318+0000 7f71b77fe640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f71a4078bc0 0x7f71a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.319+0000 7f71c59ea740 1 Processor -- start 2024-09-21T05:45:23.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.319+0000 7f71c59ea740 1 -- start start 2024-09-21T05:45:23.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.319+0000 7f71b77fe640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f71a4078bc0 0x7f71a407b0d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f71b815c2f0 tx=0x7f71ac007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a4a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71c59ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5655029929a0 con 0x565502678350 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565502981dd0 con 0x56550264f490 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x565502985be0 con 0x565502657670 2024-09-21T05:45:23.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a49d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33402/0 (socket says 172.21.15.22:33402) 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 -- 172.21.15.22:0/3259022703 learned_addr learned my addr 172.21.15.22:0/3259022703 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b77fe640 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a4a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 -- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 msgr2=0x565502a671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 -- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 msgr2=0x565502a4a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a4a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71b7fff640 1 -- 172.21.15.22:0/3259022703 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550291a700 con 0x565502678350 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.320+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.321+0000 7f71b7fff640 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a49d80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f71a805f990 tx=0x7f71a8062ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa1426477975dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.321+0000 7f719dffb640 1 -- 172.21.15.22:0/3259022703 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a806cab0 con 0x565502678350 2024-09-21T05:45:23.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.321+0000 7f719dffb640 1 -- 172.21.15.22:0/3259022703 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a809b0c0 con 0x565502678350 2024-09-21T05:45:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.321+0000 7f719dffb640 1 -- 172.21.15.22:0/3259022703 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a80891d0 con 0x565502678350 2024-09-21T05:45:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 -- 172.21.15.22:0/3259022703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 msgr2=0x565502a49d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a49d80 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f71a805f990 tx=0x7f71a8062ce0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 -- 172.21.15.22:0/3259022703 shutdown_connections 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a4a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3259022703 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a49d80 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 -- 172.21.15.22:0/3259022703 >> 172.21.15.22:0/3259022703 conn(0x56550266c4a0 msgr2=0x565502a20040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 -- 172.21.15.22:0/3259022703 shutdown_connections 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.322+0000 7f71c59ea740 1 -- 172.21.15.22:0/3259022703 wait complete. 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.323+0000 7f71c59ea740 1 -- start start 2024-09-21T05:45:23.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.323+0000 7f71c59ea740 1 Processor -- start 2024-09-21T05:45:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a37b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5655026f2700 con 0x565502678350 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565502927340 con 0x56550264f490 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x565502a4a800 con 0x565502657670 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71c59ea740 10 client.0 _ll_register_callbacks cb 0x565502925ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71bcf3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71bcf3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a68610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33412/0 (socket says 172.21.15.22:33412) 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71bcf3e640 1 -- 172.21.15.22:0/444879698 learned_addr learned my addr 172.21.15.22:0/444879698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71b7fff640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a37b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.324+0000 7f71b77fe640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71bcf3e640 1 -- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 msgr2=0x565502a680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71bcf3e640 1 -- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 msgr2=0x565502a37b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a37b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71bcf3e640 1 -- 172.21.15.22:0/444879698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550275c7c0 con 0x565502678350 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a68610 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f71b002ae10 tx=0x7f71b0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac226dcae0255b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f7196ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f7196ffd640 1 -- 172.21.15.22:0/444879698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0041030 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f7196ffd640 1 -- 172.21.15.22:0/444879698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f71b002f040 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a809b0c0 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f7196ffd640 1 -- 172.21.15.22:0/444879698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f71b004d050 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71c59ea740 10 client.7143 fetch_fsmap learned FSMap version 69 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x5655027bcf50 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.325+0000 7f7196ffd640 1 -- 172.21.15.22:0/444879698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a809b0c0 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f7196ffd640 1 -- 172.21.15.22:0/444879698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f71b0050020 con 0x565502678350 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71c59ea740 10 client.7143 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71c59ea740 10 client.7143 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71c59ea740 20 client.7143 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71c59ea740 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502637130 0x565502a422f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71c59ea740 4 client.7143 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655029948f0 con 0x565502637130 2024-09-21T05:45:23.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71b7fff640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502637130 0x565502a422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f7196ffd640 10 client.7143 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.326+0000 7f71b7fff640 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502637130 0x565502a422f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55bdb894de4cf89e server_cookie=58719583b05a2c2 in_seq=0 out_seq=0 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f7196ffd640 1 -- 172.21.15.22:0/444879698 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5655029948f0 con 0x565502637130 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f7196ffd640 10 client.7143 handle_command_reply: tid=0 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 client.7143 shutdown 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 msgr2=0x565502a68610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a68610 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f71b002ae10 tx=0x7f71b0003ef0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 shutdown_connections 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502637130 0x565502a422f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502678350 0x565502a68610 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502657670 0x565502a680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f7196ffd640 0 client.7143 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 --2- 172.21.15.22:0/444879698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502a37b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 >> 172.21.15.22:0/444879698 conn(0x56550266c4a0 msgr2=0x565502a66790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 shutdown_connections 2024-09-21T05:45:23.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 1 -- 172.21.15.22:0/444879698 wait complete. 2024-09-21T05:45:23.334 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.327+0000 7f71c59ea740 20 client.7143 trim_cache size 0 max 16384 2024-09-21T05:45:23.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.341+0000 7f71c59ea740 1 Processor -- start 2024-09-21T05:45:23.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.341+0000 7f71c59ea740 1 -- start start 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502a1b810 0x565502a200c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550266c4a0 0x565502a88180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 0x565502a886c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71c59ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81f9f80 con 0x56550264f490 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81f92e0 con 0x56550266c4a0 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f71b81f9930 con 0x565502a1b810 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71bcf3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 0x565502a886c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71bcf3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 0x565502a886c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33426/0 (socket says 172.21.15.22:33426) 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71bcf3e640 1 -- 172.21.15.22:0/4271915906 learned_addr learned my addr 172.21.15.22:0/4271915906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:23.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71b7fff640 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502a1b810 0x565502a200c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.342+0000 7f71b77fe640 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550266c4a0 0x565502a88180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71bcf3e640 1 -- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502a1b810 msgr2=0x565502a200c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502a1b810 0x565502a200c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71bcf3e640 1 -- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550266c4a0 msgr2=0x565502a88180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550266c4a0 0x565502a88180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71bcf3e640 1 -- 172.21.15.22:0/4271915906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655026c1660 con 0x56550264f490 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71b77fe640 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550266c4a0 0x565502a88180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 0x565502a886c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f71b00096f0 tx=0x7f71b0003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f2cf8d1667e03f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f719d7fa640 1 -- 172.21.15.22:0/4271915906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0041040 con 0x56550264f490 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f719d7fa640 1 -- 172.21.15.22:0/4271915906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71b0004b40 con 0x56550264f490 2024-09-21T05:45:23.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.343+0000 7f719d7fa640 1 -- 172.21.15.22:0/4271915906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037040 con 0x56550264f490 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 -- 172.21.15.22:0/4271915906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 msgr2=0x565502a886c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 0x565502a886c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f71b00096f0 tx=0x7f71b0003de0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 -- 172.21.15.22:0/4271915906 shutdown_connections 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56550264f490 0x565502a886c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550266c4a0 0x565502a88180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 --2- 172.21.15.22:0/4271915906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x565502a1b810 0x565502a200c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 -- 172.21.15.22:0/4271915906 >> 172.21.15.22:0/4271915906 conn(0x565502a15f80 msgr2=0x565502a377c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 -- 172.21.15.22:0/4271915906 shutdown_connections 2024-09-21T05:45:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 -- 172.21.15.22:0/4271915906 wait complete. 2024-09-21T05:45:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.344+0000 7f71c59ea740 1 -- start start 2024-09-21T05:45:23.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.345+0000 7f71c59ea740 1 Processor -- start 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502ae7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56550266c4a0 0x565502a88630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 0x565502ae5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x565502a38320 con 0x565502a1b810 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x565502a5e0c0 con 0x56550264f490 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x565502a5cb60 con 0x56550266c4a0 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71c59ea740 10 client.0 _ll_register_callbacks cb 0x7f71b0009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 0x565502ae5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 0x565502ae5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33442/0 (socket says 172.21.15.22:33442) 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 -- 172.21.15.22:0/3217735176 learned_addr learned my addr 172.21.15.22:0/3217735176 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71b7fff640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502ae7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71b77fe640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56550266c4a0 0x565502a88630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 -- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56550266c4a0 msgr2=0x565502a88630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56550266c4a0 0x565502a88630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 -- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 msgr2=0x565502ae7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502ae7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.346+0000 7f71bcf3e640 1 -- 172.21.15.22:0/3217735176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565502a88b70 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71bcf3e640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 0x565502ae5090 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f71b002ae10 tx=0x7f71b00329a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4542150555dcf015 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f7196ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f7196ffd640 1 -- 172.21.15.22:0/3217735176 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037050 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f7196ffd640 1 -- 172.21.15.22:0/3217735176 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f71b0032da0 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b0004b40 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f7196ffd640 1 -- 172.21.15.22:0/3217735176 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f71b0041040 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 10 client.7149 fetch_fsmap learned FSMap version 69 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x565502ae55d0 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f7196ffd640 1 -- 172.21.15.22:0/3217735176 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71b0004b40 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f7196ffd640 1 -- 172.21.15.22:0/3217735176 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f71b0050070 con 0x565502a1b810 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 10 client.7149 fetch_fsmap finished waiting for FSMap version 69 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 10 client.7149 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:45:23.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 20 client.7149 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502678350 0x565502a411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 4 client.7149 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.347+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 --> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5655029903a0 con 0x565502678350 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71b7fff640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502678350 0x565502a411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f7196ffd640 10 client.7149 ms_handle_connect on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71b7fff640 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502678350 0x565502a411a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e0c5494beeb26d server_cookie=6fcb3a3d4f8e094 in_seq=0 out_seq=0 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f7196ffd640 1 -- 172.21.15.22:0/3217735176 <== mds.0 v2:172.21.15.22:6832/3207626530 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x5655029903a0 con 0x565502678350 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f7196ffd640 10 client.7149 handle_command_reply: tid=0 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 client.7149 shutdown 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 msgr2=0x565502ae5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 0x565502ae5090 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f71b002ae10 tx=0x7f71b00329a0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 shutdown_connections 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:6832/3207626530,v1:172.21.15.22:6834/3207626530] conn(0x565502678350 0x565502a411a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x565502a1b810 0x565502ae5090 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.353 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56550266c4a0 0x565502a88630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f7196ffd640 0 client.7149 ms_handle_reset on v2:172.21.15.22:6832/3207626530 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 --2- 172.21.15.22:0/3217735176 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56550264f490 0x565502ae7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.348+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 >> 172.21.15.22:0/3217735176 conn(0x565502a15f80 msgr2=0x565502a1ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 shutdown_connections 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71c59ea740 1 -- 172.21.15.22:0/3217735176 wait complete. 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71c59ea740 20 client.7149 trim_cache size 0 max 16384 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:45:23.354 INFO:teuthology.orchestra.run.smithi022.stdout: "damage_type": "backtrace", 2024-09-21T05:45:23.355 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 1213209192, 2024-09-21T05:45:23.355 INFO:teuthology.orchestra.run.smithi022.stdout: "ino": 1099511627777, 2024-09-21T05:45:23.355 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/testdir/hardlink" 2024-09-21T05:45:23.355 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:45:23.355 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f71a4078bc0 msgr2=0x7f71a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f71a4078bc0 0x7f71a407b0d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f71b815c2f0 tx=0x7f71ac007e90 comp rx=0 tx=0).stop 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 msgr2=0x7f71b81576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 0x7f71b81576b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f71a800c970 tx=0x7f71a800ce30 comp rx=0 tx=0).stop 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 shutdown_connections 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f71a4078bc0 0x7f71a407b0d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f71b80d14f0 0x7f71b81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f71b80b9b60 0x7f71b80b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 --2- 172.21.15.22:0/225987423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f71b80b3a10 0x7f71b81576b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 >> 172.21.15.22:0/225987423 conn(0x7f71b809d8f0 msgr2=0x7f71b806e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 shutdown_connections 2024-09-21T05:45:23.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.349+0000 7f71be9a1640 1 -- 172.21.15.22:0/225987423 wait complete. 2024-09-21T05:45:23.362 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:45:23.362 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:45:23.378 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:45:23.378 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:45:23.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 -- 172.21.15.22:0/2858295874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 msgr2=0x7f426c070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 --2- 172.21.15.22:0/2858295874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 0x7f426c070180 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4254009830 tx=0x7f425402a510 comp rx=0 tx=0).stop 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 -- 172.21.15.22:0/2858295874 shutdown_connections 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 --2- 172.21.15.22:0/2858295874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 0x7f426c070180 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 --2- 172.21.15.22:0/2858295874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c0af780 0x7f426c0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 -- 172.21.15.22:0/2858295874 >> 172.21.15.22:0/2858295874 conn(0x7f426c06d7d0 msgr2=0x7f426c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 -- 172.21.15.22:0/2858295874 shutdown_connections 2024-09-21T05:45:23.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.590+0000 7f427101f640 1 -- 172.21.15.22:0/2858295874 wait complete. 2024-09-21T05:45:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.592+0000 7f427101f640 1 Processor -- start 2024-09-21T05:45:23.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.592+0000 7f427101f640 1 -- start start 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f427101f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c0706c0 0x7f426c1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f427101f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 0x7f426c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f427101f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 0x7f426c158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f427101f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f426c073b80 con 0x7f426c0af780 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f427101f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f426c073a00 con 0x7f426c0cd3e0 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f427101f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f426c154fb0 con 0x7f426c0706c0 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 0x7f426c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 0x7f426c147be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33458/0 (socket says 172.21.15.22:33458) 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 -- 172.21.15.22:0/1150381502 learned_addr learned my addr 172.21.15.22:0/1150381502 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426ad9d640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c0706c0 0x7f426c1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426b59e640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 0x7f426c158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 -- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c0706c0 msgr2=0x7f426c1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c0706c0 0x7f426c1476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 -- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 msgr2=0x7f426c158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 0x7f426c158d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.594+0000 7f426a59c640 1 -- 172.21.15.22:0/1150381502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c157f40 con 0x7f426c0af780 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.595+0000 7f426a59c640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 0x7f426c147be0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4254000c00 tx=0x7f4254033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.595+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254031d60 con 0x7f426c0af780 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.595+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4254033e10 con 0x7f426c0af780 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.595+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c158220 con 0x7f426c0af780 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.595+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254033310 con 0x7f426c0af780 2024-09-21T05:45:23.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.595+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c1587e0 con 0x7f426c0af780 2024-09-21T05:45:23.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.596+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426c148300 con 0x7f426c0af780 2024-09-21T05:45:23.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.597+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42540334b0 con 0x7f426c0af780 2024-09-21T05:45:23.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.597+0000 7f42637fe640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f424c078bc0 0x7f424c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:23.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.597+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f42540bbd80 con 0x7f426c0af780 2024-09-21T05:45:23.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.598+0000 7f426ad9d640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f424c078bc0 0x7f424c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:23.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.598+0000 7f426ad9d640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f424c078bc0 0x7f424c07b0d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f425c009a60 tx=0x7f425c010040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:23.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.602+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f425404a030 con 0x7f426c0af780 2024-09-21T05:45:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.800+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f426c0b39c0 con 0x7f426c0af780 2024-09-21T05:45:23.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.800+0000 7f42637fe640 1 -- 172.21.15.22:0/1150381502 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v85) ==== 80+0+124 (secure 0 0 0) 0x7f425403f030 con 0x7f426c0af780 2024-09-21T05:45:23.803 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6837/2104741994 2024-09-22T05:44:14.100657+0000 2024-09-21T05:45:23.803 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6835/2104741994 2024-09-22T05:44:14.100657+0000 2024-09-21T05:45:23.803 INFO:teuthology.orchestra.run.smithi022.stderr:listed 2 entries 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.802+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f424c078bc0 msgr2=0x7f424c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.802+0000 7f427101f640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f424c078bc0 0x7f424c07b0d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f425c009a60 tx=0x7f425c010040 comp rx=0 tx=0).stop 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.802+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 msgr2=0x7f426c147be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.802+0000 7f427101f640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 0x7f426c147be0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f4254000c00 tx=0x7f4254033c20 comp rx=0 tx=0).stop 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 shutdown_connections 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f426c0cd3e0 0x7f426c158d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f426c0af780 0x7f426c147be0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f424c078bc0 0x7f424c07b0d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 --2- 172.21.15.22:0/1150381502 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f426c0706c0 0x7f426c1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 >> 172.21.15.22:0/1150381502 conn(0x7f426c06d7d0 msgr2=0x7f426c0aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 shutdown_connections 2024-09-21T05:45:23.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:23.803+0000 7f427101f640 1 -- 172.21.15.22:0/1150381502 wait complete. 2024-09-21T05:45:23.816 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:45:23.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:23.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:45:23.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:45:23.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:23.845 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:45:23.900 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:45:24.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 -- 172.21.15.22:0/2809319187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80cefd0 msgr2=0x7f84e80cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 --2- 172.21.15.22:0/2809319187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80cefd0 0x7f84e80cf3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84d0005fd0 tx=0x7f84d002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 -- 172.21.15.22:0/2809319187 shutdown_connections 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 --2- 172.21.15.22:0/2809319187 >> v2:172.21.15.134:3300/0 conn(0x7f84e80b42f0 0x7f84e80b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 --2- 172.21.15.22:0/2809319187 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80cefd0 0x7f84e80cf3e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 -- 172.21.15.22:0/2809319187 >> 172.21.15.22:0/2809319187 conn(0x7f84e809d680 msgr2=0x7f84e80d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 -- 172.21.15.22:0/2809319187 shutdown_connections 2024-09-21T05:45:24.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.056+0000 7f84e759e640 1 -- 172.21.15.22:0/2809319187 wait complete. 2024-09-21T05:45:24.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.058+0000 7f84e759e640 1 Processor -- start 2024-09-21T05:45:24.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.058+0000 7f84e759e640 1 -- start start 2024-09-21T05:45:24.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.059+0000 7f84e759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 0x7f84e815afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.059+0000 7f84e759e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80b96b0 0x7f84e81f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.059+0000 7f84e759e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e80cefd0 0x7f84e81f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e759e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80b4ec0 con 0x7f84e80b42f0 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e759e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84e80b4d20 con 0x7f84e80cefd0 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e759e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f84e80b5040 con 0x7f84e80b96b0 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 0x7f84e815afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 0x7f84e815afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33466/0 (socket says 172.21.15.22:33466) 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 -- 172.21.15.22:0/3583483435 learned_addr learned my addr 172.21.15.22:0/3583483435 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e6d9d640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e80cefd0 0x7f84e81f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e5d9b640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80b96b0 0x7f84e81f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 -- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80b96b0 msgr2=0x7f84e81f93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80b96b0 0x7f84e81f93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 -- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e80cefd0 msgr2=0x7f84e81f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e80cefd0 0x7f84e81f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 -- 172.21.15.22:0/3583483435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e81f5990 con 0x7f84e80b42f0 2024-09-21T05:45:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e659c640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 0x7f84e815afe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c7a0 tx=0x7f84d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8003bc0 con 0x7f84e80b42f0 2024-09-21T05:45:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84d8003d80 con 0x7f84e80b42f0 2024-09-21T05:45:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e81f5e10 con 0x7f84e80b42f0 2024-09-21T05:45:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.060+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800aab0 con 0x7f84e80b42f0 2024-09-21T05:45:24.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.061+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e81f8bc0 con 0x7f84e80b42f0 2024-09-21T05:45:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.062+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e81f6330 con 0x7f84e80b42f0 2024-09-21T05:45:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.062+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84d800ac50 con 0x7f84e80b42f0 2024-09-21T05:45:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.063+0000 7f84deffd640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84c8078b90 0x7f84c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.063+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7f84d809aa80 con 0x7f84e80b42f0 2024-09-21T05:45:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.063+0000 7f84e5d9b640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84c8078b90 0x7f84c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.064+0000 7f84e5d9b640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84c8078b90 0x7f84c807b0a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f84d0033000 tx=0x7f84d0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.067+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84e81f6330 con 0x7f84e80b42f0 2024-09-21T05:45:24.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.294+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84e80cebb0 con 0x7f84e80b42f0 2024-09-21T05:45:24.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.294+0000 7f84deffd640 1 -- 172.21.15.22:0/3583483435 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5227 (secure 0 0 0) 0x7f84e80cebb0 con 0x7f84e80b42f0 2024-09-21T05:45:24.296 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:45:24.296 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":69,"btime":"2024-09-21T05:44:29:414897+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:44:29.112432+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":65,"state":"up:active","state_seq":10,"addr":"172.21.15.22:6834/3207626530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3207626530},{"type":"v1","addr":"172.21.15.22:6834","nonce":3207626530}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":6}]} 2024-09-21T05:45:24.297 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 69 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84c8078b90 msgr2=0x7f84c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84c8078b90 0x7f84c807b0a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f84d0033000 tx=0x7f84d0004730 comp rx=0 tx=0).stop 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 msgr2=0x7f84e815afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 0x7f84e815afe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c7a0 tx=0x7f84d800cc60 comp rx=0 tx=0).stop 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 shutdown_connections 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84e80cefd0 0x7f84e81f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84c8078b90 0x7f84c807b0a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84e80b96b0 0x7f84e81f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 --2- 172.21.15.22:0/3583483435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84e80b42f0 0x7f84e815afe0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 >> 172.21.15.22:0/3583483435 conn(0x7f84e809d680 msgr2=0x7f84e806db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 shutdown_connections 2024-09-21T05:45:24.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.296+0000 7f84e759e640 1 -- 172.21.15.22:0/3583483435 wait complete. 2024-09-21T05:45:24.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T05:45:24.448 INFO:teuthology.orchestra.run.smithi022.stdout:{"message":"","return_code":0} 2024-09-21T05:45:24.449 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:45:24.449 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:45:24.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 -- 172.21.15.22:0/1354211407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40d4c20 msgr2=0x7fdff40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 --2- 172.21.15.22:0/1354211407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40d4c20 0x7fdff40b4040 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdff40b7140 tx=0x7fdfec02aa30 comp rx=0 tx=0).stop 2024-09-21T05:45:24.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 -- 172.21.15.22:0/1354211407 shutdown_connections 2024-09-21T05:45:24.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 --2- 172.21.15.22:0/1354211407 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40d4c20 0x7fdff40b4040 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 --2- 172.21.15.22:0/1354211407 >> v2:172.21.15.134:3300/0 conn(0x7fdff40d0ce0 0x7fdff40b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 -- 172.21.15.22:0/1354211407 >> 172.21.15.22:0/1354211407 conn(0x7fdff408bfc0 msgr2=0x7fdff409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 -- 172.21.15.22:0/1354211407 shutdown_connections 2024-09-21T05:45:24.567 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.564+0000 7fdff948c640 1 -- 172.21.15.22:0/1354211407 wait complete. 2024-09-21T05:45:24.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.566+0000 7fdff948c640 1 Processor -- start 2024-09-21T05:45:24.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.566+0000 7fdff948c640 1 -- start start 2024-09-21T05:45:24.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff948c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40cf0b0 0x7fdff40d4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff948c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdff40d0ce0 0x7fdff40d4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff948c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 0x7fdff40ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff948c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40b4a20 con 0x7fdff40ca490 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff948c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdff40b48a0 con 0x7fdff40cf0b0 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff948c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdff40b4ba0 con 0x7fdff40d0ce0 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 0x7fdff40ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 0x7fdff40ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41174/0 (socket says 172.21.15.22:41174) 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff37fe640 1 -- 172.21.15.22:0/2040906309 learned_addr learned my addr 172.21.15.22:0/2040906309 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff27fc640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdff40d0ce0 0x7fdff40d4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.567+0000 7fdff2ffd640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40cf0b0 0x7fdff40d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff37fe640 1 -- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdff40d0ce0 msgr2=0x7fdff40d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff37fe640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdff40d0ce0 0x7fdff40d4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff37fe640 1 -- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40cf0b0 msgr2=0x7fdff40d4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff37fe640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40cf0b0 0x7fdff40d4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff37fe640 1 -- 172.21.15.22:0/2040906309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff40a10e0 con 0x7fdff40ca490 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff37fe640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 0x7fdff40ca8a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdfec02b020 tx=0x7fdfec003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec033050 con 0x7fdff40ca490 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41f5720 con 0x7fdff40ca490 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfec0045f0 con 0x7fdff40ca490 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec038450 con 0x7fdff40ca490 2024-09-21T05:45:24.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.568+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41f5f30 con 0x7fdff40ca490 2024-09-21T05:45:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.569+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff41f59c0 con 0x7fdff40ca490 2024-09-21T05:45:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.570+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfec036070 con 0x7fdff40ca490 2024-09-21T05:45:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.570+0000 7fdfdb7fe640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfe0078b90 0x7fdfe007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.571+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5824+0+0 (secure 0 0 0) 0x7fdfec0bc380 con 0x7fdff40ca490 2024-09-21T05:45:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.571+0000 7fdff2ffd640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfe0078b90 0x7fdfe007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.571+0000 7fdff2ffd640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfe0078b90 0x7fdfe007b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098e0 tx=0x7fdfe4009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.577 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.575+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfec084090 con 0x7fdff40ca490 2024-09-21T05:45:24.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.801+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fdff40b5140 con 0x7fdff40ca490 2024-09-21T05:45:24.803 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:45:24.801+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:45:24.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.803+0000 7fdfdb7fe640 1 -- 172.21.15.22:0/2040906309 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) ==== 161+0+0 (secure 0 0 0) 0x7fdfec088ac0 con 0x7fdff40ca490 2024-09-21T05:45:24.805 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:45:24.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.805+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfe0078b90 msgr2=0x7fdfe007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.805+0000 7fdff948c640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfe0078b90 0x7fdfe007b0a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40098e0 tx=0x7fdfe4009060 comp rx=0 tx=0).stop 2024-09-21T05:45:24.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.805+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 msgr2=0x7fdff40ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.805+0000 7fdff948c640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 0x7fdff40ca8a0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdfec02b020 tx=0x7fdfec003f90 comp rx=0 tx=0).stop 2024-09-21T05:45:24.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 shutdown_connections 2024-09-21T05:45:24.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdff40ca490 0x7fdff40ca8a0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfe0078b90 0x7fdfe007b0a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdff40d0ce0 0x7fdff40d4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 --2- 172.21.15.22:0/2040906309 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdff40cf0b0 0x7fdff40d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 >> 172.21.15.22:0/2040906309 conn(0x7fdff408bfc0 msgr2=0x7fdff4161dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 shutdown_connections 2024-09-21T05:45:24.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.806+0000 7fdff948c640 1 -- 172.21.15.22:0/2040906309 wait complete. 2024-09-21T05:45:24.812 INFO:tasks.ceph.mds.c.smithi022.stderr: -4408> 2024-09-21T05:44:20.794+0000 7f64756ba640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-09-21T05:45:24.812 INFO:tasks.ceph.mds.c.smithi022.stderr: -4233> 2024-09-21T05:44:20.999+0000 7f64756ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0) n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x56308c75c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-21T05:45:24.812 INFO:tasks.ceph.mds.c.smithi022.stderr: -4087> 2024-09-21T05:44:22.304+0000 7f64796c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:45:24.814 INFO:tasks.ceph.mds.c.smithi022.stderr: -2296> 2024-09-21T05:44:52.446+0000 7f64756ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0)->f(v1 m2024-09-21T05:44:12.387045+0000) n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1)->n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56308c75c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-21T05:44:12.387045+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-21T05:44:12.506044+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-09-21T05:45:24.820 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-09-21T05:45:24.868 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:45:24.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.873+0000 7ff13bcc9740 1 -- start start 2024-09-21T05:45:24.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.873+0000 7ff13bcc9740 1 Processor -- start 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfd680 0x55abcda22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfdb00 0x55abcda22000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 0x55abcda22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55abcda4a180 con 0x55abccdfdf80 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abcda4a300 con 0x55abccdfdb00 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55abcda4a480 con 0x55abccdfd680 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13ac71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 0x55abcda22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13ac71640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 0x55abcda22b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41188/0 (socket says 172.21.15.22:41188) 2024-09-21T05:45:24.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13ac71640 1 -- 172.21.15.22:0/1390801139 learned_addr learned my addr 172.21.15.22:0/1390801139 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff13a470640 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfd680 0x55abcda22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.874+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfdb00 0x55abcda22000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13ac71640 1 -- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfd680 msgr2=0x55abcda22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfd680 0x55abcda22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13ac71640 1 -- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfdb00 msgr2=0x55abcda22000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfdb00 0x55abcda22000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13ac71640 1 -- 172.21.15.22:0/1390801139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc8c51680 con 0x55abccdfdf80 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 0x55abcda22b00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x55abcdb1e0f0 tx=0x55abccd5b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380c17d763b2e795 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff137c6b640 1 -- 172.21.15.22:0/1390801139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55abc8c4aea0 con 0x55abccdfdf80 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff137c6b640 1 -- 172.21.15.22:0/1390801139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55abcccaa8c0 con 0x55abccdfdf80 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff137c6b640 1 -- 172.21.15.22:0/1390801139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55abc8c4b380 con 0x55abccdfdf80 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc8c514a0 con 0x55abccdfdf80 2024-09-21T05:45:24.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55abc8c51a40 con 0x55abccdfdf80 2024-09-21T05:45:24.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13bcc9740 4 waiting for MDS map... 2024-09-21T05:45:24.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff137c6b640 1 -- 172.21.15.22:0/1390801139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x55abcdb38600 con 0x55abccdfdf80 2024-09-21T05:45:24.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff137c6b640 1 -- 172.21.15.22:0/1390801139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 70) ==== 2893+0+0 (secure 0 0 0) 0x55abcccaf180 con 0x55abccdfdf80 2024-09-21T05:45:24.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.875+0000 7ff13bcc9740 4 Got MDS map 70 2024-09-21T05:45:24.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.876+0000 7ff13bcc9740 4 datascan.main: Using metadata pool output 2024-09-21T05:45:24.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.876+0000 7ff13bcc9740 4 datascan.main: connecting to RADOS... 2024-09-21T05:45:24.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.876+0000 7ff13bcc9740 1 Processor -- start 2024-09-21T05:45:24.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.877+0000 7ff13bcc9740 1 -- start start 2024-09-21T05:45:24.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfcd80 0x55abcda23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfc900 0x55abcda23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55abcda4a600 con 0x55abccdfe880 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abcda4a780 con 0x55abccdfcd80 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55abcda4a900 con 0x55abccdfc900 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41202/0 (socket says 172.21.15.22:41202) 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 -- 172.21.15.22:0/2501474182 learned_addr learned my addr 172.21.15.22:0/2501474182 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff13a470640 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfc900 0x55abcda23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfcd80 0x55abcda23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 -- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfc900 msgr2=0x55abcda23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfc900 0x55abcda23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 -- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfcd80 msgr2=0x55abcda23080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfcd80 0x55abcda23080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 -- 172.21.15.22:0/2501474182 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc8c51860 con 0x55abccdfe880 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff139c6f640 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23b80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x55abcdb1f5c0 tx=0x55abcda3ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27abf85c77cdcc3b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff135c67640 1 -- 172.21.15.22:0/2501474182 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55abc8c4aea0 con 0x55abccdfe880 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff135c67640 1 -- 172.21.15.22:0/2501474182 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55abcccaae00 con 0x55abccdfe880 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.878+0000 7ff135c67640 1 -- 172.21.15.22:0/2501474182 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55abc8c4b380 con 0x55abccdfe880 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 -- 172.21.15.22:0/2501474182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 msgr2=0x55abcda23b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23b80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x55abcdb1f5c0 tx=0x55abcda3ecc0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 -- 172.21.15.22:0/2501474182 shutdown_connections 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23b80 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfc900 0x55abcda23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/2501474182 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfcd80 0x55abcda23080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 -- 172.21.15.22:0/2501474182 >> 172.21.15.22:0/2501474182 conn(0x55abccdfd200 msgr2=0x55abccd77800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 -- 172.21.15.22:0/2501474182 shutdown_connections 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.879+0000 7ff13bcc9740 1 -- 172.21.15.22:0/2501474182 wait complete. 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.880+0000 7ff13bcc9740 1 Processor -- start 2024-09-21T05:45:24.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.880+0000 7ff13bcc9740 1 -- start start 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfcd80 0x55abcda23600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13bcc9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfed00 0x55abcda23b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55abcda4bb00 con 0x55abccdfe880 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55abcda4b980 con 0x55abccdfed00 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13bcc9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55abcda4b800 con 0x55abccdfcd80 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41214/0 (socket says 172.21.15.22:41214) 2024-09-21T05:45:24.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 learned_addr learned my addr 172.21.15.22:0/1447817756 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfcd80 0x55abcda23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfed00 0x55abcda23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfcd80 msgr2=0x55abcda23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfcd80 0x55abcda23600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfed00 msgr2=0x55abcda23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfed00 0x55abcda23b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.881+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc8c51680 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23080 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x55abcdbb6630 tx=0x55abcda3ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff134464640 1 -- 172.21.15.22:0/1447817756 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55abc8c4a340 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff134464640 1 -- 172.21.15.22:0/1447817756 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55abcccaae00 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55abc8c514a0 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff134464640 1 -- 172.21.15.22:0/1447817756 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55abc8c4aea0 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc8c51a40 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.882+0000 7ff13bcc9740 4 datascan.init: resolving metadata pool 12 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.883+0000 7ff134464640 1 -- 172.21.15.22:0/1447817756 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55abcdbacc00 con 0x55abccdfe880 2024-09-21T05:45:24.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff134464640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55abccdfd200 0x55abcda24100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff134464640 1 -- 172.21.15.22:0/1447817756 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x55abcdbaac00 con 0x55abccdfe880 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff13bcc9740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff13bcc9740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff13bcc9740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55abccdff180 0x55abcda24680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3800 con 0x55abccdff180 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55abccdfd200 0x55abcda24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.884+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55abccdff180 0x55abcda24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.885+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55abccdfd200 0x55abcda24100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55abcdb1f710 tx=0x55abcda3ef00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.885+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55abccdff180 0x55abcda24680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.887+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.887+0000 7ff135466640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55abccdffa80 0x55abcd076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.887+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a000 con 0x55abccdffa80 2024-09-21T05:45:24.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.888+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55abccdffa80 0x55abcd076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.888+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55abccdffa80 0x55abcd076000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.888+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.888+0000 7ff135c67640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55abcd080000 0x55abcd076580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.888+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a400 con 0x55abcd080000 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.889+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55abcd080000 0x55abcd076580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.889+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55abcd080000 0x55abcd076580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.889+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.889+0000 7ff135466640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55abcd080480 0x55abcd076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.889+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a800 con 0x55abcd080480 2024-09-21T05:45:24.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.890+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55abcd080480 0x55abcd076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.890+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55abcd080480 0x55abcd076b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.890+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.890+0000 7ff135c67640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55abcd080d80 0x55abcd077600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.890+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b000 con 0x55abcd080d80 2024-09-21T05:45:24.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.890+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55abcd080d80 0x55abcd077600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.891+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55abcd080d80 0x55abcd077600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.893+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.893+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07ac00 con 0x55abcd080d80 2024-09-21T05:45:24.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.893+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== osd_op_reply(6 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.893+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b400 con 0x55abccdff180 2024-09-21T05:45:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.894+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.894+0000 7ff135466640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55abcd080900 0x55abcd077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.894+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35400 con 0x55abcd080900 2024-09-21T05:45:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.894+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55abcd080900 0x55abcd077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.896 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.894+0000 7ff13a470640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55abcd080900 0x55abcd077080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(8 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a800 con 0x55abccdffa80 2024-09-21T05:45:24.899 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb2400 con 0x55abccdffa80 2024-09-21T05:45:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 3 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff135c67640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55abcd081200 0x55abcd077b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.897+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b400 con 0x55abcd081200 2024-09-21T05:45:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.898+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55abcd081200 0x55abcd077b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.900 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.898+0000 7ff139c6f640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55abcd081200 0x55abcd077b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.900+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(11 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd081200 2024-09-21T05:45:24.902 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.900+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3800 con 0x55abccdffa80 2024-09-21T05:45:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.901+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 4 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.901+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35000 con 0x55abcd080000 2024-09-21T05:45:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.901+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== osd_op_reply(13 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.901+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35400 con 0x55abcd080d80 2024-09-21T05:45:24.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.901+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 3 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.904 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.901+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a400 con 0x55abccdffa80 2024-09-21T05:45:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.902+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 5 ==== osd_op_reply(15 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.902+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3400 con 0x55abcd080d80 2024-09-21T05:45:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.902+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.902+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a000 con 0x55abcd080900 2024-09-21T05:45:24.905 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.902+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.902+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3000 con 0x55abcd080000 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.903+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 3 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.903+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b000 con 0x55abcd081200 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.903+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd081200 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.903+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb2c00 con 0x55abcd080d80 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.903+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 5 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.903+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35000 con 0x55abcd080d80 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.904+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 6 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.904+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a800 con 0x55abcd080000 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.904+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 4 ==== osd_op_reply(22 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.904+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a400 con 0x55abcd080000 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.904+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 5 ==== osd_op_reply(23 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.904+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb2400 con 0x55abcd080900 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(24 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07ac00 con 0x55abcd080480 2024-09-21T05:45:24.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3800 con 0x55abccdff180 2024-09-21T05:45:24.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff135c67640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55abcd081680 0x55abcd078100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.905+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b800 con 0x55abcd081680 2024-09-21T05:45:24.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.906+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55abcd081680 0x55abcd078100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.906+0000 7ff13ac71640 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55abcd081680 0x55abcd078100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.908+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(27 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd081680 2024-09-21T05:45:24.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.908+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3400 con 0x55abccdffa80 2024-09-21T05:45:24.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.908+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 6 ==== osd_op_reply(28 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.908+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07bc00 con 0x55abccdff180 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.909+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 4 ==== osd_op_reply(29 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.909+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35400 con 0x55abcd080d80 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.909+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 7 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.909+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b400 con 0x55abcd080480 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.909+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(31 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.909+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0000 con 0x55abcd080000 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.910+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 6 ==== osd_op_reply(32 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.910+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b000 con 0x55abccdffa80 2024-09-21T05:45:24.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.910+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 7 ==== osd_op_reply(33 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.910+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0400 con 0x55abcd080480 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.911+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(34 [pgnls start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.911+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07ac00 con 0x55abcd081680 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.911+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== osd_op_reply(35 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd081680 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.911+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0800 con 0x55abcd081680 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.911+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.3 v2:172.21.15.22:6808/3633643032 3 ==== osd_op_reply(36 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd081680 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.911+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07bc00 con 0x55abcd080900 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.912+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 4 ==== osd_op_reply(37 [pgnls start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.912+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0c00 con 0x55abccdffa80 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.912+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 8 ==== osd_op_reply(38 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.912+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3800 con 0x55abcd081680 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.912+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.3 v2:172.21.15.22:6808/3633643032 4 ==== osd_op_reply(39 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd081680 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.912+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a1000 con 0x55abcd080d80 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.913+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 8 ==== osd_op_reply(40 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.913+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a000 con 0x55abcd080480 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.913+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 5 ==== osd_op_reply(41 [pgnls start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.913+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a1400 con 0x55abcd080480 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.913+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 6 ==== osd_op_reply(42 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.913+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35000 con 0x55abcd080480 2024-09-21T05:45:24.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 7 ==== osd_op_reply(43 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0000 con 0x55abcd080000 2024-09-21T05:45:24.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 7 ==== osd_op_reply(44 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35400 con 0x55abcd080480 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 8 ==== osd_op_reply(45 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a800 con 0x55abcd081200 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.914+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(46 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd081200 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.915+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3000 con 0x55abccdff180 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.915+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 5 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.915+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a400 con 0x55abccdff180 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.915+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 6 ==== osd_op_reply(48 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.915+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb2400 con 0x55abcd081200 2024-09-21T05:45:24.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd081200 2024-09-21T05:45:24.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07bc00 con 0x55abcd080000 2024-09-21T05:45:24.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 8 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb2c00 con 0x55abccdff180 2024-09-21T05:45:24.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 7 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b800 con 0x55abccdff180 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 8 ==== osd_op_reply(52 [pgnls start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.916+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3400 con 0x55abcd080900 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.917+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 5 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.917+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07b000 con 0x55abcd080d80 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.917+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.2 v2:172.21.15.22:6816/3460873616 9 ==== osd_op_reply(54 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abcd080d80 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.917+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0400 con 0x55abcd080480 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.918+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 9 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.918+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07ac00 con 0x55abcd080480 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.918+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 10 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.918+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a1400 con 0x55abccdff180 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.918+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 9 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v82'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.918+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3c00 con 0x55abccdff180 2024-09-21T05:45:24.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.919+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.919+0000 7ff13bcc9740 10 datascan.operator(): handling object 1099511627777.0 2024-09-21T05:45:24.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.919+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb3000 con 0x55abccdff180 2024-09-21T05:45:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.920+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.7 v2:172.21.15.134:6818/1266427391 11 ==== osd_op_reply(59 10000000001.00000000 [call] v86'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55abccd54000 con 0x55abccdff180 2024-09-21T05:45:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.920+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdbb2400 con 0x55abcd080000 2024-09-21T05:45:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.920+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.1 v2:172.21.15.22:6824/48261475 9 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080000 2024-09-21T05:45:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.920+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a1800 con 0x55abcd080900 2024-09-21T05:45:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.921+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 6 ==== osd_op_reply(61 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.921+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a0400 con 0x55abcd080900 2024-09-21T05:45:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.921+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 <== osd.5 v2:172.21.15.134:6802/1222371224 7 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54780 con 0x55abcd080900 2024-09-21T05:45:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.921+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd0a1c00 con 0x55abccdffa80 2024-09-21T05:45:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.922+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 9 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.922+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a800 con 0x55abcd080480 2024-09-21T05:45:24.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.922+0000 7ff139c6f640 1 -- 172.21.15.22:0/1447817756 <== osd.4 v2:172.21.15.134:6826/3950009634 11 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54000 con 0x55abcd080480 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.922+0000 7ff135c67640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcdb35000 con 0x55abccdffa80 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 10 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff135466640 1 -- 172.21.15.22:0/1447817756 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55abcd07a400 con 0x55abccdffa80 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13ac71640 1 -- 172.21.15.22:0/1447817756 <== osd.6 v2:172.21.15.134:6810/640262620 11 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55abccd54500 con 0x55abccdffa80 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55abcd081200 msgr2=0x55abcd077b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55abcd081200 0x55abcd077b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55abcd080000 msgr2=0x55abcd076580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55abcd080000 0x55abcd076580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55abcd080d80 msgr2=0x55abcd077600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55abcd080d80 0x55abcd077600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55abcd081680 msgr2=0x55abcd078100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55abcd081680 0x55abcd078100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55abcd080480 msgr2=0x55abcd076b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55abcd080480 0x55abcd076b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55abcd080900 msgr2=0x55abcd077080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55abcd080900 0x55abcd077080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55abccdffa80 msgr2=0x55abcd076000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55abccdffa80 0x55abcd076000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55abccdff180 msgr2=0x55abcda24680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55abccdff180 0x55abcda24680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 reap_dead start 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 reap_dead start 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13a470640 1 -- 172.21.15.22:0/1447817756 reap_dead start 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55abccdfd200 msgr2=0x55abcda24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55abccdfd200 0x55abcda24100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55abcdb1f710 tx=0x55abcda3ef00 comp rx=0 tx=0).stop 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 msgr2=0x55abcda23080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23080 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x55abcdbb6630 tx=0x55abcda3ef60 comp rx=0 tx=0).stop 2024-09-21T05:45:24.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 shutdown_connections 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfe880 0x55abcda23080 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1447817756 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55abccdfd200 0x55abcda24100 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 >> 172.21.15.22:0/1447817756 conn(0x55abccdfc900 msgr2=0x55abccd77800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 shutdown_connections 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.923+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1447817756 wait complete. 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 msgr2=0x55abcda22b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 0x55abcda22b00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x55abcdb1e0f0 tx=0x55abccd5b320 comp rx=0 tx=0).stop 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 shutdown_connections 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55abccdfdf80 0x55abcda22b00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55abccdfdb00 0x55abcda22000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 --2- 172.21.15.22:0/1390801139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55abccdfd680 0x55abcda22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 >> 172.21.15.22:0/1390801139 conn(0x55abccdfe400 msgr2=0x55abccd77000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 shutdown_connections 2024-09-21T05:45:24.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.924+0000 7ff13bcc9740 1 -- 172.21.15.22:0/1390801139 wait complete. 2024-09-21T05:45:24.932 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.111586, result: 2024-09-21T05:45:24.932 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-09-21T05:45:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.983+0000 7ff7f322c740 1 -- start start 2024-09-21T05:45:24.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.983+0000 7ff7f322c740 1 Processor -- start 2024-09-21T05:45:24.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441b680 0x55b455054000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 0x55b455054580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441bf80 0x55b455054b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f322c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b455068600 con 0x55b45441bb00 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f322c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b455068480 con 0x55b45441b680 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f322c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b455068300 con 0x55b45441bf80 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f0284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 0x55b455054580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f0284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 0x55b455054580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41222/0 (socket says 172.21.15.22:41222) 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f0284640 1 -- 172.21.15.22:0/2021675321 learned_addr learned my addr 172.21.15.22:0/2021675321 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441b680 0x55b455054000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.984+0000 7ff7f1286640 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441bf80 0x55b455054b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0284640 1 -- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441bf80 msgr2=0x55b455054b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0284640 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441bf80 0x55b455054b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0284640 1 -- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441b680 msgr2=0x55b455054000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0284640 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441b680 0x55b455054000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0284640 1 -- 172.21.15.22:0/2021675321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45026f680 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441b680 0x55b455054000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f0284640 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 0x55b455054580 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x55b455152120 tx=0x55b454360600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a613895380ee242e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7ee280640 1 -- 172.21.15.22:0/2021675321 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b450268ea0 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7ee280640 1 -- 172.21.15.22:0/2021675321 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b4542c8e00 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7ee280640 1 -- 172.21.15.22:0/2021675321 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b450268340 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b45026f4a0 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b45026f860 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f322c740 4 waiting for MDS map... 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7ee280640 1 -- 172.21.15.22:0/2021675321 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x55b45516e400 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7ee280640 1 -- 172.21.15.22:0/2021675321 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 70) ==== 2893+0+0 (secure 0 0 0) 0x55b4542cd180 con 0x55b45441bb00 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.985+0000 7ff7f322c740 4 Got MDS map 70 2024-09-21T05:45:24.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.986+0000 7ff7f322c740 4 datascan.main: Using metadata pool output 2024-09-21T05:45:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.986+0000 7ff7f322c740 4 datascan.main: connecting to RADOS... 2024-09-21T05:45:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.986+0000 7ff7f322c740 1 Processor -- start 2024-09-21T05:45:24.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.986+0000 7ff7f322c740 1 -- start start 2024-09-21T05:45:24.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f322c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b455068180 con 0x55b45441c880 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f322c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b4542d5380 con 0x55b45441a900 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f322c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b4542d5800 con 0x55b45441ad80 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41230/0 (socket says 172.21.15.22:41230) 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 -- 172.21.15.22:0/2069900200 learned_addr learned my addr 172.21.15.22:0/2069900200 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f1286640 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 -- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441ad80 msgr2=0x55b455055080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 -- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 msgr2=0x55b455055600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 -- 172.21.15.22:0/2069900200 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45026f0e0 con 0x55b45441c880 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7f0284640 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 0x55b455055b80 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55b4551529f0 tx=0x55b454360a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a1fa2831d82e0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7ec27c640 1 -- 172.21.15.22:0/2069900200 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b450268ea0 con 0x55b45441c880 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7ec27c640 1 -- 172.21.15.22:0/2069900200 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b4542c8c40 con 0x55b45441c880 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.988+0000 7ff7ec27c640 1 -- 172.21.15.22:0/2069900200 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b450268340 con 0x55b45441c880 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 -- 172.21.15.22:0/2069900200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 msgr2=0x55b455055b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 0x55b455055b80 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x55b4551529f0 tx=0x55b454360a20 comp rx=0 tx=0).stop 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 -- 172.21.15.22:0/2069900200 shutdown_connections 2024-09-21T05:45:24.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2069900200 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 -- 172.21.15.22:0/2069900200 >> 172.21.15.22:0/2069900200 conn(0x55b45441b200 msgr2=0x55b454395800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 -- 172.21.15.22:0/2069900200 shutdown_connections 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.989+0000 7ff7f322c740 1 -- 172.21.15.22:0/2069900200 wait complete. 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.990+0000 7ff7f322c740 1 Processor -- start 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.990+0000 7ff7f322c740 1 -- start start 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f322c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f322c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b455069380 con 0x55b45441ad80 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f322c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b455069500 con 0x55b45441a900 2024-09-21T05:45:24.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f322c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b455068180 con 0x55b45441c880 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41242/0 (socket says 172.21.15.22:41242) 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 learned_addr learned my addr 172.21.15.22:0/45855382 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441c880 msgr2=0x55b455055b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441c880 0x55b455055b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 msgr2=0x55b455055600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441a900 0x55b455055600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.991+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45026fa40 con 0x55b45441ad80 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 0x55b455055080 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x55b4551ce6f0 tx=0x55b45517aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7eaa79640 1 -- 172.21.15.22:0/45855382 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b450268ea0 con 0x55b45441ad80 2024-09-21T05:45:24.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7eaa79640 1 -- 172.21.15.22:0/45855382 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b4542c88c0 con 0x55b45441ad80 2024-09-21T05:45:24.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b45026f680 con 0x55b45441ad80 2024-09-21T05:45:24.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7eaa79640 1 -- 172.21.15.22:0/45855382 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b450268340 con 0x55b45441ad80 2024-09-21T05:45:24.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b45026f4a0 con 0x55b45441ad80 2024-09-21T05:45:24.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.992+0000 7ff7f322c740 4 datascan.init: resolving metadata pool 12 2024-09-21T05:45:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.993+0000 7ff7eaa79640 1 -- 172.21.15.22:0/45855382 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55b455066c00 con 0x55b45441ad80 2024-09-21T05:45:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7eaa79640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b45441da80 0x55b455056100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7eaa79640 1 -- 172.21.15.22:0/45855382 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x55b4551daa00 con 0x55b45441ad80 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f322c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f322c740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f322c740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55b45441cd00 0x55b455056680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9400 con 0x55b45441cd00 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b45441da80 0x55b455056100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.994+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55b45441cd00 0x55b455056680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.995+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b45441da80 0x55b455056100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55b4543670e0 tx=0x55b454646480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.995+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55b45441cd00 0x55b455056680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.995+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.995+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55b45441d180 0x55b455056c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.995+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9800 con 0x55b45441d180 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.995+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55b45441d180 0x55b455056c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.996+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55b45441d180 0x55b455056c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.996+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.996+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55b45441d600 0x55b455057180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.996+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45441d600 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.996+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55b45441d600 0x55b455057180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.997+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55b45441d600 0x55b455057180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.997+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.997+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694000 con 0x55b45441d180 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.997+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.998+0000 7ff7ec27c640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55b45469c000 0x55b4546a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.998+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694400 con 0x55b45469c000 2024-09-21T05:45:25.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.998+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55b45469c000 0x55b4546a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.998+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55b45469c000 0x55b4546a4000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.999+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.999+0000 7ff7eba7b640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55b45469c480 0x55b4546a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.999+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694800 con 0x55b45469c480 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.999+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55b45469c480 0x55b4546a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.999+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55b45469c480 0x55b4546a4580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:24.999+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.000+0000 7ff7ec27c640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b45469c900 0x55b4546a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.000+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695000 con 0x55b45469c900 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.000+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b45469c900 0x55b4546a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.000+0000 7ff7f0a85640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b45469c900 0x55b4546a4b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.000+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.000+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694c00 con 0x55b45469c900 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.001+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.001+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695400 con 0x55b45441d600 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.001+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.001+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9000 con 0x55b45441cd00 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.001+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.001+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9400 con 0x55b45441d180 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d8c00 con 0x55b45441d180 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7ec27c640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b45469cd80 0x55b4546a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b400 con 0x55b45469cd80 2024-09-21T05:45:25.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b45469cd80 0x55b4546a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.002+0000 7ff7f0284640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b45469cd80 0x55b4546a5080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.003+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.003+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45441d180 2024-09-21T05:45:25.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.003+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.003+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b800 con 0x55b45469c000 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.003+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.003+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9800 con 0x55b45469c900 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694000 con 0x55b45441d180 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9400 con 0x55b45469c900 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694800 con 0x55b45441cd00 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.004+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d8c00 con 0x55b45469c000 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694400 con 0x55b45469cd80 2024-09-21T05:45:25.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9000 con 0x55b45469c900 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.005+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695000 con 0x55b45469c900 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.006+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.006+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694c00 con 0x55b45469c000 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.006+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.006+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694800 con 0x55b45469c000 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.006+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.006+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695800 con 0x55b45441cd00 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694400 con 0x55b45469c480 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695c00 con 0x55b45441d600 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.017 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7ec27c640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55b45469d200 0x55b4546a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d8c00 con 0x55b45469d200 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.007+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55b45469d200 0x55b4546a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.008+0000 7ff7f1286640 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55b45469d200 0x55b4546a5600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.008+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.008+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695400 con 0x55b45441d180 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.008+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.008+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b400 con 0x55b45441d600 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.009+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.009+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695000 con 0x55b45469c900 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.009+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.009+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b800 con 0x55b45469c480 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.010+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.010+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695c00 con 0x55b45469c000 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.010+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.010+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694000 con 0x55b45441d180 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.010+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.010+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9800 con 0x55b45469c480 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694400 con 0x55b45469d200 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9400 con 0x55b45469d200 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695400 con 0x55b45441cd00 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.011+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9000 con 0x55b45441d180 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.012+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.012+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b800 con 0x55b45469d200 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.012+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 4 ==== osd_op_reply(41 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.012+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694c00 con 0x55b45469c900 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.012+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.012+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45469c480 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.013+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.013+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9800 con 0x55b45469c480 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.013+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.013+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d8c00 con 0x55b45469c480 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.013+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9400 con 0x55b45469c000 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0000 con 0x55b45469c480 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9000 con 0x55b45469cd80 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.014+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0400 con 0x55b45441d600 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.015+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.015+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45441d600 2024-09-21T05:45:25.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.015+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.015+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0800 con 0x55b45469cd80 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.015+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.015+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b800 con 0x55b45469c000 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.016+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.016+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0c00 con 0x55b45441d600 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.016+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.016+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694000 con 0x55b45441d600 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.016+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.016+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1000 con 0x55b45441cd00 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.017+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.017+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694400 con 0x55b45469c900 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.017+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.017+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1400 con 0x55b45469c480 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.018+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.018+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45469c480 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.018+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.018+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1800 con 0x55b45441d600 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.019+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 86 out=81b] v86'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.019+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0400 con 0x55b45441d600 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.019+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.019+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0800 con 0x55b45441d180 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.019+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695000 con 0x55b45469c000 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1c00 con 0x55b45469c480 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694c00 con 0x55b45469c900 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.020+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2000 con 0x55b45469c900 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.021+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.021+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694000 con 0x55b45441d600 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.021+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.021+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2400 con 0x55b45441cd00 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.021+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.021+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694800 con 0x55b45441d180 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.022+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.022+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2800 con 0x55b45441d180 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.022+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.022+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695800 con 0x55b45469cd80 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.023+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.023+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2c00 con 0x55b45441d180 2024-09-21T05:45:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.023+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.029 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.023+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695c00 con 0x55b45469c000 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.023+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.023+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3000 con 0x55b45469c900 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.024+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.024+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695400 con 0x55b45441d180 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.024+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.024+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3400 con 0x55b45469c900 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.024+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.024+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b400 con 0x55b45441cd00 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.025+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.025+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3800 con 0x55b45469c000 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.025+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.025+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9800 con 0x55b45469cd80 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.025+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.025+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3c00 con 0x55b45469c900 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d8c00 con 0x55b45469c900 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695000 con 0x55b45469c000 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45469c000 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.026+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695800 con 0x55b45441cd00 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.027+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.027+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2400 con 0x55b45469c480 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.027+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.027+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695c00 con 0x55b45441d600 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.027+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.028+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3800 con 0x55b45469d200 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.028+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 5 ==== osd_op_reply(86 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.028+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1000 con 0x55b45441d180 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.028+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.028+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2000 con 0x55b45441d600 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.029+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.029+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1800 con 0x55b45469c900 2024-09-21T05:45:25.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.029+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.029+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3000 con 0x55b45469c480 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.029+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.029+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0400 con 0x55b45469c000 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.030+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 14 ==== osd_op_reply(91 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.030+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3400 con 0x55b45441d180 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.030+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.030+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0000 con 0x55b45469c480 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.030+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 86 out=49b] v82'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.030+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b800 con 0x55b45469d200 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.031+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 6 ==== osd_op_reply(94 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.031+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1400 con 0x55b45469d200 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.031+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.031+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b45516b400 con 0x55b45441cd00 2024-09-21T05:45:25.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.031+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.031+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0800 con 0x55b45441d180 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.032+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.032+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454695400 con 0x55b45469d200 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.032+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.3 v2:172.21.15.22:6808/3633643032 8 ==== osd_op_reply(98 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469d200 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.032+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1c00 con 0x55b45469c900 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.032+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.032+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694400 con 0x55b45469c480 2024-09-21T05:45:25.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 86 out=49b] v82'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694000 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694c00 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b454694800 con 0x55b45469c000 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.033+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1000 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.034+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.034+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cc000 con 0x55b45469cd80 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.034+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.034+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0000 con 0x55b45441d600 2024-09-21T05:45:25.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.034+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.034+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cc400 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.035+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.035+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2800 con 0x55b45469cd80 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.035+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.035+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cc800 con 0x55b45469c000 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.035+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 86 out=49b] v82'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.035+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2c00 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 86 out=49b] v82'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546ccc00 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 86 out=49b] v82'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2400 con 0x55b45441cd00 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cd000 con 0x55b45469c900 2024-09-21T05:45:25.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.2 v2:172.21.15.22:6816/3460873616 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372780 con 0x55b45469c900 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.036+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3800 con 0x55b45469c480 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.037+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.037+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cd400 con 0x55b45469c480 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.037+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 86 out=49b] v82'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.037+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3000 con 0x55b45441d600 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 86 out=81b] v86'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f322c740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f322c740 10 datascan.operator(): handling object 10000000001.0 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9400 con 0x55b45441d600 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 <== osd.7 v2:172.21.15.134:6818/1266427391 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x55b454372780 con 0x55b45441d600 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f322c740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f322c740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.038+0000 7ff7f322c740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-09-21T05:45:25.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9000 con 0x55b45469cd80 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c1400 con 0x55b45441d180 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 20 datascan.get_frag_of: frag is * 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c0800 con 0x55b45469cd80 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.039+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d8c00 con 0x55b45469cd80 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-21T05:45:25.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 20 datascan.read_dentry: file_head not found in result 2024-09-21T05:45:25.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-09-21T05:45:25.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.040+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9800 con 0x55b45469cd80 2024-09-21T05:45:25.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.041+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.0 v2:172.21.15.22:6800/3448237758 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v86'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55b454372000 con 0x55b45469cd80 2024-09-21T05:45:25.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.041+0000 7ff7f322c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-21T05:45:25.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.041+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4551d9c00 con 0x55b45469c000 2024-09-21T05:45:25.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.041+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.1 v2:172.21.15.22:6824/48261475 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 86 out=49b] v82'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45469c000 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3400 con 0x55b45441cd00 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cd800 con 0x55b45441cd00 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7f0284640 1 -- 172.21.15.22:0/45855382 <== osd.5 v2:172.21.15.134:6802/1222371224 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 86 out=49b] v82'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372000 con 0x55b45441cd00 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3800 con 0x55b45441d180 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 86 out=49b] v82'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.042+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546cdc00 con 0x55b45469c480 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.043+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.4 v2:172.21.15.134:6826/3950009634 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45469c480 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.043+0000 7ff7eba7b640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c3c00 con 0x55b45441d180 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.043+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 86 out=49b] v82'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.043+0000 7ff7ec27c640 1 -- 172.21.15.22:0/45855382 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55b4546c2000 con 0x55b45441d180 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f1286640 1 -- 172.21.15.22:0/45855382 <== osd.6 v2:172.21.15.134:6810/640262620 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 86 out=49b] v82'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b454372500 con 0x55b45441d180 2024-09-21T05:45:25.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b45469cd80 msgr2=0x55b4546a5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55b45469cd80 0x55b4546a5080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55b45469c000 msgr2=0x55b4546a4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55b45469c000 0x55b4546a4000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b45469c900 msgr2=0x55b4546a4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b45469c900 0x55b4546a4b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55b45469d200 msgr2=0x55b4546a5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55b45469d200 0x55b4546a5600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 reap_dead start 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55b45469c480 msgr2=0x55b4546a4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55b45469c480 0x55b4546a4580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55b45441cd00 msgr2=0x55b455056680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55b45441cd00 0x55b455056680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55b45441d180 msgr2=0x55b455056c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55b45441d180 0x55b455056c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55b45441d600 msgr2=0x55b455057180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55b45441d600 0x55b455057180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f0a85640 1 -- 172.21.15.22:0/45855382 reap_dead start 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b45441da80 msgr2=0x55b455056100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b45441da80 0x55b455056100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55b4543670e0 tx=0x55b454646480 comp rx=0 tx=0).stop 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 msgr2=0x55b455055080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 0x55b455055080 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x55b4551ce6f0 tx=0x55b45517aea0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 shutdown_connections 2024-09-21T05:45:25.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b45441da80 0x55b455056100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 --2- 172.21.15.22:0/45855382 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441ad80 0x55b455055080 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 >> 172.21.15.22:0/45855382 conn(0x55b45441b200 msgr2=0x55b454395800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 shutdown_connections 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/45855382 wait complete. 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.044+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 msgr2=0x55b455054580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 0x55b455054580 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x55b455152120 tx=0x55b454360600 comp rx=0 tx=0).stop 2024-09-21T05:45:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 shutdown_connections 2024-09-21T05:45:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b45441bf80 0x55b455054b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b45441bb00 0x55b455054580 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 --2- 172.21.15.22:0/2021675321 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b45441b680 0x55b455054000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 >> 172.21.15.22:0/2021675321 conn(0x55b45441c400 msgr2=0x55b454393800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 shutdown_connections 2024-09-21T05:45:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.045+0000 7ff7f322c740 1 -- 172.21.15.22:0/2021675321 wait complete. 2024-09-21T05:45:25.055 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.123313, result: 2024-09-21T05:45:25.056 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-09-21T05:45:25.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.104+0000 7f1bf3f22740 1 -- start start 2024-09-21T05:45:25.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.104+0000 7f1bf3f22740 1 Processor -- start 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189181680 0x55f189db6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189181b00 0x55f189db6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 0x55f189db6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f189dce600 con 0x55f189181f80 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f189dce480 con 0x55f189181b00 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55f189dce300 con 0x55f189181680 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf26a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 0x55f189db6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf26a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 0x55f189db6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41248/0 (socket says 172.21.15.22:41248) 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2346403566 learned_addr learned my addr 172.21.15.22:0/2346403566 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189181680 0x55f189db6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.105+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189181b00 0x55f189db6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189181680 msgr2=0x55f189db6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189181680 0x55f189db6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189181b00 msgr2=0x55f189db6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189181b00 0x55f189db6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2346403566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f184fd5680 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 0x55f189db6b00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x55f189ebc0f0 tx=0x55f1890c6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d76cc6deef413bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bef6a2640 1 -- 172.21.15.22:0/2346403566 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f184fceea0 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bef6a2640 1 -- 172.21.15.22:0/2346403566 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f18902ee00 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bef6a2640 1 -- 172.21.15.22:0/2346403566 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f184fcf380 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f184fd5860 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55f184fd52c0 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf3f22740 4 waiting for MDS map... 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bef6a2640 1 -- 172.21.15.22:0/2346403566 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x55f189ed2600 con 0x55f189181f80 2024-09-21T05:45:25.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bef6a2640 1 -- 172.21.15.22:0/2346403566 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 70) ==== 2893+0+0 (secure 0 0 0) 0x55f189033180 con 0x55f189181f80 2024-09-21T05:45:25.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.106+0000 7f1bf3f22740 4 Got MDS map 70 2024-09-21T05:45:25.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.107+0000 7f1bf3f22740 4 datascan.main: Using metadata pool output 2024-09-21T05:45:25.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.107+0000 7f1bf3f22740 4 datascan.main: connecting to RADOS... 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.107+0000 7f1bf3f22740 1 Processor -- start 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.108+0000 7f1bf3f22740 1 -- start start 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f189dce180 con 0x55f189182880 2024-09-21T05:45:25.113 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f18903b380 con 0x55f189180d80 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55f18903b800 con 0x55f189180900 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41260/0 (socket says 172.21.15.22:41260) 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2095502239 learned_addr learned my addr 172.21.15.22:0/2095502239 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 msgr2=0x55f189db7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 msgr2=0x55f189db7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2095502239 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f184fd54a0 con 0x55f189182880 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x55f189ebd5f0 tx=0x55f189ed6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d0e55c91c203f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bed69e640 1 -- 172.21.15.22:0/2095502239 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f184fceea0 con 0x55f189182880 2024-09-21T05:45:25.114 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bed69e640 1 -- 172.21.15.22:0/2095502239 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f18902e8c0 con 0x55f189182880 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.109+0000 7f1bed69e640 1 -- 172.21.15.22:0/2095502239 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f184fcf380 con 0x55f189182880 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2095502239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 msgr2=0x55f189db7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x55f189ebd5f0 tx=0x55f189ed6cc0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2095502239 shutdown_connections 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2095502239 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2095502239 >> 172.21.15.22:0/2095502239 conn(0x55f189181200 msgr2=0x55f1890fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2095502239 shutdown_connections 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.110+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2095502239 wait complete. 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.111+0000 7f1bf3f22740 1 Processor -- start 2024-09-21T05:45:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.111+0000 7f1bf3f22740 1 -- start start 2024-09-21T05:45:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf3f22740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55f189dcf380 con 0x55f189182880 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55f189dce180 con 0x55f189180d80 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf3f22740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55f189dcf500 con 0x55f189180900 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf16a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf16a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41270/0 (socket says 172.21.15.22:41270) 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 learned_addr learned my addr 172.21.15.22:0/2514212161 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.112+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 msgr2=0x55f189db7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189180900 0x55f189db7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 msgr2=0x55f189db7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f184fd5680 con 0x55f189180d80 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189182880 0x55f189db7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55f189e05830 tx=0x55f189ed6f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bebe9b640 1 -- 172.21.15.22:0/2514212161 <== mon.1 v2:172.21.15.134:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f184fce340 con 0x55f189180d80 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bebe9b640 1 -- 172.21.15.22:0/2514212161 <== mon.1 v2:172.21.15.134:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f18902e000 con 0x55f189180d80 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f184fd5860 con 0x55f189180d80 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bebe9b640 1 -- 172.21.15.22:0/2514212161 <== mon.1 v2:172.21.15.134:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f184fceea0 con 0x55f189180d80 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f184fd52c0 con 0x55f189180d80 2024-09-21T05:45:25.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.113+0000 7f1bf3f22740 4 datascan.init: resolving metadata pool 12 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bebe9b640 1 -- 172.21.15.22:0/2514212161 <== mon.1 v2:172.21.15.134:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55f189f46c00 con 0x55f189180d80 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bebe9b640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55f189183a80 0x55f189db8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bebe9b640 1 -- 172.21.15.22:0/2514212161 <== mon.1 v2:172.21.15.134:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x55f189f44a00 con 0x55f189180d80 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bf3f22740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bf3f22740 4 datascan.main: resolving metadata pool 12 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55f189183a80 0x55f189db8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55f189182d00 0x55f189db8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f189182d00 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.115+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55f189182d00 0x55f189db8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.116+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55f189183a80 0x55f189db8100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55f189e05dd0 tx=0x55f189ed6d80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.116+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55f189182d00 0x55f189db8680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.116+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.116+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55f1893ea000 0x55f1893ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.116+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.116+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55f1893ea000 0x55f1893ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.117+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55f1893ea000 0x55f1893ee000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.117+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=177b] v86'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.117+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55f1893ea480 0x55f1893ee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.117+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea480 2024-09-21T05:45:25.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.117+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55f1893ea480 0x55f1893ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.117+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55f1893ea480 0x55f1893ee580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893ea480 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f1893ea900 0x55f1893eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea900 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f1893ea900 0x55f1893eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f1893ea900 0x55f1893eeb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55f1893ead80 0x55f1893ef080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.118+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.119+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55f1893ead80 0x55f1893ef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.119+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55f1893ead80 0x55f1893ef080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.119+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.119+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55f1893eb680 0x55f1893efb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.119+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893eb680 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.120+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55f1893eb680 0x55f1893efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.120+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55f1893eb680 0x55f1893efb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.120+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(6 [pgnls start_epoch 86 out=149b] v85'26 uv26 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893eb680 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.120+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(8 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f189182d00 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea480 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893ea480 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf1ea7640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55f1893ebf80 0x55f1893f0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.121+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ebf80 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.122+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55f1893ebf80 0x55f1893f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.122+0000 7f1bf16a6640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55f1893ebf80 0x55f1893f0680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.122+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(11 [pgnls start_epoch 86 out=81b] v85'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ebf80 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.122+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f189182d00 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.122+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.122+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea900 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.123+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== osd_op_reply(13 [pgnls start_epoch 86 out=77b] v85'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.123+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.123+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.123+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.123+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(15 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.123+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f189182d00 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.124+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.124+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.124+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 3 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.124+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893eb680 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.124+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893eb680 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.124+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ebf80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ebf80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 5 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 4 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 5 ==== osd_op_reply(22 [pgnls start_epoch 86 out=97b] v85'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.125+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55f1893ebb00 0x55f1893f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.126+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ebb00 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.126+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55f1893ebb00 0x55f1893f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.126+0000 7f1bf26a8640 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55f1893ebb00 0x55f1893f0100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.126+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(23 [pgnls start_epoch 86 out=74b] v81'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ebb00 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.127+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.127+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 6 ==== osd_op_reply(24 [pgnls start_epoch 86 out=73b] v85'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.127+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.127+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 7 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.127+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea900 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.128+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=98b] v85'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.128+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea480 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.128+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.3 v2:172.21.15.22:6808/3633643032 3 ==== osd_op_reply(27 [pgnls start_epoch 86 out=76b] v85'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893ea480 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.128+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f189182d00 2024-09-21T05:45:25.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.128+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 5 ==== osd_op_reply(28 [pgnls start_epoch 86 out=73b] v85'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.128+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea900 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 4 ==== osd_op_reply(29 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 6 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 7 ==== osd_op_reply(31 [pgnls start_epoch 86 out=73b] v81'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 8 ==== osd_op_reply(32 [pgnls start_epoch 86 out=101b] v85'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.129+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43c00 con 0x55f1893ea000 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412000 con 0x55f1893ea000 2024-09-21T05:45:25.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412400 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.130+0000 7f1bf3f22740 20 datascan.scan_links: injected ino detected 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412800 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-09-21T05:45:25.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.131+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43c00 con 0x55f1893eb680 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893eb680 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189ecf400 con 0x55f1893ea000 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189ecf000 con 0x55f1893ebf80 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.2 v2:172.21.15.22:6816/3460873616 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ebf80 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-09-21T05:45:25.143 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.132+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412000 con 0x55f1893ea000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412c00 con 0x55f1893ead80 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413000 con 0x55f1893ea000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-21T05:45:25.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.133+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea900 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413800 con 0x55f1893ea900 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.134+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412400 con 0x55f1893ead80 2024-09-21T05:45:25.145 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.135+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.135+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189412800 con 0x55f189182d00 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.135+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 6 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.135+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.135+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 16 ==== osd_op_reply(48 [pgnls start_epoch 86 out=177b] v86'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.135+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea480 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.3 v2:172.21.15.22:6808/3633643032 4 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893ea480 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea900 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 7 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 11 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893eb680 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.5 v2:172.21.15.134:6802/1222371224 4 ==== osd_op_reply(52 [pgnls start_epoch 86 out=149b] v85'26 uv26 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893eb680 2024-09-21T05:45:25.146 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.136+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.137+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 17 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.137+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.137+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 18 ==== osd_op_reply(54 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.137+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f189182d00 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.137+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 7 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.137+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea480 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.3 v2:172.21.15.22:6808/3633643032 5 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893ea480 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ebf80 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.2 v2:172.21.15.22:6816/3460873616 4 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v85'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ebf80 2024-09-21T05:45:25.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f189182d00 2024-09-21T05:45:25.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 8 ==== osd_op_reply(58 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea900 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 8 ==== osd_op_reply(59 [pgnls start_epoch 86 out=77b] v85'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.138+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.139+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 12 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.139+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.139+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 19 ==== osd_op_reply(61 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.139+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f189182d00 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.139+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 9 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v83'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.139+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 13 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893eb680 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.5 v2:172.21.15.134:6802/1222371224 5 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893eb680 2024-09-21T05:45:25.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ebf80 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.2 v2:172.21.15.22:6816/3460873616 5 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ebf80 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 20 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.140+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.141+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 14 ==== osd_op_reply(67 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.141+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.141+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 15 ==== osd_op_reply(68 [pgnls start_epoch 86 out=97b] v85'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.141+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ebb00 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.141+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== osd_op_reply(69 [pgnls start_epoch 86 out=74b] v81'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ebb00 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.141+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 21 ==== osd_op_reply(70 [pgnls start_epoch 86 out=73b] v85'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 22 ==== osd_op_reply(71 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea900 2024-09-21T05:45:25.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 9 ==== osd_op_reply(72 [pgnls start_epoch 86 out=98b] v85'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea480 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.3 v2:172.21.15.22:6808/3633643032 6 ==== osd_op_reply(73 [pgnls start_epoch 86 out=76b] v85'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893ea480 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.142+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f189182d00 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.143+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.4 v2:172.21.15.134:6826/3950009634 10 ==== osd_op_reply(74 [pgnls start_epoch 86 out=73b] v85'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8000 con 0x55f189182d00 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.143+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea900 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.143+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 10 ==== osd_op_reply(75 [pgnls start_epoch 86 out=73b] v85'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.143+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.143+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 16 ==== osd_op_reply(76 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.143+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 17 ==== osd_op_reply(77 [pgnls start_epoch 86 out=73b] v81'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ead80 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 18 ==== osd_op_reply(78 [pgnls start_epoch 86 out=101b] v85'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412000 con 0x55f1893ea000 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.144+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412c00 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189ecf400 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 20 datascan.scan_links: injected ino detected 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413000 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.145+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412400 con 0x55f1893eb680 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 <== osd.5 v2:172.21.15.134:6802/1222371224 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8780 con 0x55f1893eb680 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413400 con 0x55f1893ea000 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-09-21T05:45:25.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412000 con 0x55f1893ebf80 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.2 v2:172.21.15.22:6816/3460873616 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ebf80 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.146+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189412c00 con 0x55f1893ea000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189f43800 con 0x55f1893ead80 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189ecf400 con 0x55f1893ea000 2024-09-21T05:45:25.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.147+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413c00 con 0x55f1893ea900 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189420000 con 0x55f1893ea900 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-09-21T05:45:25.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189420400 con 0x55f1893ead80 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.6 v2:172.21.15.134:6810/640262620 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ead80 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.148+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189420800 con 0x55f1893ea000 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.149+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.149+0000 7f1bf3f22740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.149+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55f189420c00 con 0x55f1893ea000 2024-09-21T05:45:25.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v86'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 10 datascan.scan_links: processing 1 injected_inos 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189421000 con 0x55f1893ea000 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 20 datascan.scan_links: first is now 2 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.150+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55f189421400 con 0x55f1893ea000 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.151+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.0 v2:172.21.15.22:6800/3448237758 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v86'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55f1890d8500 con 0x55f1893ea000 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.151+0000 7f1bf3f22740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.151+0000 7f1bf3f22740 10 datascan.scan_links: updating inotable 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.151+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189421800 con 0x55f1893ea900 2024-09-21T05:45:25.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.152+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.152+0000 7f1bf3f22740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.152+0000 7f1bf3f22740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.152+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) -- 0x55f189421c00 con 0x55f1893ea900 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.153+0000 7f1bf16a6640 1 -- 172.21.15.22:0/2514212161 <== osd.1 v2:172.21.15.22:6824/48261475 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v86'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55f1890d8000 con 0x55f1893ea900 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.153+0000 7f1bf3f22740 10 datascan.scan_links: updating snaptable 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.153+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) -- 0x55f189413000 con 0x55f1893ebb00 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf26a8640 1 -- 172.21.15.22:0/2514212161 <== osd.7 v2:172.21.15.134:6818/1266427391 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55f1890d8280 con 0x55f1893ebb00 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55f1893ea000 msgr2=0x55f1893ee000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x55f1893ea000 0x55f1893ee000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f1893ea900 msgr2=0x55f1893eeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55f1893ea900 0x55f1893eeb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55f1893ebf80 msgr2=0x55f1893f0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55f1893ebf80 0x55f1893f0680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55f1893ea480 msgr2=0x55f1893ee580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55f1893ea480 0x55f1893ee580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 reap_dead start 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55f189182d00 msgr2=0x55f189db8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55f189182d00 0x55f189db8680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 reap_dead start 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55f1893eb680 msgr2=0x55f1893efb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55f1893eb680 0x55f1893efb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55f1893ead80 msgr2=0x55f1893ef080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x55f1893ead80 0x55f1893ef080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55f1893ebb00 msgr2=0x55f1893f0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55f1893ebb00 0x55f1893f0100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55f189183a80 msgr2=0x55f189db8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55f189183a80 0x55f189db8100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55f189e05dd0 tx=0x55f189ed6d80 comp rx=0 tx=0).stop 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 msgr2=0x55f189db7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2514212161 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189180d80 0x55f189db7080 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55f189e05830 tx=0x55f189ed6f60 comp rx=0 tx=0).stop 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf1ea7640 1 -- 172.21.15.22:0/2514212161 reap_dead start 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 shutdown_connections 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 >> 172.21.15.22:0/2514212161 conn(0x55f189181200 msgr2=0x55f1890fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 shutdown_connections 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2514212161 wait complete. 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 msgr2=0x55f189db6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.154+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 0x55f189db6b00 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x55f189ebc0f0 tx=0x55f1890c6600 comp rx=0 tx=0).stop 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 shutdown_connections 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55f189181f80 0x55f189db6b00 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55f189181b00 0x55f189db6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 --2- 172.21.15.22:0/2346403566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55f189181680 0x55f189db6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 >> 172.21.15.22:0/2346403566 conn(0x55f189182400 msgr2=0x55f1890f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 shutdown_connections 2024-09-21T05:45:25.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.155+0000 7f1bf3f22740 1 -- 172.21.15.22:0/2346403566 wait complete. 2024-09-21T05:45:25.170 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.114860, result: 2024-09-21T05:45:25.171 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T05:45:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 -- 172.21.15.22:0/1603438890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c06eee0 msgr2=0x7fcd6c06f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 --2- 172.21.15.22:0/1603438890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c06eee0 0x7fcd6c06f2f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcd60005fd0 tx=0x7fcd6002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 -- 172.21.15.22:0/1603438890 shutdown_connections 2024-09-21T05:45:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 --2- 172.21.15.22:0/1603438890 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6c0d0c20 0x7fcd6c0d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 --2- 172.21.15.22:0/1603438890 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c06eee0 0x7fcd6c06f2f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 -- 172.21.15.22:0/1603438890 >> 172.21.15.22:0/1603438890 conn(0x7fcd6c09d8c0 msgr2=0x7fcd6c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 -- 172.21.15.22:0/1603438890 shutdown_connections 2024-09-21T05:45:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.288+0000 7fcd71f89640 1 -- 172.21.15.22:0/1603438890 wait complete. 2024-09-21T05:45:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.289+0000 7fcd71f89640 1 Processor -- start 2024-09-21T05:45:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.290+0000 7fcd71f89640 1 -- start start 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd71f89640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd6c06eee0 0x7fcd6c0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd71f89640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 0x7fcd6c152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd71f89640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6c0d0c20 0x7fcd6c15f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd71f89640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c0b3d80 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd71f89640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6c0b3c00 con 0x7fcd6c06eee0 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd71f89640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6c0d1570 con 0x7fcd6c0d0c20 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 0x7fcd6c152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 0x7fcd6c152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41282/0 (socket says 172.21.15.22:41282) 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 -- 172.21.15.22:0/736353084 learned_addr learned my addr 172.21.15.22:0/736353084 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6bfff640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6c0d0c20 0x7fcd6c15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6b7fe640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd6c06eee0 0x7fcd6c0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 -- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6c0d0c20 msgr2=0x7fcd6c15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6c0d0c20 0x7fcd6c15f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 -- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd6c06eee0 msgr2=0x7fcd6c0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd6c06eee0 0x7fcd6c0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.291+0000 7fcd6affd640 1 -- 172.21.15.22:0/736353084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd6c0b4130 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.292+0000 7fcd6affd640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 0x7fcd6c152af0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcd60034040 tx=0x7fcd600035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.292+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6003e030 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.292+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd60003ae0 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.292+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd6c0b4410 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.292+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd60035040 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.292+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd6c0b49d0 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.293+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6c0cc560 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.294+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcd60041030 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.294+0000 7fcd53fff640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd58078b90 0x7fcd5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.294+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fcd600bb650 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.294+0000 7fcd6b7fe640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd58078b90 0x7fcd5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.295+0000 7fcd6b7fe640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd58078b90 0x7fcd5807b0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c007fd0 tx=0x7fcd5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.299+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd60083370 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.527+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fcd6c06f7a0 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.804+0000 7fcd53fff640 1 -- 172.21.15.22:0/736353084 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) ==== 162+0+0 (secure 0 0 0) 0x7fcd6c06f7a0 con 0x7fcd6c0b96c0 2024-09-21T05:45:25.807 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.807+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd58078b90 msgr2=0x7fcd5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.807+0000 7fcd71f89640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd58078b90 0x7fcd5807b0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c007fd0 tx=0x7fcd5c009040 comp rx=0 tx=0).stop 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.807+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 msgr2=0x7fcd6c152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.807+0000 7fcd71f89640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 0x7fcd6c152af0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcd60034040 tx=0x7fcd600035f0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 shutdown_connections 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcd58078b90 0x7fcd5807b0a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcd6c0d0c20 0x7fcd6c15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcd6c0b96c0 0x7fcd6c152af0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 --2- 172.21.15.22:0/736353084 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcd6c06eee0 0x7fcd6c0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 >> 172.21.15.22:0/736353084 conn(0x7fcd6c09d8c0 msgr2=0x7fcd6c161bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 shutdown_connections 2024-09-21T05:45:25.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.808+0000 7fcd71f89640 1 -- 172.21.15.22:0/736353084 wait complete. 2024-09-21T05:45:25.824 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:45:25.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f09257da640 1 -- 172.21.15.22:0/3215177736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f090c004000 msgr2=0x7f090c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f09257da640 1 --2- 172.21.15.22:0/3215177736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f090c004000 0x7f090c004800 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f09200ba1b0 tx=0x7f090c005830 comp rx=0 tx=0).stop 2024-09-21T05:45:25.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f091f7fe640 1 -- 172.21.15.22:0/3215177736 reap_dead start 2024-09-21T05:45:25.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f09257da640 1 -- 172.21.15.22:0/3215177736 shutdown_connections 2024-09-21T05:45:25.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f09257da640 1 -- 172.21.15.22:0/3215177736 >> 172.21.15.22:0/3215177736 conn(0x7f092008bfc0 msgr2=0x7f092009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:25.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f09257da640 1 -- 172.21.15.22:0/3215177736 shutdown_connections 2024-09-21T05:45:25.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.941+0000 7f09257da640 1 -- 172.21.15.22:0/3215177736 wait complete. 2024-09-21T05:45:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.943+0000 7f09257da640 1 Processor -- start 2024-09-21T05:45:25.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.943+0000 7f09257da640 1 -- start start 2024-09-21T05:45:25.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f09257da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f090c004000 0x7f09200c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f09257da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09200b8340 0x7f092015eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f09257da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 0x7f09201f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f09257da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09200cf0d0 con 0x7f09201f56a0 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f09257da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09200cef50 con 0x7f09200b8340 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f09257da640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09200cf250 con 0x7f090c004000 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f091f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 0x7f09201f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f091f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 0x7f09201f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41288/0 (socket says 172.21.15.22:41288) 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.944+0000 7f091f7fe640 1 -- 172.21.15.22:0/1727920936 learned_addr learned my addr 172.21.15.22:0/1727920936 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091e7fc640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09200b8340 0x7f092015eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091effd640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f090c004000 0x7f09200c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091f7fe640 1 -- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f090c004000 msgr2=0x7f09200c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091f7fe640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f090c004000 0x7f09200c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091f7fe640 1 -- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09200b8340 msgr2=0x7f092015eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091f7fe640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09200b8340 0x7f092015eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:25.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091f7fe640 1 -- 172.21.15.22:0/1727920936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09201eb070 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f091f7fe640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 0x7f09201f5ab0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0910005fd0 tx=0x7f0910011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091000fcb0 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f091000fe50 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09201f9330 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910004d60 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.945+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09201f5180 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.946+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092006e170 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.947+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0910020020 con 0x7f09201f56a0 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.948+0000 7f09177fe640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0900078b90 0x7f090007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:25.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.948+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f091009b1e0 con 0x7f09201f56a0 2024-09-21T05:45:25.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.948+0000 7f091effd640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0900078b90 0x7f090007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:25.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.948+0000 7f091effd640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0900078b90 0x7f090007b0a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0918009cd0 tx=0x7f0918005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:25.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:25.952+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0910063540 con 0x7f09201f56a0 2024-09-21T05:45:26.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.181+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09200b8ab0 con 0x7f09201f56a0 2024-09-21T05:45:26.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.181+0000 7f09177fe640 1 -- 172.21.15.22:0/1727920936 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5213 (secure 0 0 0) 0x7f0910062e40 con 0x7f09201f56a0 2024-09-21T05:45:26.183 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:45:26.184 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":72,"btime":"2024-09-21T05:45:25:809326+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:25.809301+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T05:45:26.184 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 72 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0900078b90 msgr2=0x7f090007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0900078b90 0x7f090007b0a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0918009cd0 tx=0x7f0918005f90 comp rx=0 tx=0).stop 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 msgr2=0x7f09201f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 0x7f09201f5ab0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0910005fd0 tx=0x7f0910011ed0 comp rx=0 tx=0).stop 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 shutdown_connections 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09201f56a0 0x7f09201f5ab0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09200b8340 0x7f092015eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0900078b90 0x7f090007b0a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:26.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 --2- 172.21.15.22:0/1727920936 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f090c004000 0x7f09200c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:26.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 >> 172.21.15.22:0/1727920936 conn(0x7f092008bfc0 msgr2=0x7f09200b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:26.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 shutdown_connections 2024-09-21T05:45:26.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:26.183+0000 7f09257da640 1 -- 172.21.15.22:0/1727920936 wait complete. 2024-09-21T05:45:26.599 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:45:25.809301+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.22:6834/43392257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 43392257}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 43392257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:45:26.599 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7167:up:replay 2024-09-21T05:45:27.600 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:45:27.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 -- 172.21.15.22:0/3360364778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 msgr2=0x7ff1600cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:27.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 --2- 172.21.15.22:0/3360364778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1600cf540 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff158009be0 tx=0x7ff15802a840 comp rx=0 tx=0).stop 2024-09-21T05:45:27.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 -- 172.21.15.22:0/3360364778 shutdown_connections 2024-09-21T05:45:27.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 --2- 172.21.15.22:0/3360364778 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1600cf540 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 -- 172.21.15.22:0/3360364778 >> 172.21.15.22:0/3360364778 conn(0x7ff16009d7f0 msgr2=0x7ff1600d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 -- 172.21.15.22:0/3360364778 shutdown_connections 2024-09-21T05:45:27.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.730+0000 7ff1685ad640 1 -- 172.21.15.22:0/3360364778 wait complete. 2024-09-21T05:45:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.732+0000 7ff1685ad640 1 Processor -- start 2024-09-21T05:45:27.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.732+0000 7ff1685ad640 1 -- start start 2024-09-21T05:45:27.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff1685ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1600b9b60 0x7ff1601f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:27.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff1685ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1601f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:27.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff1685ad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1601f5aa0 0x7ff1601f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff1685ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff16006f7a0 con 0x7ff1600d4d80 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff1685ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff16006f620 con 0x7ff1600b9b60 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff1685ad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff16006f920 con 0x7ff1601f5aa0 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1601f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1601f5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41300/0 (socket says 172.21.15.22:41300) 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 -- 172.21.15.22:0/563329335 learned_addr learned my addr 172.21.15.22:0/563329335 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff166b4a640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1601f5aa0 0x7ff1601f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff166349640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1600b9b60 0x7ff1601f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 -- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1601f5aa0 msgr2=0x7ff1601f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1601f5aa0 0x7ff1601f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 -- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1600b9b60 msgr2=0x7ff1601f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1600b9b60 0x7ff1601f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.733+0000 7ff165b48640 1 -- 172.21.15.22:0/563329335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1601f62b0 con 0x7ff1600d4d80 2024-09-21T05:45:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.734+0000 7ff165b48640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1601f5560 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c3c0 tx=0x7ff15000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.734+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150017070 con 0x7ff1600d4d80 2024-09-21T05:45:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.734+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1500035f0 con 0x7ff1600d4d80 2024-09-21T05:45:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.734+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150003a40 con 0x7ff1600d4d80 2024-09-21T05:45:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.734+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1600c91d0 con 0x7ff1600d4d80 2024-09-21T05:45:27.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.734+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1600c9790 con 0x7ff1600d4d80 2024-09-21T05:45:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.735+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1600cf540 con 0x7ff1600d4d80 2024-09-21T05:45:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.736+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff15001a070 con 0x7ff1600d4d80 2024-09-21T05:45:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.736+0000 7ff156ffd640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff144078b90 0x7ff14407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.737+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7ff150099fc0 con 0x7ff1600d4d80 2024-09-21T05:45:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.737+0000 7ff166349640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff144078b90 0x7ff14407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:27.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.737+0000 7ff166349640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff144078b90 0x7ff14407b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff15c000fb0 tx=0x7ff15c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:27.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.741+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1500623c0 con 0x7ff1600d4d80 2024-09-21T05:45:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.968+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff16006ef50 con 0x7ff1600d4d80 2024-09-21T05:45:27.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.968+0000 7ff156ffd640 1 -- 172.21.15.22:0/563329335 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5213 (secure 0 0 0) 0x7ff150061cc0 con 0x7ff1600d4d80 2024-09-21T05:45:27.971 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:45:27.971 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":74,"btime":"2024-09-21T05:45:27:818391+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:26.820538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-21T05:45:27.971 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 74 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.970+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff144078b90 msgr2=0x7ff14407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.970+0000 7ff1685ad640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff144078b90 0x7ff14407b0a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff15c000fb0 tx=0x7ff15c009040 comp rx=0 tx=0).stop 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.970+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 msgr2=0x7ff1601f5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.970+0000 7ff1685ad640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1601f5560 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff15000c3c0 tx=0x7ff15000c880 comp rx=0 tx=0).stop 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 shutdown_connections 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff144078b90 0x7ff14407b0a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff1601f5aa0 0x7ff1601f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1600d4d80 0x7ff1601f5560 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 --2- 172.21.15.22:0/563329335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1600b9b60 0x7ff1601f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:27.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 >> 172.21.15.22:0/563329335 conn(0x7ff16009d7f0 msgr2=0x7ff1600b1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:27.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 shutdown_connections 2024-09-21T05:45:27.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:27.971+0000 7ff1685ad640 1 -- 172.21.15.22:0/563329335 wait complete. 2024-09-21T05:45:28.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:45:26.820538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.22:6834/43392257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 43392257}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 43392257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:45:28.387 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7167:up:rejoin 2024-09-21T05:45:29.388 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:45:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 -- 172.21.15.22:0/3374100719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c0b9b60 msgr2=0x7f399c0d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:29.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 --2- 172.21.15.22:0/3374100719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c0b9b60 0x7f399c0d0fb0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f399c0d4470 tx=0x7f398c02a980 comp rx=0 tx=0).stop 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 -- 172.21.15.22:0/3374100719 shutdown_connections 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 --2- 172.21.15.22:0/3374100719 >> v2:172.21.15.134:3300/0 conn(0x7f399c162430 0x7f399c162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 --2- 172.21.15.22:0/3374100719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f399c0b3a10 0x7f399c0d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 --2- 172.21.15.22:0/3374100719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c0b9b60 0x7f399c0d0fb0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 -- 172.21.15.22:0/3374100719 >> 172.21.15.22:0/3374100719 conn(0x7f399c09d8f0 msgr2=0x7f399c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 -- 172.21.15.22:0/3374100719 shutdown_connections 2024-09-21T05:45:29.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.517+0000 7f39a1739640 1 -- 172.21.15.22:0/3374100719 wait complete. 2024-09-21T05:45:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.519+0000 7f39a1739640 1 Processor -- start 2024-09-21T05:45:29.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.519+0000 7f39a1739640 1 -- start start 2024-09-21T05:45:29.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f39a1739640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f399c0b3a10 0x7f399c161760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f39a1739640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f399c0b9b60 0x7f399c157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f39a1739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 0x7f399c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f39a1739640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f399c163110 con 0x7f399c162430 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f39a1739640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f399c162f90 con 0x7f399c0b3a10 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f39a1739640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f399c163290 con 0x7f399c0b9b60 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 0x7f399c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.520+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 0x7f399c0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41316/0 (socket says 172.21.15.22:41316) 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 -- 172.21.15.22:0/1785040815 learned_addr learned my addr 172.21.15.22:0/1785040815 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399affd640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f399c0b3a10 0x7f399c161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399a7fc640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f399c0b9b60 0x7f399c157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 -- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f399c0b9b60 msgr2=0x7f399c157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f399c0b9b60 0x7f399c157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 -- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f399c0b3a10 msgr2=0x7f399c161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:29.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f399c0b3a10 0x7f399c161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 -- 172.21.15.22:0/1785040815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c1f63d0 con 0x7f399c162430 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f399b7fe640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 0x7f399c0c9c90 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f399400c740 tx=0x7f399400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994007b50 con 0x7f399c162430 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f399400ce50 con 0x7f399c162430 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c1f66b0 con 0x7f399c162430 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994005280 con 0x7f399c162430 2024-09-21T05:45:29.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.521+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c1f6c70 con 0x7f399c162430 2024-09-21T05:45:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.522+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399c0d54e0 con 0x7f399c162430 2024-09-21T05:45:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.523+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3994005420 con 0x7f399c162430 2024-09-21T05:45:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.524+0000 7f39937fe640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f397c078b90 0x7f397c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.524+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f399409b1a0 con 0x7f399c162430 2024-09-21T05:45:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.524+0000 7f399affd640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f397c078b90 0x7f397c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:29.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.525+0000 7f399affd640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f397c078b90 0x7f397c07b0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f398c035000 tx=0x7f398c004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:29.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.528+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f399c0d54e0 con 0x7f399c162430 2024-09-21T05:45:29.756 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.754+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f399c162840 con 0x7f399c162430 2024-09-21T05:45:29.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.755+0000 7f39937fe640 1 -- 172.21.15.22:0/1785040815 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5220 (secure 0 0 0) 0x7f3994014030 con 0x7f399c162430 2024-09-21T05:45:29.757 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:45:29.757 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":75,"btime":"2024-09-21T05:45:28:820289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:28.820279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-21T05:45:29.757 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 75 2024-09-21T05:45:29.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f397c078b90 msgr2=0x7f397c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:29.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f397c078b90 0x7f397c07b0a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f398c035000 tx=0x7f398c004ab0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 msgr2=0x7f399c0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 0x7f399c0c9c90 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f399400c740 tx=0x7f399400cc00 comp rx=0 tx=0).stop 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 shutdown_connections 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f399c162430 0x7f399c0c9c90 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f397c078b90 0x7f397c07b0a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f399c0b9b60 0x7f399c157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 --2- 172.21.15.22:0/1785040815 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f399c0b3a10 0x7f399c161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 >> 172.21.15.22:0/1785040815 conn(0x7f399c09d8f0 msgr2=0x7f399c0b7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 shutdown_connections 2024-09-21T05:45:29.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:29.757+0000 7f39a1739640 1 -- 172.21.15.22:0/1785040815 wait complete. 2024-09-21T05:45:30.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:44:04.719803+0000', 'modified': '2024-09-21T05:45:28.820279+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 86, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7167}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7167': {'gid': 7167, 'name': 'c', 'rank': 0, 'incarnation': 72, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.22:6834/43392257', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 43392257}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 43392257}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7167, 'qdb_cluster': [7167]} 2024-09-21T05:45:30.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:45:30.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 -- 172.21.15.22:0/3996044481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880b3540 msgr2=0x7f17880d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3996044481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880b3540 0x7f17880d3330 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f17880ba090 tx=0x7f177002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 -- 172.21.15.22:0/3996044481 shutdown_connections 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3996044481 >> v2:172.21.15.134:3300/0 conn(0x7f17880d3870 0x7f17880ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3996044481 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880b3540 0x7f17880d3330 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 -- 172.21.15.22:0/3996044481 >> 172.21.15.22:0/3996044481 conn(0x7f178808bfc0 msgr2=0x7f178809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 -- 172.21.15.22:0/3996044481 shutdown_connections 2024-09-21T05:45:30.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.293+0000 7f178d4c4640 1 -- 172.21.15.22:0/3996044481 wait complete. 2024-09-21T05:45:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.295+0000 7f178d4c4640 1 Processor -- start 2024-09-21T05:45:30.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.295+0000 7f178d4c4640 1 -- start start 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f178d4c4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17880b3540 0x7f1788160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f178d4c4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 0x7f1788151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f178d4c4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788153bc0 0x7f1788153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f178d4c4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d0ce0 con 0x7f17880d3870 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f178d4c4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17880cf490 con 0x7f1788153bc0 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f178d4c4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17880d0e60 con 0x7f17880b3540 2024-09-21T05:45:30.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 0x7f1788151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 0x7f1788151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41332/0 (socket says 172.21.15.22:41332) 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 -- 172.21.15.22:0/3278878447 learned_addr learned my addr 172.21.15.22:0/3278878447 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f1786ffd640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17880b3540 0x7f1788160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17877fe640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788153bc0 0x7f1788153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 -- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17880b3540 msgr2=0x7f1788160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17880b3540 0x7f1788160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 -- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788153bc0 msgr2=0x7f1788153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788153bc0 0x7f1788153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 -- 172.21.15.22:0/3278878447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880ce4f0 con 0x7f17880d3870 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f17867fc640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 0x7f1788151250 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1770009920 tx=0x7f1770033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.297+0000 7f177f7fe640 1 -- 172.21.15.22:0/3278878447 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770031d60 con 0x7f17880d3870 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.298+0000 7f177f7fe640 1 -- 172.21.15.22:0/3278878447 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1770005020 con 0x7f17880d3870 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.298+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17880ce770 con 0x7f17880d3870 2024-09-21T05:45:30.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.298+0000 7f177f7fe640 1 -- 172.21.15.22:0/3278878447 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770005470 con 0x7f17880d3870 2024-09-21T05:45:30.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.298+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17881535c0 con 0x7f17880d3870 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.299+0000 7f177f7fe640 1 -- 172.21.15.22:0/3278878447 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17700047a0 con 0x7f17880d3870 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.300+0000 7f177f7fe640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1768078bc0 0x7f176807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.300+0000 7f177f7fe640 1 -- 172.21.15.22:0/3278878447 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f17700bbee0 con 0x7f17880d3870 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.300+0000 7f1786ffd640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1768078bc0 0x7f176807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.301+0000 7f1794430740 1 Processor -- start 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.301+0000 7f1786ffd640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1768078bc0 0x7f176807b0d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1780002200 tx=0x7f17800019f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.301+0000 7f1794430740 1 -- start start 2024-09-21T05:45:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb87843fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb874493b0 0x55fb878444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1794430740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1794430740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb8778c820 con 0x55fb87451590 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb8777bc50 con 0x55fb87472270 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fb8777fa60 con 0x55fb874493b0 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f17877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f17877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41334/0 (socket says 172.21.15.22:41334) 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f17877fe640 1 -- 172.21.15.22:0/630101869 learned_addr learned my addr 172.21.15.22:0/630101869 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f17867fc640 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb874493b0 0x55fb878444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.302+0000 7f1786ffd640 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb87843fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f17877fe640 1 -- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb874493b0 msgr2=0x55fb878444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f17877fe640 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb874493b0 0x55fb878444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f17877fe640 1 -- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 msgr2=0x55fb87843fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f17877fe640 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb87843fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f17877fe640 1 -- 172.21.15.22:0/630101869 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb874dd850 con 0x55fb87451590 2024-09-21T05:45:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f1786ffd640 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb87843fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f17877fe640 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878613e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1778004650 tx=0x7f1778003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d869dfcdbc248a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f177cff9640 1 -- 172.21.15.22:0/630101869 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177800ba90 con 0x55fb87451590 2024-09-21T05:45:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f177cff9640 1 -- 172.21.15.22:0/630101869 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1778015070 con 0x55fb87451590 2024-09-21T05:45:30.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.303+0000 7f177cff9640 1 -- 172.21.15.22:0/630101869 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778005020 con 0x55fb87451590 2024-09-21T05:45:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 -- 172.21.15.22:0/630101869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 msgr2=0x55fb878613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878613e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f1778004650 tx=0x7f1778003b00 comp rx=0 tx=0).stop 2024-09-21T05:45:30.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 -- 172.21.15.22:0/630101869 shutdown_connections 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878613e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb874493b0 0x55fb878444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 --2- 172.21.15.22:0/630101869 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb87843fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 -- 172.21.15.22:0/630101869 >> 172.21.15.22:0/630101869 conn(0x55fb874663c0 msgr2=0x55fb8781a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 -- 172.21.15.22:0/630101869 shutdown_connections 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.304+0000 7f1794430740 1 -- 172.21.15.22:0/630101869 wait complete. 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.305+0000 7f1794430740 1 -- start start 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.305+0000 7f1794430740 1 Processor -- start 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8786e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb8785f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb8782e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb874ec620 con 0x55fb874493b0 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb877215f0 con 0x55fb87472270 2024-09-21T05:45:30.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fb87844a20 con 0x55fb87451590 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1794430740 10 client.0 _ll_register_callbacks cb 0x55fb877250a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1786ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8786e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1786ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8786e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41346/0 (socket says 172.21.15.22:41346) 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f1786ffd640 1 -- 172.21.15.22:0/3070427196 learned_addr learned my addr 172.21.15.22:0/3070427196 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f17867fc640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb8785f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.306+0000 7f17877fe640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb8782e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1786ffd640 1 -- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 msgr2=0x55fb8785f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1786ffd640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb8785f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1786ffd640 1 -- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 msgr2=0x55fb8782e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1786ffd640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb8782e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1786ffd640 1 -- 172.21.15.22:0/3070427196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb875546b0 con 0x55fb874493b0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1786ffd640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8786e110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f1780014ef0 tx=0x7f1780000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb4912cc87e1735 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:30.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1765ffb640 1 -- 172.21.15.22:0/3070427196 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780025020 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1765ffb640 1 -- 172.21.15.22:0/3070427196 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f1780021420 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1778015070 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1765ffb640 1 -- 172.21.15.22:0/3070427196 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f178001b070 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1794430740 10 client.7242 fetch_fsmap learned FSMap version 75 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55fb8782e1b0 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1765ffb640 1 -- 172.21.15.22:0/3070427196 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1778015070 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1765ffb640 1 -- 172.21.15.22:0/3070427196 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f1780032080 con 0x55fb874493b0 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.307+0000 7f1794430740 10 client.7242 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f1794430740 10 client.7242 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f1794430740 20 client.7242 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f1794430740 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb87430e40 0x55fb87855dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f1794430740 4 client.7242 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb8778e770 con 0x55fb87430e40 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f17867fc640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb87430e40 0x55fb87855dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f17867fc640 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb87430e40 0x55fb87855dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f3a7db0956eec4 server_cookie=699969d7078351b6 in_seq=0 out_seq=0 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.308+0000 7f1765ffb640 10 client.7242 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1765ffb640 1 -- 172.21.15.22:0/3070427196 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb8778e770 con 0x55fb87430e40 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1765ffb640 10 client.7242 handle_command_reply: tid=0 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 client.7242 shutdown 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 msgr2=0x55fb8786e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8786e110 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f1780014ef0 tx=0x7f1780000f40 comp rx=0 tx=0).stop 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 shutdown_connections 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb87430e40 0x55fb87855dc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb8782e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb8785f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 --2- 172.21.15.22:0/3070427196 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8786e110 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 >> 172.21.15.22:0/3070427196 conn(0x55fb874663c0 msgr2=0x55fb8781a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1765ffb640 0 client.7242 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 shutdown_connections 2024-09-21T05:45:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 1 -- 172.21.15.22:0/3070427196 wait complete. 2024-09-21T05:45:30.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.309+0000 7f1794430740 20 client.7242 trim_cache size 0 max 16384 2024-09-21T05:45:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.328+0000 7f1794430740 1 Processor -- start 2024-09-21T05:45:30.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.328+0000 7f1794430740 1 -- start start 2024-09-21T05:45:30.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1794430740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8781a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb878d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb878d7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1794430740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1788201530 con 0x55fb874493b0 2024-09-21T05:45:30.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1788152ff0 con 0x55fb87472270 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1788200f40 con 0x55fb87451590 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1786ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8781a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1786ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8781a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41360/0 (socket says 172.21.15.22:41360) 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f1786ffd640 1 -- 172.21.15.22:0/282389655 learned_addr learned my addr 172.21.15.22:0/282389655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.329+0000 7f17867fc640 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb878d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f17877fe640 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb878d7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1786ffd640 1 -- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 msgr2=0x55fb878d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1786ffd640 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb878d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1786ffd640 1 -- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 msgr2=0x55fb878d7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1786ffd640 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb878d7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1786ffd640 1 -- 172.21.15.22:0/282389655 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb87871bf0 con 0x55fb874493b0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1786ffd640 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8781a2e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f178000fbc0 tx=0x7f17800012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6b487dd11e2e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1767fff640 1 -- 172.21.15.22:0/282389655 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780025040 con 0x55fb874493b0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1767fff640 1 -- 172.21.15.22:0/282389655 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f178001c040 con 0x55fb874493b0 2024-09-21T05:45:30.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.330+0000 7f1767fff640 1 -- 172.21.15.22:0/282389655 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780021da0 con 0x55fb874493b0 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 -- 172.21.15.22:0/282389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 msgr2=0x55fb8781a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8781a2e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f178000fbc0 tx=0x7f17800012f0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 -- 172.21.15.22:0/282389655 shutdown_connections 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb87472270 0x55fb878d7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87451590 0x55fb878d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 --2- 172.21.15.22:0/282389655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb874493b0 0x55fb8781a2e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 -- 172.21.15.22:0/282389655 >> 172.21.15.22:0/282389655 conn(0x55fb874663c0 msgr2=0x55fb87857dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 -- 172.21.15.22:0/282389655 shutdown_connections 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.331+0000 7f1794430740 1 -- 172.21.15.22:0/282389655 wait complete. 2024-09-21T05:45:30.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.332+0000 7f1794430740 1 -- start start 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.332+0000 7f1794430740 1 Processor -- start 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb874493b0 0x55fb8785add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87472270 0x55fb87881d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55fb87871990 con 0x55fb87451590 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55fb8786e700 con 0x55fb874493b0 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55fb8782f030 con 0x55fb87472270 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1794430740 10 client.0 _ll_register_callbacks cb 0x7f1780001f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41376/0 (socket says 172.21.15.22:41376) 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 -- 172.21.15.22:0/967165175 learned_addr learned my addr 172.21.15.22:0/967165175 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17877fe640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87472270 0x55fb87881d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f1786ffd640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb874493b0 0x55fb8785add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 -- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87472270 msgr2=0x55fb87881d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87472270 0x55fb87881d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 -- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb874493b0 msgr2=0x55fb8785add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb874493b0 0x55fb8785add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.333+0000 7f17867fc640 1 -- 172.21.15.22:0/967165175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb878e1410 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f17867fc640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878413b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1770083c30 tx=0x7f17700836b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a1532aba336e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1765ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1765ffb640 1 -- 172.21.15.22:0/967165175 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1770048040 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1765ffb640 1 -- 172.21.15.22:0/967165175 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f1770083f00 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178001c040 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1765ffb640 1 -- 172.21.15.22:0/967165175 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f177006f220 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 10 client.7248 fetch_fsmap learned FSMap version 75 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55fb878418f0 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1765ffb640 1 -- 172.21.15.22:0/967165175 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f178001c040 con 0x55fb87451590 2024-09-21T05:45:30.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1765ffb640 1 -- 172.21.15.22:0/967165175 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f177008d0f0 con 0x55fb87451590 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 10 client.7248 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 10 client.7248 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 20 client.7248 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb878341b0 0x55fb878dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 4 client.7248 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "status"}) -- 0x55fb87815310 con 0x55fb878341b0 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.334+0000 7f1786ffd640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb878341b0 0x55fb878dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1765ffb640 10 client.7248 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1786ffd640 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb878341b0 0x55fb878dde60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30cc289213c7258e server_cookie=606df2aca2f81685 in_seq=0 out_seq=0 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1765ffb640 1 -- 172.21.15.22:0/967165175 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55fb87815310 con 0x55fb878341b0 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1765ffb640 10 client.7248 handle_command_reply: tid=0 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 client.7248 shutdown 2024-09-21T05:45:30.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 msgr2=0x55fb878413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878413b0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f1770083c30 tx=0x7f17700836b0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 shutdown_connections 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55fb878341b0 0x55fb878dde60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55fb87472270 0x55fb87881d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55fb87451590 0x55fb878413b0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 --2- 172.21.15.22:0/967165175 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55fb874493b0 0x55fb8785add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1765ffb640 0 client.7248 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 >> 172.21.15.22:0/967165175 conn(0x55fb874663c0 msgr2=0x55fb87857dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 shutdown_connections 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 1 -- 172.21.15.22:0/967165175 wait complete. 2024-09-21T05:45:30.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.335+0000 7f1794430740 20 client.7248 trim_cache size 0 max 16384 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 7167, 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 4.5226111800000002, 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 75, 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 86, 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T05:45:30.343 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 5.4667517009999997 2024-09-21T05:45:30.344 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:45:30.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1768078bc0 msgr2=0x7f176807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1768078bc0 0x7f176807b0d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1780002200 tx=0x7f17800019f0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 msgr2=0x7f1788151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 0x7f1788151250 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f1770009920 tx=0x7f1770033c90 comp rx=0 tx=0).stop 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 shutdown_connections 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1788153bc0 0x7f1788153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17880d3870 0x7f1788151250 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1768078bc0 0x7f176807b0d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 --2- 172.21.15.22:0/3278878447 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17880b3540 0x7f1788160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 >> 172.21.15.22:0/3278878447 conn(0x7f178808bfc0 msgr2=0x7f178806f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 shutdown_connections 2024-09-21T05:45:30.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.336+0000 7f178d4c4640 1 -- 172.21.15.22:0/3278878447 wait complete. 2024-09-21T05:45:30.349 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:45:30.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.463+0000 7f5d04263640 1 -- 172.21.15.22:0/613844073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 msgr2=0x7f5cfc0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.463+0000 7f5d04263640 1 --2- 172.21.15.22:0/613844073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0d0190 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4005fd0 tx=0x7f5cf402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.464+0000 7f5d04263640 1 -- 172.21.15.22:0/613844073 shutdown_connections 2024-09-21T05:45:30.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.464+0000 7f5d04263640 1 --2- 172.21.15.22:0/613844073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0d0190 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.464+0000 7f5d04263640 1 --2- 172.21.15.22:0/613844073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cfc0cf440 0x7f5cfc15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.464+0000 7f5d04263640 1 -- 172.21.15.22:0/613844073 >> 172.21.15.22:0/613844073 conn(0x7f5cfc0710b0 msgr2=0x7f5cfc0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.464+0000 7f5d04263640 1 -- 172.21.15.22:0/613844073 shutdown_connections 2024-09-21T05:45:30.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.464+0000 7f5d04263640 1 -- 172.21.15.22:0/613844073 wait complete. 2024-09-21T05:45:30.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.466+0000 7f5d04263640 1 Processor -- start 2024-09-21T05:45:30.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.466+0000 7f5d04263640 1 -- start start 2024-09-21T05:45:30.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d04263640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d04263640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cfc0cd360 0x7f5cfc1f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d04263640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cfc159740 0x7f5cfc1f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d04263640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc0d0b70 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d04263640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc0d09f0 con 0x7f5cfc0cd360 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d04263640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5cfc0d0cf0 con 0x7f5cfc159740 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41392/0 (socket says 172.21.15.22:41392) 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 -- 172.21.15.22:0/666166065 learned_addr learned my addr 172.21.15.22:0/666166065 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d017fe640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cfc0cd360 0x7f5cfc1f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d02800640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cfc159740 0x7f5cfc1f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 -- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cfc159740 msgr2=0x7f5cfc1f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cfc159740 0x7f5cfc1f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 -- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cfc0cd360 msgr2=0x7f5cfc1f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.471 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cfc0cd360 0x7f5cfc1f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.467+0000 7f5d01fff640 1 -- 172.21.15.22:0/666166065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc0c7940 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.468+0000 7f5d01fff640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0c7400 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00ba80 tx=0x7f5cec00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.468+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec00cc00 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.468+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cec003970 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.468+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc0b1230 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.468+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec003e00 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.468+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc06f7a0 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.469+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cfc0d1250 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.470+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5cec004000 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.470+0000 7f5cfa7fc640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ce8078b90 0x7f5ce807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:30.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.470+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f5cec09acf0 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.471+0000 7f5d017fe640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ce8078b90 0x7f5ce807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:30.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.471+0000 7f5d017fe640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ce8078b90 0x7f5ce807b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0c5c50 tx=0x7f5cf000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:30.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.475+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cec062980 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.700+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5cfc0b16f0 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.700+0000 7f5cfa7fc640 1 -- 172.21.15.22:0/666166065 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) ==== 53+0+73 (secure 0 0 0) 0x7f5cec0673b0 con 0x7f5cfc0b7fe0 2024-09-21T05:45:30.703 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ce8078b90 msgr2=0x7f5ce807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ce8078b90 0x7f5ce807b0a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0c5c50 tx=0x7f5cf000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 msgr2=0x7f5cfc0c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0c7400 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5cec00ba80 tx=0x7f5cec00bf40 comp rx=0 tx=0).stop 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 shutdown_connections 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5ce8078b90 0x7f5ce807b0a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5cfc159740 0x7f5cfc1f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5cfc0cd360 0x7f5cfc1f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 --2- 172.21.15.22:0/666166065 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5cfc0b7fe0 0x7f5cfc0c7400 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 >> 172.21.15.22:0/666166065 conn(0x7f5cfc0710b0 msgr2=0x7f5cfc0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:30.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 shutdown_connections 2024-09-21T05:45:30.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30.702+0000 7f5d04263640 1 -- 172.21.15.22:0/666166065 wait complete. 2024-09-21T05:45:30.715 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:45:30.716 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:45:30.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:30.716 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:45:30.733 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42413sec preferred_lft 42413sec 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:45:30.734 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:45:30.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:30.736 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:45:30.797 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:45:30.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:30.799 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:45:30.815 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:45:30.813+0000 7fc9e0398640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:45:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:45:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:45:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:45:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:45:30.831 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:45:30.831 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:45:30.831 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:45:30.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:45:30.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:45:30.832 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:45:30.832 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:45:30.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:45:30.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:45:30.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:30.959 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:45:30.975 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:45:30.975 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:45:30.975 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:45:30.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:30.976 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:45:31.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:45:31.041 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:45:31.097 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:45:31.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:45:31.097 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:45:31.152 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:45:31.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:31.153 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:45:31.248 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:45:31.248 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:45:31.248 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:45:31.259 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:45:31.259 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:45:31.259 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:45:31.259 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:45:31.259 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:45:31.265 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:45:31.266 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:45:31.266 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:45:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:45:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:45:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:45:31.266 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:45:31.268 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:45:31.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:31.268 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:45:31.293 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:45:31.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:45:31.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:45:31.391 INFO:teuthology.orchestra.run.smithi022.stdout:client6221 2024-09-21T05:45:31.391 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:45:31.391 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:45:31.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:31.408 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 -- 172.21.15.22:0/3191616173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c180ce3a0 msgr2=0x7f9c180b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 --2- 172.21.15.22:0/3191616173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c180ce3a0 0x7f9c180b4c00 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009830 tx=0x7f9c0c02a530 comp rx=0 tx=0).stop 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 -- 172.21.15.22:0/3191616173 shutdown_connections 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 --2- 172.21.15.22:0/3191616173 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c180ce3a0 0x7f9c180b4c00 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 -- 172.21.15.22:0/3191616173 >> 172.21.15.22:0/3191616173 conn(0x7f9c18074190 msgr2=0x7f9c180d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 -- 172.21.15.22:0/3191616173 shutdown_connections 2024-09-21T05:45:31.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.566+0000 7f9c1da94640 1 -- 172.21.15.22:0/3191616173 wait complete. 2024-09-21T05:45:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.568+0000 7f9c1da94640 1 Processor -- start 2024-09-21T05:45:31.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.568+0000 7f9c1da94640 1 -- start start 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c1da94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c1809aa30 0x7f9c181560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c1da94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c1815b8f0 0x7f9c1815fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c1da94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 0x7f9c1815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c1da94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9c180b59d0 con 0x7f9c1809e590 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c1da94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c180b5850 con 0x7f9c1809aa30 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c1da94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9c180b5b50 con 0x7f9c1815b8f0 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 0x7f9c1815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 0x7f9c1815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41406/0 (socket says 172.21.15.22:41406) 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 -- 172.21.15.22:0/1503501878 learned_addr learned my addr 172.21.15.22:0/1503501878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.573 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c1809aa30 0x7f9c181560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c1815b8f0 0x7f9c1815fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 -- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c1815b8f0 msgr2=0x7f9c1815fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c1815b8f0 0x7f9c1815fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 -- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c1809aa30 msgr2=0x7f9c181560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c1809aa30 0x7f9c181560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 -- 172.21.15.22:0/1503501878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1809d990 con 0x7f9c1809e590 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9c17fff640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 0x7f9c1815e250 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c10002230 tx=0x7f9c1000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.570+0000 7f9bfffff640 1 -- 172.21.15.22:0/1503501878 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10003b70 con 0x7f9c1809e590 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.571+0000 7f9bfffff640 1 -- 172.21.15.22:0/1503501878 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c10003d10 con 0x7f9c1809e590 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.571+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1809d6b0 con 0x7f9c1809e590 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.571+0000 7f9bfffff640 1 -- 172.21.15.22:0/1503501878 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10005330 con 0x7f9c1809e590 2024-09-21T05:45:31.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.571+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1815c870 con 0x7f9c1809e590 2024-09-21T05:45:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.573+0000 7f9bfffff640 1 -- 172.21.15.22:0/1503501878 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9c100054d0 con 0x7f9c1809e590 2024-09-21T05:45:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.573+0000 7f9bfffff640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c04078bc0 0x7f9c0407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.573+0000 7f9bfffff640 1 -- 172.21.15.22:0/1503501878 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f9c1009aa10 con 0x7f9c1809e590 2024-09-21T05:45:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.573+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c04078bc0 0x7f9c0407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.574+0000 7f9c24a01740 1 Processor -- start 2024-09-21T05:45:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.574+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c04078bc0 0x7f9c0407b0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9c080046b0 tx=0x7f9c08007e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.574+0000 7f9c24a01740 1 -- start start 2024-09-21T05:45:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c6c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c24a01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562956b97120 con 0x56295687c410 2024-09-21T05:45:31.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562956b86550 con 0x56295685b730 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562956b8a360 con 0x562956853550 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.575+0000 7f9c177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41416/0 (socket says 172.21.15.22:41416) 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 -- 172.21.15.22:0/1474119041 learned_addr learned my addr 172.21.15.22:0/1474119041 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c17fff640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c6c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 -- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 msgr2=0x562956c4f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 -- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 msgr2=0x562956c6c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c6c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 -- 172.21.15.22:0/1474119041 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562956878470 con 0x56295687c410 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c17fff640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c6c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4ed10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800fb20 tx=0x7f9c08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e971e468c3973a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9bfd7fa640 1 -- 172.21.15.22:0/1474119041 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0801b050 con 0x56295687c410 2024-09-21T05:45:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9bfd7fa640 1 -- 172.21.15.22:0/1474119041 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c08025070 con 0x56295687c410 2024-09-21T05:45:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.576+0000 7f9bfd7fa640 1 -- 172.21.15.22:0/1474119041 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08020440 con 0x56295687c410 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 -- 172.21.15.22:0/1474119041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 msgr2=0x562956c4ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4ed10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800fb20 tx=0x7f9c08012000 comp rx=0 tx=0).stop 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 -- 172.21.15.22:0/1474119041 shutdown_connections 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c6c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1474119041 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4ed10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 -- 172.21.15.22:0/1474119041 >> 172.21.15.22:0/1474119041 conn(0x562956870560 msgr2=0x562956c24fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 -- 172.21.15.22:0/1474119041 shutdown_connections 2024-09-21T05:45:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.577+0000 7f9c24a01740 1 -- 172.21.15.22:0/1474119041 wait complete. 2024-09-21T05:45:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.578+0000 7f9c24a01740 1 -- start start 2024-09-21T05:45:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.578+0000 7f9c24a01740 1 Processor -- start 2024-09-21T05:45:31.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 0x562956c60900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295687c410 0x562956c48620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629568f67c0 con 0x56295685b730 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562956b2b890 con 0x56295687c410 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562956c4f790 con 0x562956853550 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 10 client.0 _ll_register_callbacks cb 0x562956b7fce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 0x562956c60900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 0x562956c60900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41428/0 (socket says 172.21.15.22:41428) 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 -- 172.21.15.22:0/1324954153 learned_addr learned my addr 172.21.15.22:0/1324954153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c17fff640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295687c410 0x562956c48620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 -- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 msgr2=0x562956c4bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 -- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295687c410 msgr2=0x562956c48620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295687c410 0x562956c48620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 -- 172.21.15.22:0/1324954153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295695c7e0 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 0x562956c60900 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c005eb0 tx=0x7f9c0c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9791a37256bf3d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9bf67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9bf67fc640 1 -- 172.21.15.22:0/1324954153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c003bb0 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9bf67fc640 1 -- 172.21.15.22:0/1324954153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f9c0c003d50 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.580+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c08025070 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9bf67fc640 1 -- 172.21.15.22:0/1324954153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f9c0c005020 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 10 client.7269 fetch_fsmap learned FSMap version 75 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5629569bcf70 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9bf67fc640 1 -- 172.21.15.22:0/1324954153 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c08025070 con 0x56295685b730 2024-09-21T05:45:31.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9bf67fc640 1 -- 172.21.15.22:0/1324954153 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f9c0c0051c0 con 0x56295685b730 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 10 client.7269 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 10 client.7269 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 20 client.7269 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x56295683b1f0 0x562956c7c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 4 client.7269 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562956b99070 con 0x56295683b1f0 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x56295683b1f0 0x562956c7c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9bf67fc640 10 client.7269 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.581+0000 7f9c177fe640 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x56295683b1f0 0x562956c7c470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c76fd8d5e267b51 server_cookie=978d3724920c050c in_seq=0 out_seq=0 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9bf67fc640 1 -- 172.21.15.22:0/1324954153 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562956b99070 con 0x56295683b1f0 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9bf67fc640 10 client.7269 handle_command_reply: tid=0 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 client.7269 shutdown 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 msgr2=0x562956c60900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 0x562956c60900 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c005eb0 tx=0x7f9c0c004610 comp rx=0 tx=0).stop 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 shutdown_connections 2024-09-21T05:45:31.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x56295683b1f0 0x562956c7c470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295687c410 0x562956c48620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295685b730 0x562956c60900 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 --2- 172.21.15.22:0/1324954153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c4bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9bf67fc640 0 client.7269 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 >> 172.21.15.22:0/1324954153 conn(0x562956870560 msgr2=0x562956c6b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 shutdown_connections 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.582+0000 7f9c24a01740 1 -- 172.21.15.22:0/1324954153 wait complete. 2024-09-21T05:45:31.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.583+0000 7f9c24a01740 20 client.7269 trim_cache size 0 max 16384 2024-09-21T05:45:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.601+0000 7f9c24a01740 1 Processor -- start 2024-09-21T05:45:31.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.601+0000 7f9c24a01740 1 -- start start 2024-09-21T05:45:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c78100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c24a01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18200270 con 0x56295687c410 2024-09-21T05:45:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181ff3e0 con 0x56295685b730 2024-09-21T05:45:31.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f9c181f7420 con 0x562956853550 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.602+0000 7f9c17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41444/0 (socket says 172.21.15.22:41444) 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 -- 172.21.15.22:0/772135933 learned_addr learned my addr 172.21.15.22:0/772135933 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c177fe640 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c78100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 -- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 msgr2=0x562956c78100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c78100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 -- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 msgr2=0x562956c4eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c4eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 -- 172.21.15.22:0/772135933 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629568c5720 con 0x56295687c410 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c177fe640 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9c17fff640 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4f2e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9c10062750 tx=0x7f9c10062690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e265b1f97dce47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9bfcff9640 1 -- 172.21.15.22:0/772135933 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100627c0 con 0x56295687c410 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9bfcff9640 1 -- 172.21.15.22:0/772135933 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c100671f0 con 0x56295687c410 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.603+0000 7f9bfcff9640 1 -- 172.21.15.22:0/772135933 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10061c40 con 0x56295687c410 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 -- 172.21.15.22:0/772135933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 msgr2=0x562956c4f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4f2e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9c10062750 tx=0x7f9c10062690 comp rx=0 tx=0).stop 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 -- 172.21.15.22:0/772135933 shutdown_connections 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956c4f2e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956c4eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 --2- 172.21.15.22:0/772135933 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956c78100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 -- 172.21.15.22:0/772135933 >> 172.21.15.22:0/772135933 conn(0x562956870560 msgr2=0x562956c4a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 -- 172.21.15.22:0/772135933 shutdown_connections 2024-09-21T05:45:31.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.604+0000 7f9c24a01740 1 -- 172.21.15.22:0/772135933 wait complete. 2024-09-21T05:45:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.605+0000 7f9c24a01740 1 -- start start 2024-09-21T05:45:31.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.605+0000 7f9c24a01740 1 Processor -- start 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956cec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956ce86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956ce93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562956c4f820 con 0x56295687c410 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9c181f7a70 con 0x56295685b730 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562956c6bc90 con 0x562956853550 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.606+0000 7f9c24a01740 10 client.0 _ll_register_callbacks cb 0x7f9c0c009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956ce93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956ce93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41450/0 (socket says 172.21.15.22:41450) 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 -- 172.21.15.22:0/3982192926 learned_addr learned my addr 172.21.15.22:0/3982192926 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c16ffd640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956ce86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c177fe640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956cec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 -- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 msgr2=0x562956cec2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956cec2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 -- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 msgr2=0x562956ce86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956ce86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 -- 172.21.15.22:0/3982192926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562956c8d310 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c17fff640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956ce93e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9c10062700 tx=0x7f9c1005ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c9d9b130bea3d8e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9bf67fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9bf67fc640 1 -- 172.21.15.22:0/3982192926 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100627c0 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9bf67fc640 1 -- 172.21.15.22:0/3982192926 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f9c10013ba0 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c100671f0 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9bf67fc640 1 -- 172.21.15.22:0/3982192926 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f9c10013da0 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c24a01740 10 client.7275 fetch_fsmap learned FSMap version 75 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.607+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x562956c8cad0 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9bf67fc640 1 -- 172.21.15.22:0/3982192926 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c100671f0 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9bf67fc640 1 -- 172.21.15.22:0/3982192926 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f9c1006d080 con 0x56295687c410 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c24a01740 10 client.7275 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c24a01740 10 client.7275 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:45:31.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c24a01740 20 client.7275 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c24a01740 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x562956ce9920 0x562956c5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c24a01740 4 client.7275 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x562956b94b20 con 0x562956ce9920 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c177fe640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x562956ce9920 0x562956c5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9bf67fc640 10 client.7275 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.608+0000 7f9c177fe640 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x562956ce9920 0x562956c5fb00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d836a45c75ff0c1 server_cookie=b7de02ccc19e800d in_seq=0 out_seq=0 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9bf67fc640 1 -- 172.21.15.22:0/3982192926 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562956b94b20 con 0x562956ce9920 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9bf67fc640 10 client.7275 handle_command_reply: tid=0 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 client.7275 shutdown 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 msgr2=0x562956ce93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956ce93e0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f9c10062700 tx=0x7f9c1005ec60 comp rx=0 tx=0).stop 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 shutdown_connections 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x562956ce9920 0x562956c5fb00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56295687c410 0x562956ce93e0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56295685b730 0x562956ce86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 --2- 172.21.15.22:0/3982192926 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562956853550 0x562956cec2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 >> 172.21.15.22:0/3982192926 conn(0x562956870560 msgr2=0x562956c78100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9bf67fc640 0 client.7275 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 shutdown_connections 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 1 -- 172.21.15.22:0/3982192926 wait complete. 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.609+0000 7f9c24a01740 20 client.7275 trim_cache size 0 max 16384 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "120b7187-8917-4da5-9d97-c98a40635a3a", 2024-09-21T05:45:31.616 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:45:31.617 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:45:31.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c04078bc0 msgr2=0x7f9c0407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c04078bc0 0x7f9c0407b0d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9c080046b0 tx=0x7f9c08007e90 comp rx=0 tx=0).stop 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 msgr2=0x7f9c1815e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 0x7f9c1815e250 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9c10002230 tx=0x7f9c1000cb90 comp rx=0 tx=0).stop 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 shutdown_connections 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f9c1809e590 0x7f9c1815e250 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9c04078bc0 0x7f9c0407b0d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9c1815b8f0 0x7f9c1815fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 --2- 172.21.15.22:0/1503501878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9c1809aa30 0x7f9c181560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 >> 172.21.15.22:0/1503501878 conn(0x7f9c18074190 msgr2=0x7f9c18071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 shutdown_connections 2024-09-21T05:45:31.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.610+0000 7f9c1da94640 1 -- 172.21.15.22:0/1503501878 wait complete. 2024-09-21T05:45:31.618 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:45:31.611+0000 7fc9dc390640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x55bd54624680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:45:31.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:45:31.623 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T05:45:31.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25eda9640 1 -- 172.21.15.22:0/2470143215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc248004d40 msgr2=0x7fc248005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2470143215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc248004d40 0x7fc248005190 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc258075010 tx=0x7fc248005830 comp rx=0 tx=0).stop 2024-09-21T05:45:31.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25d346640 1 -- 172.21.15.22:0/2470143215 reap_dead start 2024-09-21T05:45:31.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25eda9640 1 -- 172.21.15.22:0/2470143215 shutdown_connections 2024-09-21T05:45:31.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25eda9640 1 -- 172.21.15.22:0/2470143215 >> 172.21.15.22:0/2470143215 conn(0x7fc2580cee20 msgr2=0x7fc2580cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25eda9640 1 -- 172.21.15.22:0/2470143215 shutdown_connections 2024-09-21T05:45:31.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.740+0000 7fc25eda9640 1 -- 172.21.15.22:0/2470143215 wait complete. 2024-09-21T05:45:31.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.742+0000 7fc25eda9640 1 Processor -- start 2024-09-21T05:45:31.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.742+0000 7fc25eda9640 1 -- start start 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.743+0000 7fc25eda9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc248004d40 0x7fc2580b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.743+0000 7fc25eda9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 0x7fc2581f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.743+0000 7fc25eda9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2581f3570 0x7fc2580b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.743+0000 7fc25eda9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2580d1480 con 0x7fc25815cd80 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.743+0000 7fc25eda9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2580d1300 con 0x7fc2581f3570 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.743+0000 7fc25eda9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2580d1600 con 0x7fc248004d40 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 0x7fc2581f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 0x7fc2581f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41472/0 (socket says 172.21.15.22:41472) 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 -- 172.21.15.22:0/2801002222 learned_addr learned my addr 172.21.15.22:0/2801002222 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc248004d40 0x7fc2580b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc25d346640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2581f3570 0x7fc2580b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 -- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc248004d40 msgr2=0x7fc2580b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc248004d40 0x7fc2580b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 -- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2581f3570 msgr2=0x7fc2580b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2581f3570 0x7fc2580b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 -- 172.21.15.22:0/2801002222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2580b3810 con 0x7fc25815cd80 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc25d346640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2581f3570 0x7fc2580b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc248004d40 0x7fc2580b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc257fff640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 0x7fc2581f3030 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc244003fa0 tx=0x7fc244001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc2557fa640 1 -- 172.21.15.22:0/2801002222 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244015070 con 0x7fc25815cd80 2024-09-21T05:45:31.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc2557fa640 1 -- 172.21.15.22:0/2801002222 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc244011900 con 0x7fc25815cd80 2024-09-21T05:45:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc2557fa640 1 -- 172.21.15.22:0/2801002222 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244011d50 con 0x7fc25815cd80 2024-09-21T05:45:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2581f3c80 con 0x7fc25815cd80 2024-09-21T05:45:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.744+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2580b27f0 con 0x7fc25815cd80 2024-09-21T05:45:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.746+0000 7fc2557fa640 1 -- 172.21.15.22:0/2801002222 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc244018070 con 0x7fc25815cd80 2024-09-21T05:45:31.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.747+0000 7fc2557fa640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc23c078bc0 0x7fc23c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.747+0000 7fc2557fa640 1 -- 172.21.15.22:0/2801002222 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fc24409bb10 con 0x7fc25815cd80 2024-09-21T05:45:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.747+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc23c078bc0 0x7fc23c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.748+0000 7fc265ded740 1 Processor -- start 2024-09-21T05:45:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.748+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc23c078bc0 0x7fc23c07b0d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc24c001830 tx=0x7fc24c006720 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.748+0000 7fc265ded740 1 -- start start 2024-09-21T05:45:31.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc265ded740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca145a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca11a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca11ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc265ded740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608ca073a60 con 0x5608c9d59390 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608ca062e90 con 0x5608c9d304d0 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5608ca066ca0 con 0x5608c9d386b0 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc25cb45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc25cb45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca145a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41480/0 (socket says 172.21.15.22:41480) 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc25cb45640 1 -- 172.21.15.22:0/2690145850 learned_addr learned my addr 172.21.15.22:0/2690145850 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc257fff640 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca11a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.749+0000 7fc25d346640 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca11ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25cb45640 1 -- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 msgr2=0x5608ca11ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca11ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25cb45640 1 -- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 msgr2=0x5608ca11a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca11a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25cb45640 1 -- 172.21.15.22:0/2690145850 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608c9e3b7d0 con 0x5608c9d59390 2024-09-21T05:45:31.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25d346640 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca11ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca145a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc24c003930 tx=0x7fc24c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d080d62532d63c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc2427fc640 1 -- 172.21.15.22:0/2690145850 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c022070 con 0x5608c9d59390 2024-09-21T05:45:31.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc2427fc640 1 -- 172.21.15.22:0/2690145850 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24c012bc0 con 0x5608c9d59390 2024-09-21T05:45:31.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.750+0000 7fc2427fc640 1 -- 172.21.15.22:0/2690145850 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c016070 con 0x5608c9d59390 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 -- 172.21.15.22:0/2690145850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 msgr2=0x5608ca145a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca145a10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc24c003930 tx=0x7fc24c013000 comp rx=0 tx=0).stop 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 -- 172.21.15.22:0/2690145850 shutdown_connections 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca11ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca11a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 --2- 172.21.15.22:0/2690145850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca145a10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 -- 172.21.15.22:0/2690145850 >> 172.21.15.22:0/2690145850 conn(0x5608c9d4d4e0 msgr2=0x5608ca100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 -- 172.21.15.22:0/2690145850 shutdown_connections 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.751+0000 7fc265ded740 1 -- 172.21.15.22:0/2690145850 wait complete. 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.752+0000 7fc265ded740 1 -- start start 2024-09-21T05:45:31.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.752+0000 7fc265ded740 1 Processor -- start 2024-09-21T05:45:31.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca13cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca13cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca158800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608c9dd3740 con 0x5608c9d59390 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608ca008970 con 0x5608c9d304d0 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5608ca142e00 con 0x5608c9d386b0 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc265ded740 10 client.0 _ll_register_callbacks cb 0x5608ca017780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.753+0000 7fc25d346640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41492/0 (socket says 172.21.15.22:41492) 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 -- 172.21.15.22:0/2592548775 learned_addr learned my addr 172.21.15.22:0/2592548775 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc257fff640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca13cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca13cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 -- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 msgr2=0x5608ca13cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca13cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 -- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 msgr2=0x5608ca13cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca13cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 -- 172.21.15.22:0/2592548775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608c9dc4970 con 0x5608c9d59390 2024-09-21T05:45:31.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc257fff640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc25d346640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca158800 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc250036040 tx=0x7fc2500049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca2a88243424be8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc2377fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc2377fe640 1 -- 172.21.15.22:0/2592548775 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250038040 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc2377fe640 1 -- 172.21.15.22:0/2592548775 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fc250004b20 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc24c012bc0 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc2377fe640 1 -- 172.21.15.22:0/2592548775 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fc25004c050 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc265ded740 10 client.7287 fetch_fsmap learned FSMap version 75 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.754+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5608c9da26a0 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc2377fe640 1 -- 172.21.15.22:0/2592548775 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24c012bc0 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc2377fe640 1 -- 172.21.15.22:0/2592548775 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7fc25004f020 con 0x5608c9d59390 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc265ded740 10 client.7287 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc265ded740 10 client.7287 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc265ded740 20 client.7287 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc265ded740 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608c9d17f60 0x5608ca158d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc265ded740 4 client.7287 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608ca0759b0 con 0x5608c9d17f60 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608c9d17f60 0x5608ca158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc25cb45640 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608c9d17f60 0x5608ca158d40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa435568cc406cb server_cookie=f36a28f1afa73620 in_seq=0 out_seq=0 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.755+0000 7fc2377fe640 10 client.7287 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc2377fe640 1 -- 172.21.15.22:0/2592548775 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5608ca0759b0 con 0x5608c9d17f60 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc2377fe640 10 client.7287 handle_command_reply: tid=0 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 client.7287 shutdown 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 msgr2=0x5608ca158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca158800 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc250036040 tx=0x7fc2500049c0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 shutdown_connections 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608c9d17f60 0x5608ca158d40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d59390 0x5608ca158800 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d386b0 0x5608ca13cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 --2- 172.21.15.22:0/2592548775 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d304d0 0x5608ca13cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc2377fe640 0 client.7287 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 >> 172.21.15.22:0/2592548775 conn(0x5608c9d4d4e0 msgr2=0x5608ca0fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 shutdown_connections 2024-09-21T05:45:31.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 1 -- 172.21.15.22:0/2592548775 wait complete. 2024-09-21T05:45:31.763 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.756+0000 7fc265ded740 20 client.7287 trim_cache size 0 max 16384 2024-09-21T05:45:31.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.774+0000 7fc265ded740 1 Processor -- start 2024-09-21T05:45:31.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.774+0000 7fc265ded740 1 -- start start 2024-09-21T05:45:31.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc265ded740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 0x5608ca1412c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d386b0 0x5608ca167cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d59390 0x5608ca168200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc265ded740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2582000f0 con 0x5608c9d304d0 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581ff0f0 con 0x5608c9d386b0 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2581f7120 con 0x5608c9d59390 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 0x5608ca1412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 0x5608ca1412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41508/0 (socket says 172.21.15.22:41508) 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 -- 172.21.15.22:0/227920766 learned_addr learned my addr 172.21.15.22:0/227920766 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25d346640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d59390 0x5608ca168200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc257fff640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d386b0 0x5608ca167cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 -- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d59390 msgr2=0x5608ca168200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d59390 0x5608ca168200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 -- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d386b0 msgr2=0x5608ca167cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d386b0 0x5608ca167cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 -- 172.21.15.22:0/227920766 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ca0fd860 con 0x5608c9d304d0 2024-09-21T05:45:31.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25d346640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d59390 0x5608ca168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc257fff640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d386b0 0x5608ca167cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc25cb45640 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 0x5608ca1412c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc24c001ea0 tx=0x7fc2580ccee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f408601e539f30c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc241ffb640 1 -- 172.21.15.22:0/227920766 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c016040 con 0x5608c9d304d0 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc241ffb640 1 -- 172.21.15.22:0/227920766 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24c012940 con 0x5608c9d304d0 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.776+0000 7fc241ffb640 1 -- 172.21.15.22:0/227920766 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24c022040 con 0x5608c9d304d0 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 -- 172.21.15.22:0/227920766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 msgr2=0x5608ca1412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 0x5608ca1412c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc24c001ea0 tx=0x7fc2580ccee0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 -- 172.21.15.22:0/227920766 shutdown_connections 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d59390 0x5608ca168200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d386b0 0x5608ca167cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 --2- 172.21.15.22:0/227920766 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d304d0 0x5608ca1412c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 -- 172.21.15.22:0/227920766 >> 172.21.15.22:0/227920766 conn(0x5608c9d4d4e0 msgr2=0x5608ca1277f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 -- 172.21.15.22:0/227920766 shutdown_connections 2024-09-21T05:45:31.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.777+0000 7fc265ded740 1 -- 172.21.15.22:0/227920766 wait complete. 2024-09-21T05:45:31.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.778+0000 7fc265ded740 1 -- start start 2024-09-21T05:45:31.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.778+0000 7fc265ded740 1 Processor -- start 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d304d0 0x5608ca167150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 0x5608ca1bb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d59390 0x5608ca118ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581fead0 con 0x5608c9d386b0 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608ca1679e0 con 0x5608c9d59390 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5608ca1c5460 con 0x5608c9d304d0 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc265ded740 10 client.0 _ll_register_callbacks cb 0x7fc250036070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 0x5608ca1bb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 0x5608ca1bb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41524/0 (socket says 172.21.15.22:41524) 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 -- 172.21.15.22:0/11810273 learned_addr learned my addr 172.21.15.22:0/11810273 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc25cb45640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d304d0 0x5608ca167150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc25d346640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d59390 0x5608ca118ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 -- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d304d0 msgr2=0x5608ca167150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d304d0 0x5608ca167150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 -- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d59390 msgr2=0x5608ca118ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d59390 0x5608ca118ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 -- 172.21.15.22:0/11810273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608ca12b960 con 0x5608c9d386b0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc25d346640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d59390 0x5608ca118ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc257fff640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 0x5608ca1bb3e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc244001030 tx=0x7fc244099e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7245d508c46fa69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc2377fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.780+0000 7fc2377fe640 1 -- 172.21.15.22:0/11810273 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24401f040 con 0x5608c9d386b0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc2377fe640 1 -- 172.21.15.22:0/11810273 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fc244064b20 con 0x5608c9d386b0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc24c012940 con 0x5608c9d386b0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc2377fe640 1 -- 172.21.15.22:0/11810273 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fc244062f60 con 0x5608c9d386b0 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 10 client.7293 fetch_fsmap learned FSMap version 75 2024-09-21T05:45:31.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x5608ca167720 con 0x5608c9d386b0 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc2377fe640 1 -- 172.21.15.22:0/11810273 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24c012940 con 0x5608c9d386b0 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc2377fe640 1 -- 172.21.15.22:0/11810273 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7fc24405f140 con 0x5608c9d386b0 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 10 client.7293 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 10 client.7293 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 20 client.7293 populate_metadata read hostname 'smithi022' 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608ca119410 0x5608ca1c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 4 client.7293 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5608ca071460 con 0x5608ca119410 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.781+0000 7fc25cb45640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608ca119410 0x5608ca1c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc2377fe640 10 client.7293 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc25cb45640 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608ca119410 0x5608ca1c7fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb0aaa2e6657c7f server_cookie=9009692e7390a61 in_seq=0 out_seq=0 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc2377fe640 1 -- 172.21.15.22:0/11810273 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+577 (crc 0 0 0) 0x5608ca071460 con 0x5608ca119410 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc2377fe640 10 client.7293 handle_command_reply: tid=0 2024-09-21T05:45:31.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 client.7293 shutdown 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 msgr2=0x5608ca1bb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 0x5608ca1bb3e0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc244001030 tx=0x7fc244099e20 comp rx=0 tx=0).stop 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 shutdown_connections 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x5608ca119410 0x5608ca1c7fd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608c9d59390 0x5608ca118ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608c9d386b0 0x5608ca1bb3e0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 --2- 172.21.15.22:0/11810273 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608c9d304d0 0x5608ca167150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 >> 172.21.15.22:0/11810273 conn(0x5608c9d4d4e0 msgr2=0x5608ca1422c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc2377fe640 0 client.7293 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 shutdown_connections 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 1 -- 172.21.15.22:0/11810273 wait complete. 2024-09-21T05:45:31.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.782+0000 7fc265ded740 20 client.7293 trim_cache size 0 max 16384 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "120b7187-8917-4da5-9d97-c98a40635a3a": { 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "120b7187-8917-4da5-9d97-c98a40635a3a", 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,repair" 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "120b7187-8917-4da5-9d97-c98a40635a3a": { 2024-09-21T05:45:31.790 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:45:31.608018+0000", 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:45:31.791 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc23c078bc0 msgr2=0x7fc23c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc23c078bc0 0x7fc23c07b0d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc24c001830 tx=0x7fc24c006720 comp rx=0 tx=0).stop 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 msgr2=0x7fc2581f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 0x7fc2581f3030 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc244003fa0 tx=0x7fc244001770 comp rx=0 tx=0).stop 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 shutdown_connections 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2581f3570 0x7fc2580b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc25815cd80 0x7fc2581f3030 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc23c078bc0 0x7fc23c07b0d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 --2- 172.21.15.22:0/2801002222 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc248004d40 0x7fc2580b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:45:31.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 >> 172.21.15.22:0/2801002222 conn(0x7fc2580cee20 msgr2=0x7fc2580b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:45:31.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 shutdown_connections 2024-09-21T05:45:31.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:45:31.783+0000 7fc25eda9640 1 -- 172.21.15.22:0/2801002222 wait complete. 2024-09-21T05:45:31.796 INFO:tasks.cephfs.filesystem:scrub status for tag:120b7187-8917-4da5-9d97-c98a40635a3a - {'path': '/dir', 'tag': '120b7187-8917-4da5-9d97-c98a40635a3a', 'options': 'recursive,repair'} 2024-09-21T05:45:52.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:45:52.360 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:46:01.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:01.797 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-21T05:46:01.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f984a841640 1 -- 172.21.15.22:0/3964232789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f982c004800 msgr2=0x7f982c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f984a841640 1 --2- 172.21.15.22:0/3964232789 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f982c004800 0x7f982c004c10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f9844074550 tx=0x7f982c005830 comp rx=0 tx=0).stop 2024-09-21T05:46:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f9848dde640 1 -- 172.21.15.22:0/3964232789 reap_dead start 2024-09-21T05:46:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f984a841640 1 -- 172.21.15.22:0/3964232789 shutdown_connections 2024-09-21T05:46:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f984a841640 1 -- 172.21.15.22:0/3964232789 >> 172.21.15.22:0/3964232789 conn(0x7f98440b3f10 msgr2=0x7f98440b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f984a841640 1 -- 172.21.15.22:0/3964232789 shutdown_connections 2024-09-21T05:46:01.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.930+0000 7f984a841640 1 -- 172.21.15.22:0/3964232789 wait complete. 2024-09-21T05:46:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.932+0000 7f984a841640 1 Processor -- start 2024-09-21T05:46:01.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.932+0000 7f984a841640 1 -- start start 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.933+0000 7f984a841640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f982c004800 0x7f98440b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.933+0000 7f984a841640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 0x7f984415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.933+0000 7f984a841640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f984415cc30 0x7f98441f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.933+0000 7f984a841640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98440d0fe0 con 0x7f98440cbb50 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.933+0000 7f984a841640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98440d0e60 con 0x7f984415cc30 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.933+0000 7f984a841640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98440d1160 con 0x7f982c004800 2024-09-21T05:46:01.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 0x7f984415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 0x7f984415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49426/0 (socket says 172.21.15.22:49426) 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 -- 172.21.15.22:0/1023923713 learned_addr learned my addr 172.21.15.22:0/1023923713 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f9843fff640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f982c004800 0x7f98440b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f9848dde640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f984415cc30 0x7f98441f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 -- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f982c004800 msgr2=0x7f98440b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f982c004800 0x7f98440b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 -- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f984415cc30 msgr2=0x7f98441f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f984415cc30 0x7f98441f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 -- 172.21.15.22:0/1023923713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984415d040 con 0x7f98440cbb50 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f98437fe640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 0x7f984415c6f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f983802b4a0 tx=0x7f9838005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f9840ff9640 1 -- 172.21.15.22:0/1023923713 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838038050 con 0x7f98440cbb50 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f9840ff9640 1 -- 172.21.15.22:0/1023923713 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9838039040 con 0x7f98440cbb50 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441f1000 con 0x7f98440cbb50 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.934+0000 7f9840ff9640 1 -- 172.21.15.22:0/1023923713 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838004280 con 0x7f98440cbb50 2024-09-21T05:46:01.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.935+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441f15c0 con 0x7f98440cbb50 2024-09-21T05:46:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.936+0000 7f9840ff9640 1 -- 172.21.15.22:0/1023923713 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9838046020 con 0x7f98440cbb50 2024-09-21T05:46:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.937+0000 7f9840ff9640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9830078bc0 0x7f983007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.937+0000 7f9840ff9640 1 -- 172.21.15.22:0/1023923713 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f98380bb8a0 con 0x7f98440cbb50 2024-09-21T05:46:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.938+0000 7f9843fff640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9830078bc0 0x7f983007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.938+0000 7f9843fff640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9830078bc0 0x7f983007b0d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9834001050 tx=0x7f98340011e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.938+0000 7f985188e740 1 Processor -- start 2024-09-21T05:46:01.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.938+0000 7f985188e740 1 -- start start 2024-09-21T05:46:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.939+0000 7f985188e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 0x563f7761e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.939+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7761ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.939+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7722c6b0 0x563f7763bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.939+0000 7f985188e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563f77567a60 con 0x563f7724d390 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.939+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f77556e90 con 0x563f7722c6b0 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.939+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563f7755aca0 con 0x563f772244d0 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 0x563f7761e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 0x563f7761e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49436/0 (socket says 172.21.15.22:49436) 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 -- 172.21.15.22:0/3377137274 learned_addr learned my addr 172.21.15.22:0/3377137274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9848dde640 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7722c6b0 0x563f7763bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f98437fe640 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7761ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 -- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 msgr2=0x563f7761ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7761ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 -- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7722c6b0 msgr2=0x563f7763bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7722c6b0 0x563f7763bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 -- 172.21.15.22:0/3377137274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f7732f7d0 con 0x563f7724d390 2024-09-21T05:46:01.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9843fff640 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 0x563f7761e740 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9834003950 tx=0x7f9834013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe93bdf930ef6de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:01.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9829ffb640 1 -- 172.21.15.22:0/3377137274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834003e10 con 0x563f7724d390 2024-09-21T05:46:01.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9829ffb640 1 -- 172.21.15.22:0/3377137274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f983401a450 con 0x563f7724d390 2024-09-21T05:46:01.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.940+0000 7f9829ffb640 1 -- 172.21.15.22:0/3377137274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983401a800 con 0x563f7724d390 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 -- 172.21.15.22:0/3377137274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 msgr2=0x563f7761e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 0x563f7761e740 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9834003950 tx=0x7f9834013000 comp rx=0 tx=0).stop 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 -- 172.21.15.22:0/3377137274 shutdown_connections 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7722c6b0 0x563f7763bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7761ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 --2- 172.21.15.22:0/3377137274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7724d390 0x563f7761e740 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 -- 172.21.15.22:0/3377137274 >> 172.21.15.22:0/3377137274 conn(0x563f772414e0 msgr2=0x563f775f4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 -- 172.21.15.22:0/3377137274 shutdown_connections 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.941+0000 7f985188e740 1 -- 172.21.15.22:0/3377137274 wait complete. 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.942+0000 7f985188e740 1 -- start start 2024-09-21T05:46:01.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.942+0000 7f985188e740 1 Processor -- start 2024-09-21T05:46:01.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7760b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f7760cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7724d390 0x563f77631e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563f772c7740 con 0x563f7722c6b0 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563f774fc970 con 0x563f7724d390 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563f7761f1c0 con 0x563f772244d0 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f985188e740 10 client.0 _ll_register_callbacks cb 0x563f774ffbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f7760cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f7760cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49452/0 (socket says 172.21.15.22:49452) 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 -- 172.21.15.22:0/3138704101 learned_addr learned my addr 172.21.15.22:0/3138704101 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f9848dde640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7724d390 0x563f77631e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f9843fff640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7760b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 -- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 msgr2=0x563f7760b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7760b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 -- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7724d390 msgr2=0x563f77631e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7724d390 0x563f77631e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 -- 172.21.15.22:0/3138704101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f772b8970 con 0x563f7722c6b0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f98437fe640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f7760cb00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9838083660 tx=0x7f9838082f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b115d4bf5c3040b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.944+0000 7f9822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f9822ffd640 1 -- 172.21.15.22:0/3138704101 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983808cc90 con 0x563f7722c6b0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f9822ffd640 1 -- 172.21.15.22:0/3138704101 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f9838082930 con 0x563f7722c6b0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983401a450 con 0x563f7722c6b0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f9822ffd640 1 -- 172.21.15.22:0/3138704101 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f98380a4050 con 0x563f7722c6b0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 10 client.7305 fetch_fsmap learned FSMap version 75 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x563f772966a0 con 0x563f7722c6b0 2024-09-21T05:46:01.952 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f9822ffd640 1 -- 172.21.15.22:0/3138704101 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f983401a450 con 0x563f7722c6b0 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f9822ffd640 1 -- 172.21.15.22:0/3138704101 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f983808a300 con 0x563f7722c6b0 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 10 client.7305 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 10 client.7305 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 20 client.7305 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f7720bf60 0x563f7763ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 4 client.7305 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f775699b0 con 0x563f7720bf60 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.945+0000 7f9843fff640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f7720bf60 0x563f7763ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.946+0000 7f9822ffd640 10 client.7305 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:01.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.946+0000 7f9843fff640 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f7720bf60 0x563f7763ce00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ef88d66c435485 server_cookie=c2f73680dc5cdb1e in_seq=0 out_seq=0 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.946+0000 7f9822ffd640 1 -- 172.21.15.22:0/3138704101 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563f775699b0 con 0x563f7720bf60 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.946+0000 7f9822ffd640 10 client.7305 handle_command_reply: tid=0 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.946+0000 7f985188e740 1 client.7305 shutdown 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 msgr2=0x563f7760cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f7760cb00 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f9838083660 tx=0x7f9838082f00 comp rx=0 tx=0).stop 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 shutdown_connections 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f7720bf60 0x563f7763ce00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f7724d390 0x563f77631e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f7760cb00 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 --2- 172.21.15.22:0/3138704101 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f772244d0 0x563f7760b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 >> 172.21.15.22:0/3138704101 conn(0x563f772414e0 msgr2=0x563f775ef2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f9822ffd640 0 client.7305 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 shutdown_connections 2024-09-21T05:46:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 1 -- 172.21.15.22:0/3138704101 wait complete. 2024-09-21T05:46:01.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.947+0000 7f985188e740 20 client.7305 trim_cache size 0 max 16384 2024-09-21T05:46:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.965+0000 7f985188e740 1 Processor -- start 2024-09-21T05:46:01.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.965+0000 7f985188e740 1 -- start start 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f77615f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f985188e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f77616480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7763b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f985188e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98441fe4f0 con 0x563f7722c6b0 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98441fd530 con 0x563f772244d0 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98441fc8c0 con 0x563f7724d390 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f77616480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f77616480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49462/0 (socket says 172.21.15.22:49462) 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f98437fe640 1 -- 172.21.15.22:0/1182853993 learned_addr learned my addr 172.21.15.22:0/1182853993 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f9843fff640 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f77615f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.966+0000 7f9848dde640 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7763b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.967+0000 7f98437fe640 1 -- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 msgr2=0x563f7763b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.967+0000 7f98437fe640 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7763b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.967+0000 7f98437fe640 1 -- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 msgr2=0x563f77615f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.967+0000 7f98437fe640 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f77615f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.967+0000 7f98437fe640 1 -- 172.21.15.22:0/1182853993 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f775f2160 con 0x563f7722c6b0 2024-09-21T05:46:01.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.967+0000 7f98437fe640 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f77616480 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9838082f90 tx=0x7f98380834a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7597f5f740cc80e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.968+0000 7f98297fa640 1 -- 172.21.15.22:0/1182853993 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983808e070 con 0x563f7722c6b0 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.968+0000 7f98297fa640 1 -- 172.21.15.22:0/1182853993 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9838088100 con 0x563f7722c6b0 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.968+0000 7f98297fa640 1 -- 172.21.15.22:0/1182853993 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983808cc90 con 0x563f7722c6b0 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 -- 172.21.15.22:0/1182853993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 msgr2=0x563f77616480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f77616480 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f9838082f90 tx=0x7f98380834a0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 -- 172.21.15.22:0/1182853993 shutdown_connections 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7763b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f77616480 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 --2- 172.21.15.22:0/1182853993 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f77615f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 -- 172.21.15.22:0/1182853993 >> 172.21.15.22:0/1182853993 conn(0x563f772414e0 msgr2=0x563f7763b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 -- 172.21.15.22:0/1182853993 shutdown_connections 2024-09-21T05:46:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.969+0000 7f985188e740 1 -- 172.21.15.22:0/1182853993 wait complete. 2024-09-21T05:46:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.970+0000 7f985188e740 1 -- start start 2024-09-21T05:46:01.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.970+0000 7f985188e740 1 Processor -- start 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.971+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f775ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.971+0000 7f985188e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f776aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.971+0000 7f985188e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7760c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563f7765c8a0 con 0x563f7722c6b0 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98441fcf10 con 0x563f772244d0 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563f7765bef0 con 0x563f7724d390 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 10 client.0 _ll_register_callbacks cb 0x7f9838083570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f776aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f776aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49476/0 (socket says 172.21.15.22:49476) 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 -- 172.21.15.22:0/3677393878 learned_addr learned my addr 172.21.15.22:0/3677393878 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f9843fff640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f775ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f9848dde640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7760c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 -- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 msgr2=0x563f7760c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7760c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 -- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 msgr2=0x563f775ee260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f775ee260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 -- 172.21.15.22:0/3677393878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f7761f1f0 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f98437fe640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f776aee40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f983807fc50 tx=0x7f983808ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bca993974fa7e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f9822ffd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f9822ffd640 1 -- 172.21.15.22:0/3677393878 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983808e040 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f9822ffd640 1 -- 172.21.15.22:0/3677393878 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f983807f020 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9838088100 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f9822ffd640 1 -- 172.21.15.22:0/3677393878 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f983808cf20 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 10 client.7311 fetch_fsmap learned FSMap version 75 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.972+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x563f775ee830 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f9822ffd640 1 -- 172.21.15.22:0/3677393878 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9838088100 con 0x563f7722c6b0 2024-09-21T05:46:01.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f9822ffd640 1 -- 172.21.15.22:0/3677393878 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f9838099120 con 0x563f7722c6b0 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f985188e740 10 client.7311 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f985188e740 10 client.7311 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f985188e740 20 client.7311 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f985188e740 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f77630990 0x563f77615f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f985188e740 4 client.7311 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f77565460 con 0x563f77630990 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f9843fff640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f77630990 0x563f77615f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f9822ffd640 10 client.7311 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.973+0000 7f9843fff640 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f77630990 0x563f77615f40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=736a3563360717e5 server_cookie=78ee96c20c1e9e5e in_seq=0 out_seq=0 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f9822ffd640 1 -- 172.21.15.22:0/3677393878 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+377 (crc 0 0 0) 0x563f77565460 con 0x563f77630990 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f9822ffd640 10 client.7311 handle_command_reply: tid=0 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 client.7311 shutdown 2024-09-21T05:46:01.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 msgr2=0x563f776aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f776aee40 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f983807fc50 tx=0x7f983808ce50 comp rx=0 tx=0).stop 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 shutdown_connections 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x563f77630990 0x563f77615f40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563f7724d390 0x563f7760c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563f7722c6b0 0x563f776aee40 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f9822ffd640 0 client.7311 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 --2- 172.21.15.22:0/3677393878 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563f772244d0 0x563f775ee260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 >> 172.21.15.22:0/3677393878 conn(0x563f772414e0 msgr2=0x563f775ef450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 shutdown_connections 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 1 -- 172.21.15.22:0/3677393878 wait complete. 2024-09-21T05:46:01.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.974+0000 7f985188e740 20 client.7311 trim_cache size 0 max 16384 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "120b7187-8917-4da5-9d97-c98a40635a3a": { 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:45:31.608018+0000", 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/dir", 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:46:01.981 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:46:01.982 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:46:01.982 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:01.982 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:01.982 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9830078bc0 msgr2=0x7f983007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9830078bc0 0x7f983007b0d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9834001050 tx=0x7f98340011e0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 msgr2=0x7f984415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 0x7f984415c6f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f983802b4a0 tx=0x7f9838005dc0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 shutdown_connections 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f984415cc30 0x7f98441f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f98440cbb50 0x7f984415c6f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9830078bc0 0x7f983007b0d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 --2- 172.21.15.22:0/1023923713 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f982c004800 0x7f98440b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 >> 172.21.15.22:0/1023923713 conn(0x7f98440b3f10 msgr2=0x7f984415a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 shutdown_connections 2024-09-21T05:46:01.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:01.975+0000 7f984a841640 1 -- 172.21.15.22:0/1023923713 wait complete. 2024-09-21T05:46:01.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:46:01.988 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3317032881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 msgr2=0x7fcb940b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3317032881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb940b63b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcb9408d160 tx=0x7fcb8802a570 comp rx=0 tx=0).stop 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3317032881 shutdown_connections 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3317032881 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 0x7fcb940cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3317032881 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb940b63b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3317032881 >> v2:172.21.15.134:3300/0 conn(0x7fcb940b9970 0x7fcb940b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3317032881 >> 172.21.15.22:0/3317032881 conn(0x7fcb9409d7f0 msgr2=0x7fcb940d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3317032881 shutdown_connections 2024-09-21T05:46:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.103+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3317032881 wait complete. 2024-09-21T05:46:02.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.104+0000 7fcb9b0e6640 1 Processor -- start 2024-09-21T05:46:02.106 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.104+0000 7fcb9b0e6640 1 -- start start 2024-09-21T05:46:02.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb9b0e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb94161d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb9b0e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb940b9970 0x7fcb9415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb9b0e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 0x7fcb940cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb9b0e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb940cf850 con 0x7fcb940b5fa0 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb9b0e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb940cf6d0 con 0x7fcb940b9970 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb9b0e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb940cf9d0 con 0x7fcb940ced80 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb94161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb94161d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49506/0 (socket says 172.21.15.22:49506) 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 -- 172.21.15.22:0/3248516431 learned_addr learned my addr 172.21.15.22:0/3248516431 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb93fff640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb940b9970 0x7fcb9415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb99683640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 0x7fcb940cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 -- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 msgr2=0x7fcb940cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 0x7fcb940cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 -- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb940b9970 msgr2=0x7fcb9415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb940b9970 0x7fcb9415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb98e82640 1 -- 172.21.15.22:0/3248516431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb940c9dc0 con 0x7fcb940b5fa0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.106+0000 7fcb99683640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 0x7fcb940cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.107+0000 7fcb98e82640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb94161d90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c940 tx=0x7fcb8400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.107+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84004220 con 0x7fcb940b5fa0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.107+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb840079b0 con 0x7fcb940b5fa0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.107+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb940ca0a0 con 0x7fcb940b5fa0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.107+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb84007e00 con 0x7fcb940b5fa0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.107+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb940cca40 con 0x7fcb940b5fa0 2024-09-21T05:46:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.108+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb940cdf40 con 0x7fcb940b5fa0 2024-09-21T05:46:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.108+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb84005030 con 0x7fcb940b5fa0 2024-09-21T05:46:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.109+0000 7fcb917fa640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb80078b90 0x7fcb8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.109+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fcb8409b330 con 0x7fcb940b5fa0 2024-09-21T05:46:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.109+0000 7fcb93fff640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb80078b90 0x7fcb8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.110+0000 7fcb93fff640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb80078b90 0x7fcb8007b0a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcb88033000 tx=0x7fcb88004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.114+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb940cdf40 con 0x7fcb940b5fa0 2024-09-21T05:46:02.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.341+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb94076820 con 0x7fcb940b5fa0 2024-09-21T05:46:02.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.341+0000 7fcb917fa640 1 -- 172.21.15.22:0/3248516431 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5220 (secure 0 0 0) 0x7fcb84014070 con 0x7fcb940b5fa0 2024-09-21T05:46:02.343 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:02.344 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":75,"btime":"2024-09-21T05:45:28:820289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:28.820279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-21T05:46:02.344 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 75 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.343+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb80078b90 msgr2=0x7fcb8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.343+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb80078b90 0x7fcb8007b0a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcb88033000 tx=0x7fcb88004730 comp rx=0 tx=0).stop 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.343+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 msgr2=0x7fcb94161d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.343+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb94161d90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c940 tx=0x7fcb8400ce00 comp rx=0 tx=0).stop 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 shutdown_connections 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb80078b90 0x7fcb8007b0a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb940ced80 0x7fcb940cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb940b9970 0x7fcb9415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 --2- 172.21.15.22:0/3248516431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcb940b5fa0 0x7fcb94161d90 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 >> 172.21.15.22:0/3248516431 conn(0x7fcb9409d7f0 msgr2=0x7fcb940b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 shutdown_connections 2024-09-21T05:46:02.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.344+0000 7fcb9b0e6640 1 -- 172.21.15.22:0/3248516431 wait complete. 2024-09-21T05:46:02.357 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-21T05:46:02.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 -- 172.21.15.22:0/562486442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc15f9b0 msgr2=0x7f7edc0ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 --2- 172.21.15.22:0/562486442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc15f9b0 0x7f7edc0ce680 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4005fd0 tx=0x7f7ed402a5d0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 -- 172.21.15.22:0/562486442 shutdown_connections 2024-09-21T05:46:02.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 --2- 172.21.15.22:0/562486442 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc15f9b0 0x7f7edc0ce680 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 --2- 172.21.15.22:0/562486442 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7edc075fa0 0x7f7edc070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 -- 172.21.15.22:0/562486442 >> 172.21.15.22:0/562486442 conn(0x7f7edc0cd530 msgr2=0x7f7edc0cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 -- 172.21.15.22:0/562486442 shutdown_connections 2024-09-21T05:46:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.474+0000 7f7ee0830640 1 -- 172.21.15.22:0/562486442 wait complete. 2024-09-21T05:46:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.476+0000 7f7ee0830640 1 Processor -- start 2024-09-21T05:46:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.476+0000 7f7ee0830640 1 -- start start 2024-09-21T05:46:02.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ee0830640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7edc06e100 0x7f7edc0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ee0830640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 0x7f7edc1e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ee0830640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7edc1f6970 0x7f7edc1e6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ee0830640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0cefe0 con 0x7f7edc075fa0 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ee0830640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0cee60 con 0x7f7edc1f6970 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ee0830640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0cf160 con 0x7f7edc06e100 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ed9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 0x7f7edc1e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ed9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 0x7f7edc1e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49512/0 (socket says 172.21.15.22:49512) 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1035393251 learned_addr learned my addr 172.21.15.22:0/1035393251 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7eda59c640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7edc06e100 0x7f7edc0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.477+0000 7f7edad9d640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7edc1f6970 0x7f7edc1e6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7edc06e100 msgr2=0x7f7edc0ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7edc06e100 0x7f7edc0ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7edc1f6970 msgr2=0x7f7edc1e6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7edc1f6970 0x7f7edc1e6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1035393251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc1f70a0 con 0x7f7edc075fa0 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 0x7f7edc1e65e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c340 tx=0x7f7ec400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed2ffd640 1 -- 172.21.15.22:0/1035393251 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4017070 con 0x7f7edc075fa0 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed2ffd640 1 -- 172.21.15.22:0/1035393251 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ec40053c0 con 0x7f7edc075fa0 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc1f7380 con 0x7f7edc075fa0 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ed2ffd640 1 -- 172.21.15.22:0/1035393251 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4003850 con 0x7f7edc075fa0 2024-09-21T05:46:02.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.478+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc0c5b50 con 0x7f7edc075fa0 2024-09-21T05:46:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.480+0000 7f7ed2ffd640 1 -- 172.21.15.22:0/1035393251 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ec401a070 con 0x7f7edc075fa0 2024-09-21T05:46:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.481+0000 7f7ed2ffd640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7ebc078b90 0x7f7ebc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.481+0000 7f7ed2ffd640 1 -- 172.21.15.22:0/1035393251 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f7ec409a170 con 0x7f7edc075fa0 2024-09-21T05:46:02.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.481+0000 7f7eda59c640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7ebc078b90 0x7f7ebc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.481+0000 7f7ee787b740 1 Processor -- start 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.481+0000 7f7eda59c640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7ebc078b90 0x7f7ebc07b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc005fd0 tx=0x7f7ecc0095e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.482+0000 7f7ee787b740 1 -- start start 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b659350 0x55641ba2ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba2b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ee787b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55641b9739a0 con 0x55641b630490 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55641b962dd0 con 0x55641b659350 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55641b966be0 con 0x55641b638670 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba2b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba2b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49526/0 (socket says 172.21.15.22:49526) 2024-09-21T05:46:02.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1920205657 learned_addr learned my addr 172.21.15.22:0/1920205657 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7eda59c640 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b659350 0x55641ba2ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7edad9d640 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 msgr2=0x55641ba481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b659350 msgr2=0x55641ba2ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b659350 0x55641ba2ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.483+0000 7f7ed9d9b640 1 -- 172.21.15.22:0/1920205657 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641b8fb700 con 0x55641b630490 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.484+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba2b2c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4061e10 tx=0x7f7ec405de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8eb6e2e6d726fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.484+0000 7f7ebbfff640 1 -- 172.21.15.22:0/1920205657 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4088020 con 0x55641b630490 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.484+0000 7f7ebbfff640 1 -- 172.21.15.22:0/1920205657 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ec40668b0 con 0x55641b630490 2024-09-21T05:46:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.484+0000 7f7ebbfff640 1 -- 172.21.15.22:0/1920205657 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4069e00 con 0x55641b630490 2024-09-21T05:46:02.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.484+0000 7f7ee787b740 1 -- 172.21.15.22:0/1920205657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 msgr2=0x55641ba2b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.484+0000 7f7ee787b740 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba2b2c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4061e10 tx=0x7f7ec405de50 comp rx=0 tx=0).stop 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 -- 172.21.15.22:0/1920205657 shutdown_connections 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba2b2c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 --2- 172.21.15.22:0/1920205657 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b659350 0x55641ba2ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 -- 172.21.15.22:0/1920205657 >> 172.21.15.22:0/1920205657 conn(0x55641b64d4a0 msgr2=0x55641ba01040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 -- 172.21.15.22:0/1920205657 shutdown_connections 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.485+0000 7f7ee787b740 1 -- 172.21.15.22:0/1920205657 wait complete. 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.486+0000 7f7ee787b740 1 -- start start 2024-09-21T05:46:02.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.486+0000 7f7ee787b740 1 Processor -- start 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba3f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b638670 0x55641ba14530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b659350 0x55641ba49110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55641b6d3700 con 0x55641b630490 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55641b908340 con 0x55641b638670 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55641ba2b800 con 0x55641b659350 2024-09-21T05:46:02.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ee787b740 10 client.0 _ll_register_callbacks cb 0x55641b906ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7eda59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba3f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7eda59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba3f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49536/0 (socket says 172.21.15.22:49536) 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7eda59c640 1 -- 172.21.15.22:0/3445806150 learned_addr learned my addr 172.21.15.22:0/3445806150 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b638670 0x55641ba14530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.487+0000 7f7edad9d640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b659350 0x55641ba49110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eda59c640 1 -- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b659350 msgr2=0x55641ba49110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eda59c640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b659350 0x55641ba49110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eda59c640 1 -- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b638670 msgr2=0x55641ba14530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eda59c640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b638670 0x55641ba14530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eda59c640 1 -- 172.21.15.22:0/3445806150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641b73d7c0 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7edad9d640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b659350 0x55641ba49110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eda59c640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba3f6b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc012bf0 tx=0x7f7ecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca7c5da42e8c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eb97fa640 1 -- 172.21.15.22:0/3445806150 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc029470 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eb97fa640 1 -- 172.21.15.22:0/3445806150 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f7ecc029610 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec40668b0 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eb97fa640 1 -- 172.21.15.22:0/3445806150 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f7ecc029810 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7ee787b740 10 client.7329 fetch_fsmap learned FSMap version 75 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55641b79df50 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.488+0000 7f7eb97fa640 1 -- 172.21.15.22:0/3445806150 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ec40668b0 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7eb97fa640 1 -- 172.21.15.22:0/3445806150 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f7ecc034080 con 0x55641b630490 2024-09-21T05:46:02.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ee787b740 10 client.7329 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ee787b740 10 client.7329 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ee787b740 20 client.7329 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ee787b740 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b618130 0x55641ba18500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ee787b740 4 client.7329 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55641b9758f0 con 0x55641b618130 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b618130 0x55641ba18500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7eb97fa640 10 client.7329 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.489+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b618130 0x55641ba18500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77717fa71b231e09 server_cookie=2161cc0cb1b006a7 in_seq=0 out_seq=0 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7eb97fa640 1 -- 172.21.15.22:0/3445806150 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55641b9758f0 con 0x55641b618130 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7eb97fa640 10 client.7329 handle_command_reply: tid=0 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 client.7329 shutdown 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 msgr2=0x55641ba3f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba3f6b0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc012bf0 tx=0x7f7ecc004a40 comp rx=0 tx=0).stop 2024-09-21T05:46:02.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 shutdown_connections 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b618130 0x55641ba18500 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b659350 0x55641ba49110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b638670 0x55641ba14530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7eb97fa640 0 client.7329 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 --2- 172.21.15.22:0/3445806150 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b630490 0x55641ba3f6b0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 >> 172.21.15.22:0/3445806150 conn(0x55641b64d4a0 msgr2=0x55641ba00ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 shutdown_connections 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 1 -- 172.21.15.22:0/3445806150 wait complete. 2024-09-21T05:46:02.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.490+0000 7f7ee787b740 20 client.7329 trim_cache size 0 max 16384 2024-09-21T05:46:02.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.504+0000 7f7ee787b740 1 Processor -- start 2024-09-21T05:46:02.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.505+0000 7f7ee787b740 1 -- start start 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 0x55641b9fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b630490 0x55641b9fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba3ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ee787b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1ff1d0 con 0x55641b618130 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc1f3cd0 con 0x55641b630490 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc1fed00 con 0x55641b638670 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 0x55641b9fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 0x55641b9fb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49548/0 (socket says 172.21.15.22:49548) 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 -- 172.21.15.22:0/4135110832 learned_addr learned my addr 172.21.15.22:0/4135110832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7edad9d640 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba3ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b630490 0x55641b9fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 -- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 msgr2=0x55641ba3ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba3ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 -- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b630490 msgr2=0x55641b9fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b630490 0x55641b9fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 -- 172.21.15.22:0/4135110832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55641b6a2660 con 0x55641b618130 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.506+0000 7f7eda59c640 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 0x55641b9fb630 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc010bd0 tx=0x7f7ecc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76aada9dbb43cda4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ebb7fe640 1 -- 172.21.15.22:0/4135110832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc041070 con 0x55641b618130 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ebb7fe640 1 -- 172.21.15.22:0/4135110832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ecc024070 con 0x55641b618130 2024-09-21T05:46:02.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ebb7fe640 1 -- 172.21.15.22:0/4135110832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc038070 con 0x55641b618130 2024-09-21T05:46:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 -- 172.21.15.22:0/4135110832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 msgr2=0x55641b9fb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 0x55641b9fb630 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc010bd0 tx=0x7f7ecc015000 comp rx=0 tx=0).stop 2024-09-21T05:46:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 -- 172.21.15.22:0/4135110832 shutdown_connections 2024-09-21T05:46:02.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b638670 0x55641ba3ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b630490 0x55641b9fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 --2- 172.21.15.22:0/4135110832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b618130 0x55641b9fb630 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.507+0000 7f7ee787b740 1 -- 172.21.15.22:0/4135110832 >> 172.21.15.22:0/4135110832 conn(0x55641b64d4a0 msgr2=0x55641ba189e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.508+0000 7f7ee787b740 1 -- 172.21.15.22:0/4135110832 shutdown_connections 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.508+0000 7f7ee787b740 1 -- 172.21.15.22:0/4135110832 wait complete. 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.508+0000 7f7ee787b740 1 -- start start 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.508+0000 7f7ee787b740 1 Processor -- start 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b618130 0x55641bac4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b630490 0x55641b9fb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 0x55641b9fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55641ba3fd80 con 0x55641b638670 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55641ba556c0 con 0x55641b618130 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55641ba15a70 con 0x55641b630490 2024-09-21T05:46:02.516 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ee787b740 10 client.0 _ll_register_callbacks cb 0x7f7ecc009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 0x55641b9fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 0x55641b9fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49554/0 (socket says 172.21.15.22:49554) 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 -- 172.21.15.22:0/411140224 learned_addr learned my addr 172.21.15.22:0/411140224 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7eda59c640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b618130 0x55641bac4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7ed9d9b640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b630490 0x55641b9fb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 -- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b630490 msgr2=0x55641b9fb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b630490 0x55641b9fb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 -- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b618130 msgr2=0x55641bac4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b618130 0x55641bac4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.510+0000 7f7edad9d640 1 -- 172.21.15.22:0/411140224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641ba40080 con 0x55641b638670 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7edad9d640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 0x55641b9fbaf0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40099f0 tx=0x7f7ed4005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7360a567ce95c6a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eb97fa640 1 -- 172.21.15.22:0/411140224 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4035040 con 0x55641b638670 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eb97fa640 1 -- 172.21.15.22:0/411140224 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f7ed4004450 con 0x55641b638670 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ecc024070 con 0x55641b638670 2024-09-21T05:46:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eb97fa640 1 -- 172.21.15.22:0/411140224 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f7ed404e050 con 0x55641b638670 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 10 client.7335 fetch_fsmap learned FSMap version 75 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) -- 0x55641ba440a0 con 0x55641b638670 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eb97fa640 1 -- 172.21.15.22:0/411140224 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ecc024070 con 0x55641b638670 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eb97fa640 1 -- 172.21.15.22:0/411140224 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 75) ==== 3502+0+0 (secure 0 0 0) 0x7f7ed4051020 con 0x55641b638670 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 10 client.7335 fetch_fsmap finished waiting for FSMap version 75 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 10 client.7335 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 20 client.7335 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b5ed570 0x55641ba3f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 4 client.7335 mds_command: new command op to 7167 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 --> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55641ba47340 con 0x55641b5ed570 2024-09-21T05:46:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.511+0000 7f7eda59c640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b5ed570 0x55641ba3f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7eb97fa640 10 client.7335 ms_handle_connect on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7eda59c640 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b5ed570 0x55641ba3f430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb66350878b17b5f server_cookie=7dec961477322c3 in_seq=0 out_seq=0 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7eb97fa640 1 -- 172.21.15.22:0/411140224 <== mds.0 v2:172.21.15.22:6832/43392257 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55641ba47340 con 0x55641b5ed570 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7eb97fa640 10 client.7335 handle_command_reply: tid=0 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 client.7335 shutdown 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 msgr2=0x55641b9fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 0x55641b9fbaf0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40099f0 tx=0x7f7ed4005610 comp rx=0 tx=0).stop 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 shutdown_connections 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:6832/43392257,v1:172.21.15.22:6834/43392257] conn(0x55641b5ed570 0x55641ba3f430 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55641b638670 0x55641b9fbaf0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7eb97fa640 0 client.7335 ms_handle_reset on v2:172.21.15.22:6832/43392257 2024-09-21T05:46:02.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55641b630490 0x55641b9fb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 --2- 172.21.15.22:0/411140224 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55641b618130 0x55641bac4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 >> 172.21.15.22:0/411140224 conn(0x55641b64d4a0 msgr2=0x55641ba14f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 shutdown_connections 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.512+0000 7f7ee787b740 1 -- 172.21.15.22:0/411140224 wait complete. 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee787b740 20 client.7335 trim_cache size 0 max 16384 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:02.520 INFO:teuthology.orchestra.run.smithi022.stdout:[] 2024-09-21T05:46:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7ebc078b90 msgr2=0x7f7ebc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7ebc078b90 0x7f7ebc07b0a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc005fd0 tx=0x7f7ecc0095e0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 msgr2=0x7f7edc1e65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 0x7f7edc1e65e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c340 tx=0x7f7ec400c800 comp rx=0 tx=0).stop 2024-09-21T05:46:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 shutdown_connections 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7edc1f6970 0x7f7edc1e6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7edc075fa0 0x7f7edc1e65e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7ebc078b90 0x7f7ebc07b0a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 --2- 172.21.15.22:0/1035393251 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f7edc06e100 0x7f7edc0ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 >> 172.21.15.22:0/1035393251 conn(0x7f7edc0cd530 msgr2=0x7f7edc1555a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 shutdown_connections 2024-09-21T05:46:02.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.513+0000 7f7ee0830640 1 -- 172.21.15.22:0/1035393251 wait complete. 2024-09-21T05:46:02.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 -- 172.21.15.22:0/1358641858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cf440 msgr2=0x7fb64c0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 --2- 172.21.15.22:0/1358641858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cf440 0x7fb64c0d0190 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fb644005fd0 tx=0x7fb64402aa10 comp rx=0 tx=0).stop 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 -- 172.21.15.22:0/1358641858 shutdown_connections 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 --2- 172.21.15.22:0/1358641858 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cf440 0x7fb64c0d0190 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 --2- 172.21.15.22:0/1358641858 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0cd360 0x7fb64c15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 -- 172.21.15.22:0/1358641858 >> 172.21.15.22:0/1358641858 conn(0x7fb64c0710b0 msgr2=0x7fb64c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 -- 172.21.15.22:0/1358641858 shutdown_connections 2024-09-21T05:46:02.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.639+0000 7fb653ef9640 1 -- 172.21.15.22:0/1358641858 wait complete. 2024-09-21T05:46:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.641+0000 7fb653ef9640 1 Processor -- start 2024-09-21T05:46:02.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.641+0000 7fb653ef9640 1 -- start start 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b7fe0 0x7fb64c14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 0x7fb64c0a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c0ca870 0x7fb64c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0d0b70 con 0x7fb64c0cd360 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb64c0d09f0 con 0x7fb64c0ca870 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb64c0d0cf0 con 0x7fb64c0b7fe0 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 0x7fb64c0a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 0x7fb64c0a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49578/0 (socket says 172.21.15.22:49578) 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 -- 172.21.15.22:0/2900258147 learned_addr learned my addr 172.21.15.22:0/2900258147 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651c95640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b7fe0 0x7fb64c14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb652496640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c0ca870 0x7fb64c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 -- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b7fe0 msgr2=0x7fb64c14c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b7fe0 0x7fb64c14c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 -- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c0ca870 msgr2=0x7fb64c0cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c0ca870 0x7fb64c0cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 -- 172.21.15.22:0/2900258147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64c0cb1c0 con 0x7fb64c0cd360 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb652496640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c0ca870 0x7fb64c0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651c95640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b7fe0 0x7fb64c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb651494640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 0x7fb64c0a94a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c3d0 tx=0x7fb63c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c017070 con 0x7fb64c0cd360 2024-09-21T05:46:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb63c0053c0 con 0x7fb64c0cd360 2024-09-21T05:46:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64c14d2d0 con 0x7fb64c0cd360 2024-09-21T05:46:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.643+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c003850 con 0x7fb64c0cd360 2024-09-21T05:46:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.644+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64c14d890 con 0x7fb64c0cd360 2024-09-21T05:46:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.644+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb64c14d040 con 0x7fb64c0cd360 2024-09-21T05:46:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.645+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb63c01a070 con 0x7fb64c0cd360 2024-09-21T05:46:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.646+0000 7fb6427fc640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb630078b90 0x7fb63007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.646+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fb63c09a1d0 con 0x7fb64c0cd360 2024-09-21T05:46:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.646+0000 7fb651c95640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb630078b90 0x7fb63007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.647+0000 7fb651c95640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb630078b90 0x7fb63007b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb648005fd0 tx=0x7fb64800f360 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.650+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb64c0cb1c0 con 0x7fb64c0cd360 2024-09-21T05:46:02.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.876+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb64c0d1210 con 0x7fb64c0cd360 2024-09-21T05:46:02.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.876+0000 7fb6427fc640 1 -- 172.21.15.22:0/2900258147 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb63c062540 con 0x7fb64c0cd360 2024-09-21T05:46:02.879 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:02.879 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T05:46:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb630078b90 msgr2=0x7fb63007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb630078b90 0x7fb63007b0a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb648005fd0 tx=0x7fb64800f360 comp rx=0 tx=0).stop 2024-09-21T05:46:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 msgr2=0x7fb64c0a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 0x7fb64c0a94a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c3d0 tx=0x7fb63c00c890 comp rx=0 tx=0).stop 2024-09-21T05:46:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 shutdown_connections 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb64c0ca870 0x7fb64c0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb64c0cd360 0x7fb64c0a94a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb630078b90 0x7fb63007b0a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 --2- 172.21.15.22:0/2900258147 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb64c0b7fe0 0x7fb64c14c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 >> 172.21.15.22:0/2900258147 conn(0x7fb64c0710b0 msgr2=0x7fb64c0b2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 shutdown_connections 2024-09-21T05:46:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:02.879+0000 7fb653ef9640 1 -- 172.21.15.22:0/2900258147 wait complete. 2024-09-21T05:46:02.892 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T05:46:02.892 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:46:02.892 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:46:02.941 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:46:02.992 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:46:02.992 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:46:02.992 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:02.992 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:03.009 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:03.009 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:03.064 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:46:03.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 -- 172.21.15.22:0/241538674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f181c005350 msgr2=0x7f181c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 --2- 172.21.15.22:0/241538674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f181c005350 0x7f181c00b9c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f1828005fd0 tx=0x7f181c005830 comp rx=0 tx=0).stop 2024-09-21T05:46:03.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 -- 172.21.15.22:0/241538674 shutdown_connections 2024-09-21T05:46:03.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 --2- 172.21.15.22:0/241538674 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f181c005350 0x7f181c00b9c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 -- 172.21.15.22:0/241538674 >> 172.21.15.22:0/241538674 conn(0x7f183008bfc0 msgr2=0x7f183009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:03.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 -- 172.21.15.22:0/241538674 shutdown_connections 2024-09-21T05:46:03.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.220+0000 7f1838719640 1 -- 172.21.15.22:0/241538674 wait complete. 2024-09-21T05:46:03.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.222+0000 7f1838719640 1 Processor -- start 2024-09-21T05:46:03.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.222+0000 7f1838719640 1 -- start start 2024-09-21T05:46:03.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1838719640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181c005350 0x7f18300c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1838719640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18300cb5c0 0x7f183015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1838719640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 0x7f183015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1838719640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f18300cef50 con 0x7f183015b960 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1838719640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f183006e3d0 con 0x7f181c005350 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1838719640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f18300cf0d0 con 0x7f18300cb5c0 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1836cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 0x7f183015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1836cb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 0x7f183015d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49608/0 (socket says 172.21.15.22:49608) 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.223+0000 7f1836cb6640 1 -- 172.21.15.22:0/1648111896 learned_addr learned my addr 172.21.15.22:0/1648111896 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f18364b5640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181c005350 0x7f18300c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1835cb4640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18300cb5c0 0x7f183015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1836cb6640 1 -- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18300cb5c0 msgr2=0x7f183015ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1836cb6640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18300cb5c0 0x7f183015ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1836cb6640 1 -- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181c005350 msgr2=0x7f18300c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1836cb6640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181c005350 0x7f18300c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1836cb6640 1 -- 172.21.15.22:0/1648111896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18300c9170 con 0x7f183015b960 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1836cb6640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 0x7f183015d910 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f182802aec0 tx=0x7f1828004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828009200 con 0x7f183015b960 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f182802bc30 con 0x7f183015b960 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f183015d150 con 0x7f183015b960 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828009ad0 con 0x7f183015b960 2024-09-21T05:46:03.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.224+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f183015f700 con 0x7f183015b960 2024-09-21T05:46:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.225+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f183006f670 con 0x7f183015b960 2024-09-21T05:46:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.226+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1828033440 con 0x7f183015b960 2024-09-21T05:46:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.226+0000 7f1826ffd640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1814078b90 0x7f181407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.227+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f18280bb510 con 0x7f183015b960 2024-09-21T05:46:03.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.227+0000 7f18364b5640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1814078b90 0x7f181407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.227+0000 7f18364b5640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1814078b90 0x7f181407b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f182c006e40 tx=0x7f182c006090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:03.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.231+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18280837e0 con 0x7f183015b960 2024-09-21T05:46:03.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.429+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f183015fe40 con 0x7f183015b960 2024-09-21T05:46:03.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.429+0000 7f1826ffd640 1 -- 172.21.15.22:0/1648111896 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v86) ==== 80+0+246 (secure 0 0 0) 0x7f18280830e0 con 0x7f183015b960 2024-09-21T05:46:03.431 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6832/3207626530 2024-09-22T05:45:24.802278+0000 2024-09-21T05:46:03.432 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6837/2104741994 2024-09-22T05:44:14.100657+0000 2024-09-21T05:46:03.432 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6834/3207626530 2024-09-22T05:45:24.802278+0000 2024-09-21T05:46:03.432 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6835/2104741994 2024-09-22T05:44:14.100657+0000 2024-09-21T05:46:03.432 INFO:teuthology.orchestra.run.smithi022.stderr:listed 4 entries 2024-09-21T05:46:03.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1814078b90 msgr2=0x7f181407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1814078b90 0x7f181407b0a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f182c006e40 tx=0x7f182c006090 comp rx=0 tx=0).stop 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 msgr2=0x7f183015d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 0x7f183015d910 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f182802aec0 tx=0x7f1828004270 comp rx=0 tx=0).stop 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 shutdown_connections 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f183015b960 0x7f183015d910 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1814078b90 0x7f181407b0a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f18300cb5c0 0x7f183015ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 --2- 172.21.15.22:0/1648111896 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f181c005350 0x7f18300c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 >> 172.21.15.22:0/1648111896 conn(0x7f183008bfc0 msgr2=0x7f18300b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 shutdown_connections 2024-09-21T05:46:03.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.431+0000 7f1838719640 1 -- 172.21.15.22:0/1648111896 wait complete. 2024-09-21T05:46:03.444 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:46:03.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:03.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:03.473 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:46:03.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:03.474 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:46:03.530 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:46:03.530 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:46:03.530 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:03.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:46:03.549 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:46:03.549 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:03.549 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:03.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:46:03.586 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:46:03.586 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:46:03.586 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:03.607 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:46:03.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 -- 172.21.15.22:0/2299594289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5ec004800 msgr2=0x7fa5ec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 --2- 172.21.15.22:0/2299594289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5ec004800 0x7fa5ec004c10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8009b50 tx=0x7fa5ec005830 comp rx=0 tx=0).stop 2024-09-21T05:46:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 -- 172.21.15.22:0/2299594289 shutdown_connections 2024-09-21T05:46:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 --2- 172.21.15.22:0/2299594289 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa5ec004800 0x7fa5ec004c10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 -- 172.21.15.22:0/2299594289 >> 172.21.15.22:0/2299594289 conn(0x7fa60406d4f0 msgr2=0x7fa604071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 -- 172.21.15.22:0/2299594289 shutdown_connections 2024-09-21T05:46:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.741+0000 7fa60b407640 1 -- 172.21.15.22:0/2299594289 wait complete. 2024-09-21T05:46:03.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.743+0000 7fa60b407640 1 Processor -- start 2024-09-21T05:46:03.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.743+0000 7fa60b407640 1 -- start start 2024-09-21T05:46:03.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa60b407640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5ec004800 0x7fa60414b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa60b407640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6040b1150 0x7fa6040b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa60b407640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 0x7fa6041f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa60b407640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6040cec10 con 0x7fa6041f3550 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa60b407640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6040cd040 con 0x7fa5ec004800 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa60b407640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6040ced90 con 0x7fa6040b1150 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 0x7fa6041f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 0x7fa6041f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49616/0 (socket says 172.21.15.22:49616) 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 -- 172.21.15.22:0/553059332 learned_addr learned my addr 172.21.15.22:0/553059332 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6089a2640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6040b1150 0x7fa6040b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6091a3640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5ec004800 0x7fa60414b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 -- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6040b1150 msgr2=0x7fa6040b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6040b1150 0x7fa6040b1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 -- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5ec004800 msgr2=0x7fa60414b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5ec004800 0x7fa60414b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 -- 172.21.15.22:0/553059332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60414e1d0 con 0x7fa6041f3550 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6091a3640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5ec004800 0x7fa60414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.744+0000 7fa6099a4640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 0x7fa6041f3960 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40013d0 tx=0x7fa5f4006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.745+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f400fa20 con 0x7fa6041f3550 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.745+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5f4018070 con 0x7fa6041f3550 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.745+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f400fe80 con 0x7fa6041f3550 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.745+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60415d650 con 0x7fa6041f3550 2024-09-21T05:46:03.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.745+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60415dc10 con 0x7fa6041f3550 2024-09-21T05:46:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.746+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60414b010 con 0x7fa6041f3550 2024-09-21T05:46:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.747+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa5f401d460 con 0x7fa6041f3550 2024-09-21T05:46:03.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.747+0000 7fa601ffb640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5f0078bc0 0x7fa5f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.747+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fa5f409b690 con 0x7fa6041f3550 2024-09-21T05:46:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.747+0000 7fa6091a3640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5f0078bc0 0x7fa5f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:03.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.748+0000 7fa6091a3640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5f0078bc0 0x7fa5f007b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc006990 tx=0x7fa5fc006ac0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:03.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.752+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa60414b010 con 0x7fa6041f3550 2024-09-21T05:46:03.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.978+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6040b3d50 con 0x7fa6041f3550 2024-09-21T05:46:03.980 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.978+0000 7fa601ffb640 1 -- 172.21.15.22:0/553059332 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5220 (secure 0 0 0) 0x7fa5f4063ac0 con 0x7fa6041f3550 2024-09-21T05:46:03.980 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:03.980 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":75,"btime":"2024-09-21T05:45:28:820289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:28.820279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-21T05:46:03.981 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 75 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5f0078bc0 msgr2=0x7fa5f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5f0078bc0 0x7fa5f007b0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc006990 tx=0x7fa5fc006ac0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 msgr2=0x7fa6041f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 0x7fa6041f3960 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40013d0 tx=0x7fa5f4006720 comp rx=0 tx=0).stop 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 shutdown_connections 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6041f3550 0x7fa6041f3960 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa5f0078bc0 0x7fa5f007b0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6040b1150 0x7fa6040b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 --2- 172.21.15.22:0/553059332 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa5ec004800 0x7fa60414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:03.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 >> 172.21.15.22:0/553059332 conn(0x7fa60406d4f0 msgr2=0x7fa6040cb390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:03.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 shutdown_connections 2024-09-21T05:46:03.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:03.981+0000 7fa60b407640 1 -- 172.21.15.22:0/553059332 wait complete. 2024-09-21T05:46:03.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6ca5084640 1 -- 172.21.15.22:0/1382666965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c8c004d40 msgr2=0x7f6c8c0051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1382666965 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c8c004d40 0x7f6c8c0051f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6ca00d3440 tx=0x7f6c8c005830 comp rx=0 tx=0).stop 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6c9f59e640 1 -- 172.21.15.22:0/1382666965 reap_dead start 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6ca5084640 1 -- 172.21.15.22:0/1382666965 shutdown_connections 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6ca5084640 1 -- 172.21.15.22:0/1382666965 >> 172.21.15.22:0/1382666965 conn(0x7f6ca00b3f10 msgr2=0x7f6ca00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6ca5084640 1 -- 172.21.15.22:0/1382666965 shutdown_connections 2024-09-21T05:46:04.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.108+0000 7f6ca5084640 1 -- 172.21.15.22:0/1382666965 wait complete. 2024-09-21T05:46:04.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.111+0000 7f6ca5084640 1 Processor -- start 2024-09-21T05:46:04.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.111+0000 7f6ca5084640 1 -- start start 2024-09-21T05:46:04.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6ca5084640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca01f3190 0x7f6ca00a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6ca5084640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 0x7f6ca00cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6ca5084640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca00ada20 0x7f6ca00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6ca5084640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca00d0fe0 con 0x7f6ca00ad610 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6ca5084640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca00d0e60 con 0x7f6ca00ada20 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6ca5084640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca00d1160 con 0x7f6ca01f3190 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 0x7f6ca00cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 0x7f6ca00cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49630/0 (socket says 172.21.15.22:49630) 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 -- 172.21.15.22:0/1764476522 learned_addr learned my addr 172.21.15.22:0/1764476522 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9ed9d640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca01f3190 0x7f6ca00a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9f59e640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca00ada20 0x7f6ca00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 -- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca01f3190 msgr2=0x7f6ca00a94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca01f3190 0x7f6ca00a94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 -- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca00ada20 msgr2=0x7f6ca00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca00ada20 0x7f6ca00cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9e59c640 1 -- 172.21.15.22:0/1764476522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca00a99e0 con 0x7f6ca00ad610 2024-09-21T05:46:04.116 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.112+0000 7f6c9ed9d640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca01f3190 0x7f6ca00a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.113+0000 7f6c9e59c640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 0x7f6ca00cb4a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6c88003ec0 tx=0x7f6c88001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.113+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88015070 con 0x7f6ca00ad610 2024-09-21T05:46:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.113+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c88011840 con 0x7f6ca00ad610 2024-09-21T05:46:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.113+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca00a9cb0 con 0x7f6ca00ad610 2024-09-21T05:46:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.113+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88011cb0 con 0x7f6ca00ad610 2024-09-21T05:46:04.117 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.113+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca00aa270 con 0x7f6ca00ad610 2024-09-21T05:46:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.114+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca00d16c0 con 0x7f6ca00ad610 2024-09-21T05:46:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.115+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c88018070 con 0x7f6ca00ad610 2024-09-21T05:46:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.115+0000 7f6c977fe640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c80078bc0 0x7f6c8007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.115+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f6c8809b860 con 0x7f6ca00ad610 2024-09-21T05:46:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.116+0000 7f6c9ed9d640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c80078bc0 0x7f6c8007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.116+0000 7f6c9ed9d640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c80078bc0 0x7f6c8007b0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c90005fd0 tx=0x7f6c90009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:04.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.120+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c88063cf0 con 0x7f6ca00ad610 2024-09-21T05:46:04.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.344+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ca015c340 con 0x7f6ca00ad610 2024-09-21T05:46:04.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.344+0000 7f6c977fe640 1 -- 172.21.15.22:0/1764476522 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5220 (secure 0 0 0) 0x7f6c880635f0 con 0x7f6ca00ad610 2024-09-21T05:46:04.347 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:04.347 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":75,"btime":"2024-09-21T05:45:28:820289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:28.820279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-21T05:46:04.347 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 75 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c80078bc0 msgr2=0x7f6c8007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c80078bc0 0x7f6c8007b0d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c90005fd0 tx=0x7f6c90009500 comp rx=0 tx=0).stop 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 msgr2=0x7f6ca00cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 0x7f6ca00cb4a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6c88003ec0 tx=0x7f6c88001630 comp rx=0 tx=0).stop 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 shutdown_connections 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ca00ada20 0x7f6ca00cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ca00ad610 0x7f6ca00cb4a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c80078bc0 0x7f6c8007b0d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 --2- 172.21.15.22:0/1764476522 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ca01f3190 0x7f6ca00a94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 >> 172.21.15.22:0/1764476522 conn(0x7f6ca00b3f10 msgr2=0x7f6ca015f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:04.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 shutdown_connections 2024-09-21T05:46:04.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.347+0000 7f6ca5084640 1 -- 172.21.15.22:0/1764476522 wait complete. 2024-09-21T05:46:04.361 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:46:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 -- 172.21.15.22:0/855265481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15fbf0 msgr2=0x7f8e9c0ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 --2- 172.21.15.22:0/855265481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c0ce360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c005fd0 tx=0x7f8e8c02a8b0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 -- 172.21.15.22:0/855265481 shutdown_connections 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 --2- 172.21.15.22:0/855265481 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e9c15f6b0 0x7f8e9c06f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 --2- 172.21.15.22:0/855265481 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c0ce360 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 -- 172.21.15.22:0/855265481 >> 172.21.15.22:0/855265481 conn(0x7f8e9c0b2280 msgr2=0x7f8e9c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 -- 172.21.15.22:0/855265481 shutdown_connections 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.476+0000 7f8ea099d640 1 -- 172.21.15.22:0/855265481 wait complete. 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.478+0000 7f8ea099d640 1 Processor -- start 2024-09-21T05:46:04.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.478+0000 7f8ea099d640 1 -- start start 2024-09-21T05:46:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8ea099d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15b5d0 0x7f8e9c0ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8ea099d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8ea099d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e9c154d80 0x7f8e9c06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8ea099d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c0702e0 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8ea099d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c070160 con 0x7f8e9c15b5d0 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8ea099d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9c070460 con 0x7f8e9c154d80 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8e9affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8e9affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58784/0 (socket says 172.21.15.22:58784) 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8e9b7fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15b5d0 0x7f8e9c0ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8e9bfff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e9c154d80 0x7f8e9c06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.479+0000 7f8e9affd640 1 -- 172.21.15.22:0/1144280587 learned_addr learned my addr 172.21.15.22:0/1144280587 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e9affd640 1 -- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e9c154d80 msgr2=0x7f8e9c06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e9affd640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e9c154d80 0x7f8e9c06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e9affd640 1 -- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15b5d0 msgr2=0x7f8e9c0ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e9affd640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15b5d0 0x7f8e9c0ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e9affd640 1 -- 172.21.15.22:0/1144280587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c1576b0 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e9affd640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c154840 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c3a0 tx=0x7f8e8400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:04.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84017070 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e840053c0 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e84003850 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c155570 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.480+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c155b30 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.481+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c1552e0 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.482+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8e8401a070 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.482+0000 7f8e93fff640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8e7c078b90 0x7f8e7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.483+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f8e8409a130 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.483+0000 7f8e9b7fe640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8e7c078b90 0x7f8e7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.483+0000 7f8e9b7fe640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8e7c078b90 0x7f8e7c07b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c034000 tx=0x7f8e8c004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:04.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.487+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8e84061db0 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.673+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e9c070910 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.676 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.674+0000 7f8e93fff640 1 -- 172.21.15.22:0/1144280587 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+17019 (secure 0 0 0) 0x7f8e840667e0 con 0x7f8e9c15fbf0 2024-09-21T05:46:04.676 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:04.676 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:45:24.802308+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:44:02.389731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T05:44:03.675773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6832/3207626530":"2024-09-22T05:45:24.802278+0000","172.21.15.134:6837/2104741994":"2024-09-22T05:44:14.100657+0000","172.21.15.22:6834/3207626530":"2024-09-22T05:45:24.802278+0000","172.21.15.134:6835/2104741994":"2024-09-22T05:44:14.100657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:46:04.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8e7c078b90 msgr2=0x7f8e7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8e7c078b90 0x7f8e7c07b0a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c034000 tx=0x7f8e8c004ab0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 msgr2=0x7f8e9c154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c154840 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c3a0 tx=0x7f8e8400c860 comp rx=0 tx=0).stop 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 shutdown_connections 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8e7c078b90 0x7f8e7c07b0a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8e9c154d80 0x7f8e9c06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8e9c15fbf0 0x7f8e9c154840 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 --2- 172.21.15.22:0/1144280587 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8e9c15b5d0 0x7f8e9c0ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 >> 172.21.15.22:0/1144280587 conn(0x7f8e9c0b2280 msgr2=0x7f8e9c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 shutdown_connections 2024-09-21T05:46:04.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.676+0000 7f8ea099d640 1 -- 172.21.15.22:0/1144280587 wait complete. 2024-09-21T05:46:04.690 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:46:04.690 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:04.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 -- 172.21.15.22:0/3773555593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b006e0e0 msgr2=0x7f36b006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 --2- 172.21.15.22:0/3773555593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b006e0e0 0x7f36b006e4f0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f36b00d3de0 tx=0x7f36a802aa30 comp rx=0 tx=0).stop 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 -- 172.21.15.22:0/3773555593 shutdown_connections 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 --2- 172.21.15.22:0/3773555593 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b006e0e0 0x7f36b006e4f0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 --2- 172.21.15.22:0/3773555593 >> v2:172.21.15.134:3300/0 conn(0x7f36b00b66b0 0x7f36b00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 -- 172.21.15.22:0/3773555593 >> 172.21.15.22:0/3773555593 conn(0x7f36b009d7f0 msgr2=0x7f36b00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 -- 172.21.15.22:0/3773555593 shutdown_connections 2024-09-21T05:46:04.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.803+0000 7f36b7a94640 1 -- 172.21.15.22:0/3773555593 wait complete. 2024-09-21T05:46:04.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.805+0000 7f36b7a94640 1 Processor -- start 2024-09-21T05:46:04.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.805+0000 7f36b7a94640 1 -- start start 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b7a94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36b006e0e0 0x7f36b00caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b7a94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 0x7f36b0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b7a94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36b0069220 0x7f36b00b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b7a94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f36b006ea30 con 0x7f36b00b9970 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b7a94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f36b00d5010 con 0x7f36b0069220 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b7a94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f36b006ebb0 con 0x7f36b006e0e0 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b502f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 0x7f36b0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b502f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 0x7f36b0156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58802/0 (socket says 172.21.15.22:58802) 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.806+0000 7f36b502f640 1 -- 172.21.15.22:0/1341730813 learned_addr learned my addr 172.21.15.22:0/1341730813 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b6031640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36b0069220 0x7f36b00b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b5830640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36b006e0e0 0x7f36b00caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b502f640 1 -- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36b006e0e0 msgr2=0x7f36b00caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b502f640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36b006e0e0 0x7f36b00caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b502f640 1 -- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36b0069220 msgr2=0x7f36b00b86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b502f640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36b0069220 0x7f36b00b86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b502f640 1 -- 172.21.15.22:0/1341730813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b01f95b0 con 0x7f36b00b9970 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b502f640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 0x7f36b0156950 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f36a000d770 tx=0x7f36a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0017070 con 0x7f36b00b9970 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a0009d20 con 0x7f36b00b9970 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b01e40a0 con 0x7f36b00b9970 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a00106c0 con 0x7f36b00b9970 2024-09-21T05:46:04.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.807+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b01e4660 con 0x7f36b00b9970 2024-09-21T05:46:04.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.808+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b006e4f0 con 0x7f36b00b9970 2024-09-21T05:46:04.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.809+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f36a001a070 con 0x7f36b00b9970 2024-09-21T05:46:04.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.809+0000 7f36a67fc640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3694078b90 0x7f369407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:04.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.810+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f36a009ae40 con 0x7f36b00b9970 2024-09-21T05:46:04.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.810+0000 7f36b5830640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3694078b90 0x7f369407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:04.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.810+0000 7f36b5830640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3694078b90 0x7f369407b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0098c0 tx=0x7f36ac009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:04.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:04.814+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36a0062be0 con 0x7f36b00b9970 2024-09-21T05:46:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.045+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36b01f9310 con 0x7f36b00b9970 2024-09-21T05:46:05.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.045+0000 7f36a67fc640 1 -- 172.21.15.22:0/1341730813 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5220 (secure 0 0 0) 0x7f36b01f9310 con 0x7f36b00b9970 2024-09-21T05:46:05.048 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:05.048 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":75,"btime":"2024-09-21T05:45:28:820289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:28.820279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-21T05:46:05.048 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 75 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3694078b90 msgr2=0x7f369407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3694078b90 0x7f369407b0a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0098c0 tx=0x7f36ac009060 comp rx=0 tx=0).stop 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 msgr2=0x7f36b0156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 0x7f36b0156950 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f36a000d770 tx=0x7f36a000dc30 comp rx=0 tx=0).stop 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 shutdown_connections 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f36b0069220 0x7f36b00b86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f36b00b9970 0x7f36b0156950 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3694078b90 0x7f369407b0a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 --2- 172.21.15.22:0/1341730813 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f36b006e0e0 0x7f36b00caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 >> 172.21.15.22:0/1341730813 conn(0x7f36b009d7f0 msgr2=0x7f36b00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:05.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 shutdown_connections 2024-09-21T05:46:05.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.048+0000 7f36b7a94640 1 -- 172.21.15.22:0/1341730813 wait complete. 2024-09-21T05:46:05.061 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.175+0000 7f63af0cc640 1 -- 172.21.15.22:0/2719815837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6394004800 msgr2=0x7f6394004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.175+0000 7f63af0cc640 1 --2- 172.21.15.22:0/2719815837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6394004800 0x7f6394004c10 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f63900013d0 tx=0x7f6394005830 comp rx=0 tx=0).stop 2024-09-21T05:46:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.176+0000 7f63af0cc640 1 -- 172.21.15.22:0/2719815837 shutdown_connections 2024-09-21T05:46:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.176+0000 7f63af0cc640 1 --2- 172.21.15.22:0/2719815837 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6394004800 0x7f6394004c10 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.176+0000 7f63af0cc640 1 -- 172.21.15.22:0/2719815837 >> 172.21.15.22:0/2719815837 conn(0x7f63a806d4f0 msgr2=0x7f63a8071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.176+0000 7f63af0cc640 1 -- 172.21.15.22:0/2719815837 shutdown_connections 2024-09-21T05:46:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.176+0000 7f63af0cc640 1 -- 172.21.15.22:0/2719815837 wait complete. 2024-09-21T05:46:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.177+0000 7f63af0cc640 1 Processor -- start 2024-09-21T05:46:05.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.177+0000 7f63af0cc640 1 -- start start 2024-09-21T05:46:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63af0cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6394004800 0x7f63a8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63af0cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 0x7f63a80cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63af0cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63a80cb1c0 0x7f63a815d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63af0cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80cec10 con 0x7f63a80c8960 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63af0cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63a80cd040 con 0x7f6394004800 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63af0cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f63a80ced90 con 0x7f63a80cb1c0 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 0x7f63a80cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 0x7f63a80cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58830/0 (socket says 172.21.15.22:58830) 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 -- 172.21.15.22:0/1625319096 learned_addr learned my addr 172.21.15.22:0/1625319096 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63ace68640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6394004800 0x7f63a8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63ad669640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63a80cb1c0 0x7f63a815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 -- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63a80cb1c0 msgr2=0x7f63a815d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63a80cb1c0 0x7f63a815d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 -- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6394004800 msgr2=0x7f63a8158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6394004800 0x7f63a8158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63a7fff640 1 -- 172.21.15.22:0/1625319096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a80cbc20 con 0x7f63a80c8960 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.179+0000 7f63ace68640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6394004800 0x7f63a8158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.180+0000 7f63a7fff640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 0x7f63a80cac80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f6390000f80 tx=0x7f6390004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.180+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390039070 con 0x7f63a80c8960 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.180+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f639002bca0 con 0x7f63a80c8960 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.180+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a80cbf00 con 0x7f63a80c8960 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.180+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390006290 con 0x7f63a80c8960 2024-09-21T05:46:05.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.180+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a80cc4c0 con 0x7f63a80c8960 2024-09-21T05:46:05.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.181+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a80b3d50 con 0x7f63a80c8960 2024-09-21T05:46:05.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.181+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6390006430 con 0x7f63a80c8960 2024-09-21T05:46:05.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.182+0000 7f63a57fa640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6388078b90 0x7f638807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.182+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f63900baea0 con 0x7f63a80c8960 2024-09-21T05:46:05.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.182+0000 7f63ace68640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6388078b90 0x7f638807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.185 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.183+0000 7f63ace68640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6388078b90 0x7f638807b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63a8052910 tx=0x7f6398005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:05.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.186+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6390004e20 con 0x7f63a80c8960 2024-09-21T05:46:05.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.414+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63a80b76d0 con 0x7f63a80c8960 2024-09-21T05:46:05.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.414+0000 7f63a57fa640 1 -- 172.21.15.22:0/1625319096 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5220 (secure 0 0 0) 0x7f6390083190 con 0x7f63a80c8960 2024-09-21T05:46:05.416 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:05.417 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":75,"btime":"2024-09-21T05:45:28:820289+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:44:04.719803+0000","modified":"2024-09-21T05:45:28.820279+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7167},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7167":{"gid":7167,"name":"c","rank":0,"incarnation":72,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/43392257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":43392257},{"type":"v1","addr":"172.21.15.22:6834","nonce":43392257}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7167,"qdb_cluster":[7167]},"id":6}]} 2024-09-21T05:46:05.417 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 75 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6388078b90 msgr2=0x7f638807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6388078b90 0x7f638807b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f63a8052910 tx=0x7f6398005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 msgr2=0x7f63a80cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 0x7f63a80cac80 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f6390000f80 tx=0x7f6390004c30 comp rx=0 tx=0).stop 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 shutdown_connections 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6388078b90 0x7f638807b0a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f63a80cb1c0 0x7f63a815d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f63a80c8960 0x7f63a80cac80 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 --2- 172.21.15.22:0/1625319096 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6394004800 0x7f63a8158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 >> 172.21.15.22:0/1625319096 conn(0x7f63a806d4f0 msgr2=0x7f63a80b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.416+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 shutdown_connections 2024-09-21T05:46:05.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.417+0000 7f63af0cc640 1 -- 172.21.15.22:0/1625319096 wait complete. 2024-09-21T05:46:05.430 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 -- 172.21.15.22:0/1176731863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006e170 msgr2=0x7fa8900b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 --2- 172.21.15.22:0/1176731863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006e170 0x7fa8900b3540 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa8900ba090 tx=0x7fa88002a870 comp rx=0 tx=0).stop 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 -- 172.21.15.22:0/1176731863 shutdown_connections 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 --2- 172.21.15.22:0/1176731863 >> v2:172.21.15.134:3300/0 conn(0x7fa8900ceb40 0x7fa8900cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 --2- 172.21.15.22:0/1176731863 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006e170 0x7fa8900b3540 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 -- 172.21.15.22:0/1176731863 >> 172.21.15.22:0/1176731863 conn(0x7fa89008bfc0 msgr2=0x7fa89009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 -- 172.21.15.22:0/1176731863 shutdown_connections 2024-09-21T05:46:05.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.544+0000 7fa894cb4640 1 -- 172.21.15.22:0/1176731863 wait complete. 2024-09-21T05:46:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.546+0000 7fa894cb4640 1 Processor -- start 2024-09-21T05:46:05.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.546+0000 7fa894cb4640 1 -- start start 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 0x7fa890156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8900ceb40 0x7fa8900b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa89015d480 0x7fa8900b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa8900d0e60 con 0x7fa89006f4b0 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8900d0ce0 con 0x7fa8900ceb40 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa8900d0fe0 con 0x7fa89015d480 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 0x7fa890156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 0x7fa890156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58852/0 (socket says 172.21.15.22:58852) 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 -- 172.21.15.22:0/4226389684 learned_addr learned my addr 172.21.15.22:0/4226389684 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:05.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88ed9d640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa89015d480 0x7fa8900b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88dd9b640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8900ceb40 0x7fa8900b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 -- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa89015d480 msgr2=0x7fa8900b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa89015d480 0x7fa8900b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 -- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8900ceb40 msgr2=0x7fa8900b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8900ceb40 0x7fa8900b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 -- 172.21.15.22:0/4226389684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8900ce0a0 con 0x7fa89006f4b0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa88e59c640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 0x7fa890156ee0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa880006010 tx=0x7fa880033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880042020 con 0x7fa89006f4b0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa880004040 con 0x7fa89006f4b0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8900ce380 con 0x7fa89006f4b0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.548+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa880037040 con 0x7fa89006f4b0 2024-09-21T05:46:05.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.549+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8900c97d0 con 0x7fa89006f4b0 2024-09-21T05:46:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.550+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8900ce8a0 con 0x7fa89006f4b0 2024-09-21T05:46:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.550+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa880045020 con 0x7fa89006f4b0 2024-09-21T05:46:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.551+0000 7fa87effd640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078b90 0x7fa87007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.551+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7fa8800bbdf0 con 0x7fa89006f4b0 2024-09-21T05:46:05.553 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.551+0000 7fa88dd9b640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078b90 0x7fa87007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.552+0000 7fa88dd9b640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078b90 0x7fa87007b0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa8900caff0 tx=0x7fa884009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:05.558 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.556+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8900ce8a0 con 0x7fa89006f4b0 2024-09-21T05:46:05.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.744+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8900d15a0 con 0x7fa89006f4b0 2024-09-21T05:46:05.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.745+0000 7fa87effd640 1 -- 172.21.15.22:0/4226389684 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+17019 (secure 0 0 0) 0x7fa88004c030 con 0x7fa89006f4b0 2024-09-21T05:46:05.747 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:05.747 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":86,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:45:24.802308+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:44:02.389731+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T05:44:03.675773+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.11","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6832/3207626530":"2024-09-22T05:45:24.802278+0000","172.21.15.134:6837/2104741994":"2024-09-22T05:44:14.100657+0000","172.21.15.22:6834/3207626530":"2024-09-22T05:45:24.802278+0000","172.21.15.134:6835/2104741994":"2024-09-22T05:44:14.100657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:46:05.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078b90 msgr2=0x7fa87007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078b90 0x7fa87007b0a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa8900caff0 tx=0x7fa884009040 comp rx=0 tx=0).stop 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 msgr2=0x7fa890156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 0x7fa890156ee0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa880006010 tx=0x7fa880033ad0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 shutdown_connections 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078b90 0x7fa87007b0a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa89015d480 0x7fa8900b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8900ceb40 0x7fa8900b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 --2- 172.21.15.22:0/4226389684 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa89006f4b0 0x7fa890156ee0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 >> 172.21.15.22:0/4226389684 conn(0x7fa89008bfc0 msgr2=0x7fa89006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 shutdown_connections 2024-09-21T05:46:05.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.747+0000 7fa894cb4640 1 -- 172.21.15.22:0/4226389684 wait complete. 2024-09-21T05:46:05.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 -- 172.21.15.22:0/138259594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 msgr2=0x7f86480d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 --2- 172.21.15.22:0/138259594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f86480d0d80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f86480cd8d0 tx=0x7f863c02a560 comp rx=0 tx=0).stop 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 -- 172.21.15.22:0/138259594 shutdown_connections 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 --2- 172.21.15.22:0/138259594 >> v2:172.21.15.134:3300/0 conn(0x7f86480d12c0 0x7f86480d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 --2- 172.21.15.22:0/138259594 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f86480d0d80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 -- 172.21.15.22:0/138259594 >> 172.21.15.22:0/138259594 conn(0x7f8648072ae0 msgr2=0x7f8648072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:05.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 -- 172.21.15.22:0/138259594 shutdown_connections 2024-09-21T05:46:05.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.875+0000 7f864e869640 1 -- 172.21.15.22:0/138259594 wait complete. 2024-09-21T05:46:05.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.877+0000 7f864e869640 1 Processor -- start 2024-09-21T05:46:05.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.877+0000 7f864e869640 1 -- start start 2024-09-21T05:46:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864e869640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86480b51c0 0x7f86480c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864e869640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f864815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864e869640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86480d12c0 0x7f8648158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864e869640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f86480cf220 con 0x7f86480ccbd0 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864e869640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f86480cf0a0 con 0x7f86480b51c0 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864e869640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f86480cf3a0 con 0x7f86480d12c0 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f864815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f864815f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58872/0 (socket says 172.21.15.22:58872) 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 -- 172.21.15.22:0/1057528276 learned_addr learned my addr 172.21.15.22:0/1057528276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f8647fff640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86480b51c0 0x7f86480c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864ce06640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86480d12c0 0x7f8648158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 -- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86480d12c0 msgr2=0x7f8648158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86480d12c0 0x7f8648158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 -- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86480b51c0 msgr2=0x7f86480c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86480b51c0 0x7f86480c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 -- 172.21.15.22:0/1057528276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86481f6db0 con 0x7f86480ccbd0 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f864ce06640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86480d12c0 0x7f8648158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.879+0000 7f86477fe640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f864815f4c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f863c0099c0 tx=0x7f863c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:05.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.880+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c03e030 con 0x7f86480ccbd0 2024-09-21T05:46:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.880+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f863c0047a0 con 0x7f86480ccbd0 2024-09-21T05:46:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.880+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86481f7090 con 0x7f86480ccbd0 2024-09-21T05:46:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.880+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c035040 con 0x7f86480ccbd0 2024-09-21T05:46:05.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.880+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86481f7650 con 0x7f86480ccbd0 2024-09-21T05:46:05.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.881+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8648069290 con 0x7f86480ccbd0 2024-09-21T05:46:05.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.882+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f863c041030 con 0x7f86480ccbd0 2024-09-21T05:46:05.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.883+0000 7f8644ff9640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8634078bc0 0x7f863407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:05.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.883+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5910+0+0 (secure 0 0 0) 0x7f863c0bbb00 con 0x7f86480ccbd0 2024-09-21T05:46:05.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.883+0000 7f8647fff640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8634078bc0 0x7f863407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:05.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.884+0000 7f8647fff640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8634078bc0 0x7f863407b0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8638006fd0 tx=0x7f8638008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:05.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:05.886+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f863c083ee0 con 0x7f86480ccbd0 2024-09-21T05:46:06.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.116+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f86481f6b40 con 0x7f86480ccbd0 2024-09-21T05:46:06.119 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:46:06.117+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:46:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.118+0000 7f8644ff9640 1 -- 172.21.15.22:0/1057528276 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7f863c0837e0 con 0x7f86480ccbd0 2024-09-21T05:46:06.120 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:46:06.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8634078bc0 msgr2=0x7f863407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:06.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8634078bc0 0x7f863407b0d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8638006fd0 tx=0x7f8638008040 comp rx=0 tx=0).stop 2024-09-21T05:46:06.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 msgr2=0x7f864815f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:06.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f864815f4c0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f863c0099c0 tx=0x7f863c0039e0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 shutdown_connections 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8634078bc0 0x7f863407b0d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f86480d12c0 0x7f8648158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f86480ccbd0 0x7f864815f4c0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 --2- 172.21.15.22:0/1057528276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f86480b51c0 0x7f86480c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 >> 172.21.15.22:0/1057528276 conn(0x7f8648072ae0 msgr2=0x7f86480b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 shutdown_connections 2024-09-21T05:46:06.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.122+0000 7f864e869640 1 -- 172.21.15.22:0/1057528276 wait complete. 2024-09-21T05:46:06.125 INFO:tasks.ceph.mds.c.smithi022.stderr: -2490> 2024-09-21T05:45:30.813+0000 7fc9e0398640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:46:06.126 INFO:tasks.ceph.mds.c.smithi022.stderr: -1926> 2024-09-21T05:45:31.611+0000 7fc9dc390640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x55bd54624680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-21T05:46:06.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:46:06.183 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 -- 172.21.15.22:0/1867006285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0cf7a0 msgr2=0x7f707c169a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 --2- 172.21.15.22:0/1867006285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0cf7a0 0x7f707c169a10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7074005fd0 tx=0x7f707402ab40 comp rx=0 tx=0).stop 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 -- 172.21.15.22:0/1867006285 shutdown_connections 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 --2- 172.21.15.22:0/1867006285 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0cf7a0 0x7f707c169a10 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 -- 172.21.15.22:0/1867006285 >> 172.21.15.22:0/1867006285 conn(0x7f707c08bfc0 msgr2=0x7f707c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 -- 172.21.15.22:0/1867006285 shutdown_connections 2024-09-21T05:46:06.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.255+0000 7f70833e6640 1 -- 172.21.15.22:0/1867006285 wait complete. 2024-09-21T05:46:06.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.257+0000 7f70833e6640 1 Processor -- start 2024-09-21T05:46:06.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.257+0000 7f70833e6640 1 -- start start 2024-09-21T05:46:06.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.258+0000 7f70833e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f707c0cf7a0 0x7f707c0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.258+0000 7f70833e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f707c0d1a80 0x7f707c0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.258+0000 7f70833e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 0x7f707c15d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.258+0000 7f70833e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f707c16a160 con 0x7f707c0d2ed0 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.258+0000 7f70833e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f707c0cfeb0 con 0x7f707c0d1a80 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.258+0000 7f70833e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f707c16a2e0 con 0x7f707c0cf7a0 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 0x7f707c15d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 0x7f707c15d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58896/0 (socket says 172.21.15.22:58896) 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 -- 172.21.15.22:0/3489816832 learned_addr learned my addr 172.21.15.22:0/3489816832 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7080981640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f707c0d1a80 0x7f707c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081182640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f707c0cf7a0 0x7f707c0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 -- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f707c0cf7a0 msgr2=0x7f707c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f707c0cf7a0 0x7f707c0cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 -- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f707c0d1a80 msgr2=0x7f707c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f707c0d1a80 0x7f707c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 -- 172.21.15.22:0/3489816832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707c161fc0 con 0x7f707c0d2ed0 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7080981640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f707c0d1a80 0x7f707c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081182640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f707c0cf7a0 0x7f707c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7081983640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 0x7f707c15d930 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7074006010 tx=0x7f707402c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7074044070 con 0x7f707c0d2ed0 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7074004250 con 0x7f707c0d2ed0 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707c15cb70 con 0x7f707c0d2ed0 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.259+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70740046a0 con 0x7f707c0d2ed0 2024-09-21T05:46:06.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.260+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f707c15d130 con 0x7f707c0d2ed0 2024-09-21T05:46:06.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.260+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f707c0cf090 con 0x7f707c0d2ed0 2024-09-21T05:46:06.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.261+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7074036400 con 0x7f707c0d2ed0 2024-09-21T05:46:06.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.262+0000 7f7071ffb640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7060078bc0 0x7f706007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:06.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.262+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5996+0+0 (secure 0 0 0) 0x7f70740bb880 con 0x7f707c0d2ed0 2024-09-21T05:46:06.264 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.262+0000 7f7081182640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7060078bc0 0x7f706007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:06.265 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.263+0000 7f7081182640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7060078bc0 0x7f706007b0d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7078000fb0 tx=0x7f7078009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:06.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.266+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7074032030 con 0x7f707c0d2ed0 2024-09-21T05:46:06.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:06.497+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f707c06cde0 con 0x7f707c0d2ed0 2024-09-21T05:46:07.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.120+0000 7f7071ffb640 1 -- 172.21.15.22:0/3489816832 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7f7074083b90 con 0x7f707c0d2ed0 2024-09-21T05:46:07.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7060078bc0 msgr2=0x7f706007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7060078bc0 0x7f706007b0d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7078000fb0 tx=0x7f7078009040 comp rx=0 tx=0).stop 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 msgr2=0x7f707c15d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 0x7f707c15d930 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f7074006010 tx=0x7f707402c8d0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 shutdown_connections 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f707c0d2ed0 0x7f707c15d930 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f707c0d1a80 0x7f707c0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7060078bc0 0x7f706007b0d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 --2- 172.21.15.22:0/3489816832 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f707c0cf7a0 0x7f707c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 >> 172.21.15.22:0/3489816832 conn(0x7f707c08bfc0 msgr2=0x7f707c0b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 shutdown_connections 2024-09-21T05:46:07.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.123+0000 7f70833e6640 1 -- 172.21.15.22:0/3489816832 wait complete. 2024-09-21T05:46:07.140 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 -- 172.21.15.22:0/1258586204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 msgr2=0x7f184c0ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 --2- 172.21.15.22:0/1258586204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c0ccf10 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f184c073ec0 tx=0x7f184402a820 comp rx=0 tx=0).stop 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 -- 172.21.15.22:0/1258586204 shutdown_connections 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 --2- 172.21.15.22:0/1258586204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c0ccf10 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 --2- 172.21.15.22:0/1258586204 >> v2:172.21.15.134:3300/0 conn(0x7f184c0b7a00 0x7f184c0d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 -- 172.21.15.22:0/1258586204 >> 172.21.15.22:0/1258586204 conn(0x7f184c0ced80 msgr2=0x7f184c0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 -- 172.21.15.22:0/1258586204 shutdown_connections 2024-09-21T05:46:07.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.266+0000 7f18539a8640 1 -- 172.21.15.22:0/1258586204 wait complete. 2024-09-21T05:46:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.268+0000 7f18539a8640 1 Processor -- start 2024-09-21T05:46:07.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.268+0000 7f18539a8640 1 -- start start 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.269+0000 7f18539a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.269+0000 7f18539a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c0b7a00 0x7f184c1e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.269+0000 7f18539a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f184c0d0e90 0x7f184c1e9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.269+0000 7f18539a8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f184c0b18b0 con 0x7f184c0b2270 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.269+0000 7f18539a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f184c0b1730 con 0x7f184c0d0e90 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.269+0000 7f18539a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f184c0b2680 con 0x7f184c0b7a00 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58902/0 (socket says 172.21.15.22:58902) 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 -- 172.21.15.22:0/1347740546 learned_addr learned my addr 172.21.15.22:0/1347740546 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851f45640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f184c0d0e90 0x7f184c1e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1850f43640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c0b7a00 0x7f184c1e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 -- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c0b7a00 msgr2=0x7f184c1e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:07.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c0b7a00 0x7f184c1e97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 -- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f184c0d0e90 msgr2=0x7f184c1e9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f184c0d0e90 0x7f184c1e9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 -- 172.21.15.22:0/1347740546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184c1f4680 con 0x7f184c0b2270 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851f45640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f184c0d0e90 0x7f184c1e9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f1851744640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c159b20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f184800d750 tx=0x7f184800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848009c30 con 0x7f184c0b2270 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f184800de60 con 0x7f184c0b2270 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848004280 con 0x7f184c0b2270 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.270+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184c15d8c0 con 0x7f184c0b2270 2024-09-21T05:46:07.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.271+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f184c1f43a0 con 0x7f184c0b2270 2024-09-21T05:46:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.272+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184c1f48e0 con 0x7f184c0b2270 2024-09-21T05:46:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.272+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1848005030 con 0x7f184c0b2270 2024-09-21T05:46:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.273+0000 7f183dffb640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1830078bc0 0x7f183007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.273+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5996+0+0 (secure 0 0 0) 0x7f184809b420 con 0x7f184c0b2270 2024-09-21T05:46:07.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.273+0000 7f1850f43640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1830078bc0 0x7f183007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:07.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.273+0000 7f1850f43640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1830078bc0 0x7f183007b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f184c1578f0 tx=0x7f1840009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:07.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.277+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1848020020 con 0x7f184c0b2270 2024-09-21T05:46:07.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:07.465+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f184c0b39d0 con 0x7f184c0b2270 2024-09-21T05:46:08.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.125+0000 7f183dffb640 1 -- 172.21.15.22:0/1347740546 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) ==== 181+0+0 (secure 0 0 0) 0x7f184c0b39d0 con 0x7f184c0b2270 2024-09-21T05:46:08.128 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1830078bc0 msgr2=0x7f183007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1830078bc0 0x7f183007b0d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f184c1578f0 tx=0x7f1840009040 comp rx=0 tx=0).stop 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 msgr2=0x7f184c159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c159b20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f184800d750 tx=0x7f184800dc10 comp rx=0 tx=0).stop 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 shutdown_connections 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f184c0d0e90 0x7f184c1e9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1830078bc0 0x7f183007b0d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f184c0b7a00 0x7f184c1e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 --2- 172.21.15.22:0/1347740546 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f184c0b2270 0x7f184c159b20 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 >> 172.21.15.22:0/1347740546 conn(0x7f184c0ced80 msgr2=0x7f184c072830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 shutdown_connections 2024-09-21T05:46:08.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.130+0000 7f18539a8640 1 -- 172.21.15.22:0/1347740546 wait complete. 2024-09-21T05:46:08.147 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 -- 172.21.15.22:0/315355397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da00b0f90 msgr2=0x7f3da00b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/315355397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da00b0f90 0x7f3da00b13a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3d98005fd0 tx=0x7f3d9802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 -- 172.21.15.22:0/315355397 shutdown_connections 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/315355397 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da00b0f90 0x7f3da00b13a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/315355397 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3da0162400 0x7f3da0162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 -- 172.21.15.22:0/315355397 >> 172.21.15.22:0/315355397 conn(0x7f3da00d4380 msgr2=0x7f3da00d4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 -- 172.21.15.22:0/315355397 shutdown_connections 2024-09-21T05:46:08.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.273+0000 7f3d9f59e640 1 -- 172.21.15.22:0/315355397 wait complete. 2024-09-21T05:46:08.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.275+0000 7f3d9f59e640 1 Processor -- start 2024-09-21T05:46:08.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.275+0000 7f3d9f59e640 1 -- start start 2024-09-21T05:46:08.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.276+0000 7f3d9f59e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3da00b0f90 0x7f3da01f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:08.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.276+0000 7f3d9f59e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3da00d4ea0 0x7f3da00cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9f59e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 0x7f3da0157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9f59e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00b1a60 con 0x7f3da0162400 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9f59e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3da00b18e0 con 0x7f3da00b0f90 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9f59e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3da00b1be0 con 0x7f3da00d4ea0 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 0x7f3da0157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 0x7f3da0157750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58918/0 (socket says 172.21.15.22:58918) 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 -- 172.21.15.22:0/3817480975 learned_addr learned my addr 172.21.15.22:0/3817480975 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9dd9b640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3da00d4ea0 0x7f3da00cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:08.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9e59c640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3da00b0f90 0x7f3da01f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 -- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3da00d4ea0 msgr2=0x7f3da00cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3da00d4ea0 0x7f3da00cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 -- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3da00b0f90 msgr2=0x7f3da01f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3da00b0f90 0x7f3da01f5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 -- 172.21.15.22:0/3817480975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da0157c90 con 0x7f3da0162400 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9e59c640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3da00b0f90 0x7f3da01f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9ed9d640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 0x7f3da0157750 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009cd0 tx=0x7f3d98003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98031b90 con 0x7f3da0162400 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d98031e20 con 0x7f3da0162400 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da01f6670 con 0x7f3da0162400 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.277+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98033290 con 0x7f3da0162400 2024-09-21T05:46:08.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.278+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da01f6c30 con 0x7f3da0162400 2024-09-21T05:46:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.279+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da00b21f0 con 0x7f3da0162400 2024-09-21T05:46:08.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.279+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d98031660 con 0x7f3da0162400 2024-09-21T05:46:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.280+0000 7f3d8effd640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d80078b90 0x7f3d8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.280+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5558+0+0 (secure 0 0 0) 0x7f3d980bc2e0 con 0x7f3da0162400 2024-09-21T05:46:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.280+0000 7f3d9e59c640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d80078b90 0x7f3d8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:08.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.281+0000 7f3d9e59c640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d80078b90 0x7f3d8007b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007fd0 tx=0x7f3d90009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:08.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.285+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3d9803f030 con 0x7f3da0162400 2024-09-21T05:46:08.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:08.479+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3da01f6400 con 0x7f3da0162400 2024-09-21T05:46:09.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.127+0000 7f3d8effd640 1 -- 172.21.15.22:0/3817480975 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) ==== 169+0+0 (secure 0 0 0) 0x7f3d98084840 con 0x7f3da0162400 2024-09-21T05:46:09.130 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.131+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d80078b90 msgr2=0x7f3d8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.131+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d80078b90 0x7f3d8007b0a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007fd0 tx=0x7f3d90009040 comp rx=0 tx=0).stop 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.131+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 msgr2=0x7f3da0157750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.131+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 0x7f3da0157750 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009cd0 tx=0x7f3d98003ba0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 shutdown_connections 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3da0162400 0x7f3da0157750 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3d80078b90 0x7f3d8007b0a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3da00d4ea0 0x7f3da00cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 --2- 172.21.15.22:0/3817480975 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3da00b0f90 0x7f3da01f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 >> 172.21.15.22:0/3817480975 conn(0x7f3da00d4380 msgr2=0x7f3da00b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 shutdown_connections 2024-09-21T05:46:09.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.132+0000 7f3d9f59e640 1 -- 172.21.15.22:0/3817480975 wait complete. 2024-09-21T05:46:09.151 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 -- 172.21.15.22:0/3812114235 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 msgr2=0x7f54e00d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3812114235 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 0x7f54e00d4d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009820 tx=0x7f54d002a3f0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 -- 172.21.15.22:0/3812114235 shutdown_connections 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3812114235 >> v2:172.21.15.134:3300/0 conn(0x7f54e00cf540 0x7f54e00b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3812114235 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 0x7f54e00d4d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 -- 172.21.15.22:0/3812114235 >> 172.21.15.22:0/3812114235 conn(0x7f54e009d7f0 msgr2=0x7f54e00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 -- 172.21.15.22:0/3812114235 shutdown_connections 2024-09-21T05:46:09.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.274+0000 7f54e5e07640 1 -- 172.21.15.22:0/3812114235 wait complete. 2024-09-21T05:46:09.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.276+0000 7f54e5e07640 1 Processor -- start 2024-09-21T05:46:09.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.276+0000 7f54e5e07640 1 -- start start 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.277+0000 7f54e5e07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e00b64e0 0x7f54e01f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.277+0000 7f54e5e07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 0x7f54e01f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54e5e07640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 0x7f54e014b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54e5e07640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00d2e40 con 0x7f54e00cf540 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54e5e07640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00d2cc0 con 0x7f54e00b64e0 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54e5e07640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54e00d2fc0 con 0x7f54e00b9b60 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 0x7f54e014b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 0x7f54e014b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58920/0 (socket says 172.21.15.22:58920) 2024-09-21T05:46:09.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 -- 172.21.15.22:0/3252646700 learned_addr learned my addr 172.21.15.22:0/3252646700 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54df7fe640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e00b64e0 0x7f54e01f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54deffd640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 0x7f54e01f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 -- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 msgr2=0x7f54e01f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 0x7f54e01f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 -- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e00b64e0 msgr2=0x7f54e01f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e00b64e0 0x7f54e01f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 -- 172.21.15.22:0/3252646700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01f6760 con 0x7f54e00cf540 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54dffff640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 0x7f54e014b040 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f54d800cb10 tx=0x7f54d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8016020 con 0x7f54e00cf540 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54d800ad00 con 0x7f54e00cf540 2024-09-21T05:46:09.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8012680 con 0x7f54e00cf540 2024-09-21T05:46:09.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.278+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01f92e0 con 0x7f54e00cf540 2024-09-21T05:46:09.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.279+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01f9ac0 con 0x7f54e00cf540 2024-09-21T05:46:09.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.280+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e00cee40 con 0x7f54e00cf540 2024-09-21T05:46:09.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.280+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54d8019020 con 0x7f54e00cf540 2024-09-21T05:46:09.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.281+0000 7f54c7fff640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54cc078b90 0x7f54cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.281+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f54d809ac50 con 0x7f54e00cf540 2024-09-21T05:46:09.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.281+0000 7f54df7fe640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54cc078b90 0x7f54cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.282+0000 7f54df7fe640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54cc078b90 0x7f54cc07b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f54d0034000 tx=0x7f54d0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:09.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.285+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54e00cee40 con 0x7f54e00cf540 2024-09-21T05:46:09.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.474+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f54e01f9570 con 0x7f54e00cf540 2024-09-21T05:46:09.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.478+0000 7f54c7fff640 1 -- 172.21.15.22:0/3252646700 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f54d80a0020 con 0x7f54e00cf540 2024-09-21T05:46:09.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54cc078b90 msgr2=0x7f54cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54cc078b90 0x7f54cc07b0a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f54d0034000 tx=0x7f54d0004730 comp rx=0 tx=0).stop 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 msgr2=0x7f54e014b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 0x7f54e014b040 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f54d800cb10 tx=0x7f54d800a060 comp rx=0 tx=0).stop 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 shutdown_connections 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54e00cf540 0x7f54e014b040 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54cc078b90 0x7f54cc07b0a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e00b9b60 0x7f54e01f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 --2- 172.21.15.22:0/3252646700 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e00b64e0 0x7f54e01f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 >> 172.21.15.22:0/3252646700 conn(0x7f54e009d7f0 msgr2=0x7f54e00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 shutdown_connections 2024-09-21T05:46:09.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.480+0000 7f54e5e07640 1 -- 172.21.15.22:0/3252646700 wait complete. 2024-09-21T05:46:09.494 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-09-21T05:46:09.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 -- 172.21.15.22:0/657371133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 msgr2=0x7fb2980d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 --2- 172.21.15.22:0/657371133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 0x7fb2980d14f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb288005fd0 tx=0x7fb28802a820 comp rx=0 tx=0).stop 2024-09-21T05:46:09.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 -- 172.21.15.22:0/657371133 shutdown_connections 2024-09-21T05:46:09.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 --2- 172.21.15.22:0/657371133 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 0x7fb2980d14f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 --2- 172.21.15.22:0/657371133 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2980b9b60 0x7fb2980d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 -- 172.21.15.22:0/657371133 >> 172.21.15.22:0/657371133 conn(0x7fb29809d8f0 msgr2=0x7fb2980d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 -- 172.21.15.22:0/657371133 shutdown_connections 2024-09-21T05:46:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.609+0000 7fb29f16e640 1 -- 172.21.15.22:0/657371133 wait complete. 2024-09-21T05:46:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.610+0000 7fb29f16e640 1 Processor -- start 2024-09-21T05:46:09.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.611+0000 7fb29f16e640 1 -- start start 2024-09-21T05:46:09.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29f16e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 0x7fb2980b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29f16e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 0x7fb2981f9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29f16e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb298162430 0x7fb2981544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29f16e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb298162f50 con 0x7fb2980b9b60 2024-09-21T05:46:09.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29f16e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980d1fa0 con 0x7fb2980b3a10 2024-09-21T05:46:09.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29f16e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2981630d0 con 0x7fb298162430 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb297fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 0x7fb2981f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb297fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 0x7fb2981f9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58934/0 (socket says 172.21.15.22:58934) 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb297fff640 1 -- 172.21.15.22:0/3043807245 learned_addr learned my addr 172.21.15.22:0/3043807245 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29cf0a640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 0x7fb2980b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.612+0000 7fb29d70b640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb298162430 0x7fb2981544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb297fff640 1 -- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb298162430 msgr2=0x7fb2981544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb297fff640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb298162430 0x7fb2981544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb297fff640 1 -- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 msgr2=0x7fb2980b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb297fff640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 0x7fb2980b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb297fff640 1 -- 172.21.15.22:0/3043807245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980b47a0 con 0x7fb2980b9b60 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb297fff640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 0x7fb2981f9830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb288009cd0 tx=0x7fb288032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288037040 con 0x7fb2980b9b60 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb288042070 con 0x7fb2980b9b60 2024-09-21T05:46:09.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2981f9d70 con 0x7fb2980b9b60 2024-09-21T05:46:09.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288004280 con 0x7fb2980b9b60 2024-09-21T05:46:09.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.613+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2980b5ea0 con 0x7fb2980b9b60 2024-09-21T05:46:09.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.614+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb298162840 con 0x7fb2980b9b60 2024-09-21T05:46:09.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.615+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb288004420 con 0x7fb2980b9b60 2024-09-21T05:46:09.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.615+0000 7fb2957fa640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb278078b90 0x7fb27807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:09.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.615+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7fb2880bb860 con 0x7fb2980b9b60 2024-09-21T05:46:09.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.616+0000 7fb29cf0a640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb278078b90 0x7fb27807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:09.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.616+0000 7fb29cf0a640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb278078b90 0x7fb27807b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb280009530 tx=0x7fb280009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:09.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.620+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2981f9d70 con 0x7fb2980b9b60 2024-09-21T05:46:09.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:09.886+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fb2980ba0d0 con 0x7fb2980b9b60 2024-09-21T05:46:10.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.135+0000 7fb2957fa640 1 -- 172.21.15.22:0/3043807245 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v324) ==== 167+0+0 (secure 0 0 0) 0x7fb2980b47a0 con 0x7fb2980b9b60 2024-09-21T05:46:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb278078b90 msgr2=0x7fb27807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb278078b90 0x7fb27807b0a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb280009530 tx=0x7fb280009260 comp rx=0 tx=0).stop 2024-09-21T05:46:10.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 msgr2=0x7fb2981f9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 0x7fb2981f9830 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb288009cd0 tx=0x7fb288032c60 comp rx=0 tx=0).stop 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 shutdown_connections 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb278078b90 0x7fb27807b0a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb298162430 0x7fb2981544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2980b9b60 0x7fb2981f9830 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 --2- 172.21.15.22:0/3043807245 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2980b3a10 0x7fb2980b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 >> 172.21.15.22:0/3043807245 conn(0x7fb29809d8f0 msgr2=0x7fb2980d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 shutdown_connections 2024-09-21T05:46:10.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.137+0000 7fb29f16e640 1 -- 172.21.15.22:0/3043807245 wait complete. 2024-09-21T05:46:10.151 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-21T05:46:10.152 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:46:10.152 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:46:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.267+0000 7f0930825640 1 -- 172.21.15.22:0/2558341082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c0b97e0 msgr2=0x7f092c06f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.267+0000 7f0930825640 1 --2- 172.21.15.22:0/2558341082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c0b97e0 0x7f092c06f6e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f0914009830 tx=0x7f091402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.268+0000 7f0930825640 1 -- 172.21.15.22:0/2558341082 shutdown_connections 2024-09-21T05:46:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.268+0000 7f0930825640 1 --2- 172.21.15.22:0/2558341082 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c0b97e0 0x7f092c06f6e0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.268+0000 7f0930825640 1 -- 172.21.15.22:0/2558341082 >> 172.21.15.22:0/2558341082 conn(0x7f092c09d840 msgr2=0x7f092c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.268+0000 7f0930825640 1 -- 172.21.15.22:0/2558341082 shutdown_connections 2024-09-21T05:46:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.268+0000 7f0930825640 1 -- 172.21.15.22:0/2558341082 wait complete. 2024-09-21T05:46:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.269+0000 7f0930825640 1 Processor -- start 2024-09-21T05:46:10.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.270+0000 7f0930825640 1 -- start start 2024-09-21T05:46:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0930825640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 0x7f092c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0930825640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f092c0b97e0 0x7f092c09f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0930825640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f092c0d0e50 0x7f092c155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0930825640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f092c0cff00 con 0x7f092c06e0a0 2024-09-21T05:46:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0930825640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f092c0cfd80 con 0x7f092c0d0e50 2024-09-21T05:46:10.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0930825640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f092c0d0080 con 0x7f092c0b97e0 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 0x7f092c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 0x7f092c154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58950/0 (socket says 172.21.15.22:58950) 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 -- 172.21.15.22:0/2133195795 learned_addr learned my addr 172.21.15.22:0/2133195795 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092ad9d640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f092c0d0e50 0x7f092c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f0929d9b640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f092c0b97e0 0x7f092c09f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 -- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f092c0b97e0 msgr2=0x7f092c09f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f092c0b97e0 0x7f092c09f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 -- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f092c0d0e50 msgr2=0x7f092c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f092c0d0e50 0x7f092c155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 -- 172.21.15.22:0/2133195795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c09e550 con 0x7f092c06e0a0 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092ad9d640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f092c0d0e50 0x7f092c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.271+0000 7f092a59c640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 0x7f092c154b30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c400 tx=0x7f091c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.272+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c017070 con 0x7f092c06e0a0 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.272+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f091c0053c0 con 0x7f092c06e0a0 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.272+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c09e830 con 0x7f092c06e0a0 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.272+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091c0038a0 con 0x7f092c06e0a0 2024-09-21T05:46:10.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.272+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c09edf0 con 0x7f092c06e0a0 2024-09-21T05:46:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.273+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092c0d0430 con 0x7f092c06e0a0 2024-09-21T05:46:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.274+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f091c01a070 con 0x7f092c06e0a0 2024-09-21T05:46:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.274+0000 7f0922ffd640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f090c078b90 0x7f090c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.274+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f091c099e60 con 0x7f092c06e0a0 2024-09-21T05:46:10.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.275+0000 7f0929d9b640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f090c078b90 0x7f090c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.275+0000 7f0929d9b640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f090c078b90 0x7f090c07b0a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f0914004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:10.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.279+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f091c0624f0 con 0x7f092c06e0a0 2024-09-21T05:46:10.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.468+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f092c0b9370 con 0x7f092c06e0a0 2024-09-21T05:46:10.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.469+0000 7f0922ffd640 1 -- 172.21.15.22:0/2133195795 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+13279 (secure 0 0 0) 0x7f091c061df0 con 0x7f092c06e0a0 2024-09-21T05:46:10.471 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:10.471 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:46:09.126768+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6834/43392257":"2024-09-22T05:46:06.118226+0000","172.21.15.22:6832/43392257":"2024-09-22T05:46:06.118226+0000","172.21.15.22:6832/3207626530":"2024-09-22T05:45:24.802278+0000","172.21.15.134:6837/2104741994":"2024-09-22T05:44:14.100657+0000","172.21.15.22:6834/3207626530":"2024-09-22T05:45:24.802278+0000","172.21.15.134:6835/2104741994":"2024-09-22T05:44:14.100657+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:46:10.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f090c078b90 msgr2=0x7f090c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f090c078b90 0x7f090c07b0a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0914034000 tx=0x7f0914004730 comp rx=0 tx=0).stop 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 msgr2=0x7f092c154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 0x7f092c154b30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f091c00c400 tx=0x7f091c00c8c0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 shutdown_connections 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f092c0d0e50 0x7f092c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f090c078b90 0x7f090c07b0a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f092c0b97e0 0x7f092c09f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 --2- 172.21.15.22:0/2133195795 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f092c06e0a0 0x7f092c154b30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 >> 172.21.15.22:0/2133195795 conn(0x7f092c09d840 msgr2=0x7f092c06e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 shutdown_connections 2024-09-21T05:46:10.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.471+0000 7f0930825640 1 -- 172.21.15.22:0/2133195795 wait complete. 2024-09-21T05:46:10.485 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:46:10.485 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:46:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 -- 172.21.15.22:0/2115252077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d0c70 msgr2=0x7f8cf80d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 --2- 172.21.15.22:0/2115252077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d0c70 0x7f8cf80d1080 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f8cec005fd0 tx=0x7f8cec02a530 comp rx=0 tx=0).stop 2024-09-21T05:46:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 -- 172.21.15.22:0/2115252077 shutdown_connections 2024-09-21T05:46:10.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 --2- 172.21.15.22:0/2115252077 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d0c70 0x7f8cf80d1080 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 --2- 172.21.15.22:0/2115252077 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cf80d2430 0x7f8cf80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 -- 172.21.15.22:0/2115252077 >> 172.21.15.22:0/2115252077 conn(0x7f8cf809d880 msgr2=0x7f8cf808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 -- 172.21.15.22:0/2115252077 shutdown_connections 2024-09-21T05:46:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.603+0000 7f8cfde07640 1 -- 172.21.15.22:0/2115252077 wait complete. 2024-09-21T05:46:10.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.605+0000 7f8cfde07640 1 Processor -- start 2024-09-21T05:46:10.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.605+0000 7f8cfde07640 1 -- start start 2024-09-21T05:46:10.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cfde07640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cf80b6020 0x7f8cf80a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cfde07640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf80d0c70 0x7f8cf81f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cfde07640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 0x7f8cf80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cfde07640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8162410 con 0x7f8cf80d2430 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cfde07640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8162290 con 0x7f8cf80d0c70 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cfde07640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf80b6b40 con 0x7f8cf80b6020 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 0x7f8cf80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cf7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 0x7f8cf80ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58966/0 (socket says 172.21.15.22:58966) 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1929475985 learned_addr learned my addr 172.21.15.22:0/1929475985 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cf6ffd640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf80d0c70 0x7f8cf81f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.606+0000 7f8cf77fe640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cf80b6020 0x7f8cf80a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cf80b6020 msgr2=0x7f8cf80a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cf7fff640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cf80b6020 0x7f8cf80a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf80d0c70 msgr2=0x7f8cf81f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cf7fff640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf80d0c70 0x7f8cf81f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cf7fff640 1 -- 172.21.15.22:0/1929475985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81f6c80 con 0x7f8cf80d2430 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cf7fff640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 0x7f8cf80ca450 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000ba10 tx=0x7f8cf000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0019070 con 0x7f8cf80d2430 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cf000cc30 con 0x7f8cf80d2430 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf80b1280 con 0x7f8cf80d2430 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000ce60 con 0x7f8cf80d2430 2024-09-21T05:46:10.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.607+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf80b1840 con 0x7f8cf80d2430 2024-09-21T05:46:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.608+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf80d1080 con 0x7f8cf80d2430 2024-09-21T05:46:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.609+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8cf001c070 con 0x7f8cf80d2430 2024-09-21T05:46:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.609+0000 7f8cdffff640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ce4078b90 0x7f8ce407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.610+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f8cf009a7b0 con 0x7f8cf80d2430 2024-09-21T05:46:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.610+0000 7f8cf77fe640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ce4078b90 0x7f8ce407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.610+0000 7f8cf77fe640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ce4078b90 0x7f8ce407b0a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8004630 tx=0x7f8ce800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:10.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.614+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8cf00627e0 con 0x7f8cf80d2430 2024-09-21T05:46:10.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.800+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8cf8161950 con 0x7f8cf80d2430 2024-09-21T05:46:10.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.800+0000 7f8cdffff640 1 -- 172.21.15.22:0/1929475985 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f8cf0067210 con 0x7f8cf80d2430 2024-09-21T05:46:10.803 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ce4078b90 msgr2=0x7f8ce407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ce4078b90 0x7f8ce407b0a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8004630 tx=0x7f8ce800b2e0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 msgr2=0x7f8cf80ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 0x7f8cf80ca450 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000ba10 tx=0x7f8cf000bed0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 shutdown_connections 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8cf80d2430 0x7f8cf80ca450 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8cf80d0c70 0x7f8cf81f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8ce4078b90 0x7f8ce407b0a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 --2- 172.21.15.22:0/1929475985 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8cf80b6020 0x7f8cf80a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.802+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 >> 172.21.15.22:0/1929475985 conn(0x7f8cf809d880 msgr2=0x7f8cf80b95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.803+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 shutdown_connections 2024-09-21T05:46:10.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.803+0000 7f8cfde07640 1 -- 172.21.15.22:0/1929475985 wait complete. 2024-09-21T05:46:10.816 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.930+0000 7f4c52430640 1 -- 172.21.15.22:0/977740477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 msgr2=0x7f4c4c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.930+0000 7f4c52430640 1 --2- 172.21.15.22:0/977740477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c0d1a80 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009820 tx=0x7f4c3c02a510 comp rx=0 tx=0).stop 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.931+0000 7f4c52430640 1 -- 172.21.15.22:0/977740477 shutdown_connections 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.931+0000 7f4c52430640 1 --2- 172.21.15.22:0/977740477 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c0d1a80 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.931+0000 7f4c52430640 1 -- 172.21.15.22:0/977740477 >> 172.21.15.22:0/977740477 conn(0x7f4c4c08bfc0 msgr2=0x7f4c4c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.931+0000 7f4c52430640 1 -- 172.21.15.22:0/977740477 shutdown_connections 2024-09-21T05:46:10.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.931+0000 7f4c52430640 1 -- 172.21.15.22:0/977740477 wait complete. 2024-09-21T05:46:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.932+0000 7f4c52430640 1 Processor -- start 2024-09-21T05:46:10.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.933+0000 7f4c52430640 1 -- start start 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c52430640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c4c0b9b70 0x7f4c4c1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c52430640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c4c0cf7a0 0x7f4c4c09e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c52430640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c52430640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c162f70 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c52430640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c162df0 con 0x7f4c4c0b9b70 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c52430640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4c4c1630f0 con 0x7f4c4c0cf7a0 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:58984/0 (socket says 172.21.15.22:58984) 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 -- 172.21.15.22:0/3464059804 learned_addr learned my addr 172.21.15.22:0/3464059804 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c4bfff640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c4c0b9b70 0x7f4c4c1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c4b7fe640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c4c0cf7a0 0x7f4c4c09e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 -- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c4c0cf7a0 msgr2=0x7f4c4c09e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c4c0cf7a0 0x7f4c4c09e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 -- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c4c0b9b70 msgr2=0x7f4c4c1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c4c0b9b70 0x7f4c4c1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 -- 172.21.15.22:0/3464059804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c1611c0 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.934+0000 7f4c509cd640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c151530 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c760 tx=0x7f4c4400cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.935+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44003b70 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.935+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c44003d10 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.935+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c0c9350 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.935+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440052d0 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.935+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c0c9910 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.936+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c0d1a80 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.937+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4c44005470 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.937+0000 7f4c48ff9640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4c38078bc0 0x7f4c3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:10.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.937+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f4c4409a730 con 0x7f4c4c0d2ed0 2024-09-21T05:46:10.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.938+0000 7f4c4bfff640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4c38078bc0 0x7f4c3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:10.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.938+0000 7f4c4bfff640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4c38078bc0 0x7f4c3807b0d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c034000 tx=0x7f4c3c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:10.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:10.942+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4c440626d0 con 0x7f4c4c0d2ed0 2024-09-21T05:46:11.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:11.203+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f4c4c0b4020 con 0x7f4c4c0d2ed0 2024-09-21T05:46:12.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.141+0000 7f4c48ff9640 1 -- 172.21.15.22:0/3464059804 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v326) ==== 149+0+0 (secure 0 0 0) 0x7f4c44067100 con 0x7f4c4c0d2ed0 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.144+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4c38078bc0 msgr2=0x7f4c3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.144+0000 7f4c52430640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4c38078bc0 0x7f4c3807b0d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c034000 tx=0x7f4c3c004730 comp rx=0 tx=0).stop 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.144+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 msgr2=0x7f4c4c151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.144+0000 7f4c52430640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c151530 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c760 tx=0x7f4c4400cb60 comp rx=0 tx=0).stop 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 shutdown_connections 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4c4c0d2ed0 0x7f4c4c151530 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4c38078bc0 0x7f4c3807b0d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4c4c0cf7a0 0x7f4c4c09e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 --2- 172.21.15.22:0/3464059804 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4c4c0b9b70 0x7f4c4c1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 >> 172.21.15.22:0/3464059804 conn(0x7f4c4c08bfc0 msgr2=0x7f4c4c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:12.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 shutdown_connections 2024-09-21T05:46:12.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.145+0000 7f4c52430640 1 -- 172.21.15.22:0/3464059804 wait complete. 2024-09-21T05:46:12.159 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f12113da640 1 -- 172.21.15.22:0/1433554324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f8004f40 msgr2=0x7f11f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f12113da640 1 --2- 172.21.15.22:0/1433554324 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f11f8004f40 0x7f11f800b9c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f11fc001d90 tx=0x7f11f8005830 comp rx=0 tx=0).stop 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f120b7fe640 1 -- 172.21.15.22:0/1433554324 reap_dead start 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f12113da640 1 -- 172.21.15.22:0/1433554324 shutdown_connections 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f12113da640 1 -- 172.21.15.22:0/1433554324 >> 172.21.15.22:0/1433554324 conn(0x7f120c09d840 msgr2=0x7f120c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f12113da640 1 -- 172.21.15.22:0/1433554324 shutdown_connections 2024-09-21T05:46:12.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.278+0000 7f12113da640 1 -- 172.21.15.22:0/1433554324 wait complete. 2024-09-21T05:46:12.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.280+0000 7f12113da640 1 Processor -- start 2024-09-21T05:46:12.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.280+0000 7f12113da640 1 -- start start 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f12113da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11f8004f40 0x7f120c1f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f12113da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 0x7f120c1f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f12113da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c1f90a0 0x7f120c0b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f12113da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f120c0b9bb0 con 0x7f120c0b6960 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f12113da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f120c0b9a30 con 0x7f120c1f90a0 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f12113da640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f120c0cfb10 con 0x7f11f8004f40 2024-09-21T05:46:12.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 0x7f120c1f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 0x7f120c1f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59014/0 (socket says 172.21.15.22:59014) 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 -- 172.21.15.22:0/1151101115 learned_addr learned my addr 172.21.15.22:0/1151101115 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120b7fe640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c1f90a0 0x7f120c0b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120affd640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11f8004f40 0x7f120c1f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 -- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11f8004f40 msgr2=0x7f120c1f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11f8004f40 0x7f120c1f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 -- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c1f90a0 msgr2=0x7f120c0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c1f90a0 0x7f120c0b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.281+0000 7f120a7fc640 1 -- 172.21.15.22:0/1151101115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c1f94b0 con 0x7f120c0b6960 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.282+0000 7f120a7fc640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 0x7f120c1f8b60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f11f40066b0 tx=0x7f11f40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.282+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4015070 con 0x7f120c0b6960 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.282+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f4011920 con 0x7f120c0b6960 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.282+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4011d70 con 0x7f120c0b6960 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.282+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c159e20 con 0x7f120c0b6960 2024-09-21T05:46:12.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.282+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c15a3e0 con 0x7f120c0b6960 2024-09-21T05:46:12.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.283+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c159b90 con 0x7f120c0b6960 2024-09-21T05:46:12.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.284+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11f4018070 con 0x7f120c0b6960 2024-09-21T05:46:12.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.284+0000 7f12037fe640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f11ec078b90 0x7f11ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.284+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f11f409b6f0 con 0x7f120c0b6960 2024-09-21T05:46:12.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.284+0000 7f120affd640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f11ec078b90 0x7f11ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.287 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.285+0000 7f120affd640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f11ec078b90 0x7f11ec07b0a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f12040057a0 tx=0x7f1204005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:12.291 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.289+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f120c1f94b0 con 0x7f120c0b6960 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.479+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f120c0b6d70 con 0x7f120c0b6960 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.480+0000 7f12037fe640 1 -- 172.21.15.22:0/1151101115 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7f11f40637d0 con 0x7f120c0b6960 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:46:12.482 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:46:12.483 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:46:12.484 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f11ec078b90 msgr2=0x7f11ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f11ec078b90 0x7f11ec07b0a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f12040057a0 tx=0x7f1204005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 msgr2=0x7f120c1f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 0x7f120c1f8b60 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f11f40066b0 tx=0x7f11f40039c0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 shutdown_connections 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f120c1f90a0 0x7f120c0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f120c0b6960 0x7f120c1f8b60 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f11ec078b90 0x7f11ec07b0a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 --2- 172.21.15.22:0/1151101115 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f11f8004f40 0x7f120c1f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.482+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 >> 172.21.15.22:0/1151101115 conn(0x7f120c09d840 msgr2=0x7f120c0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:12.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.483+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 shutdown_connections 2024-09-21T05:46:12.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.483+0000 7f12113da640 1 -- 172.21.15.22:0/1151101115 wait complete. 2024-09-21T05:46:12.497 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:46:12.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 -- 172.21.15.22:0/811670555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0cfbc0 msgr2=0x7fe5dc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/811670555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0cfbc0 0x7fe5dc0cffd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc005fd0 tx=0x7fe5cc02a5f0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 -- 172.21.15.22:0/811670555 shutdown_connections 2024-09-21T05:46:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/811670555 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0cfbc0 0x7fe5dc0cffd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 -- 172.21.15.22:0/811670555 >> 172.21.15.22:0/811670555 conn(0x7fe5dc08bfc0 msgr2=0x7fe5dc09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 -- 172.21.15.22:0/811670555 shutdown_connections 2024-09-21T05:46:12.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.617+0000 7fe5e35a8640 1 -- 172.21.15.22:0/811670555 wait complete. 2024-09-21T05:46:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.619+0000 7fe5e35a8640 1 Processor -- start 2024-09-21T05:46:12.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.619+0000 7fe5e35a8640 1 -- start start 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e35a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 0x7fe5dc1595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e35a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5dc0cfbc0 0x7fe5dc15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e35a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5dc0d4230 0x7fe5dc0c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e35a8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc0b5c00 con 0x7fe5dc0b3910 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e35a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc0b5a80 con 0x7fe5dc0cfbc0 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e35a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe5dc0d4d50 con 0x7fe5dc0d4230 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.620+0000 7fe5e1344640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 0x7fe5dc1595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 0x7fe5dc1595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59032/0 (socket says 172.21.15.22:59032) 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 -- 172.21.15.22:0/1392690247 learned_addr learned my addr 172.21.15.22:0/1392690247 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e0b43640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5dc0cfbc0 0x7fe5dc15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1b45640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5dc0d4230 0x7fe5dc0c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 -- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5dc0d4230 msgr2=0x7fe5dc0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5dc0d4230 0x7fe5dc0c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 -- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5dc0cfbc0 msgr2=0x7fe5dc15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5dc0cfbc0 0x7fe5dc15cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 -- 172.21.15.22:0/1392690247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc159af0 con 0x7fe5dc0b3910 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e1344640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 0x7fe5dc1595b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c3c0 tx=0x7fe5d800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8017070 con 0x7fe5dc0b3910 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5d80035f0 con 0x7fe5dc0b3910 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc15d130 con 0x7fe5dc0b3910 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8003a40 con 0x7fe5dc0b3910 2024-09-21T05:46:12.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.621+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc0cdf80 con 0x7fe5dc0b3910 2024-09-21T05:46:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.623+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5dc0d5710 con 0x7fe5dc0b3910 2024-09-21T05:46:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.623+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe5d801a070 con 0x7fe5dc0b3910 2024-09-21T05:46:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.624+0000 7fe5d1ffb640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe5c0078b90 0x7fe5c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.624+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7fe5d8099de0 con 0x7fe5dc0b3910 2024-09-21T05:46:12.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.624+0000 7fe5e0b43640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe5c0078b90 0x7fe5c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:12.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.625+0000 7fe5e0b43640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe5c0078b90 0x7fe5c007b0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc033000 tx=0x7fe5cc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:12.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.629+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe5dc0d5710 con 0x7fe5dc0b3910 2024-09-21T05:46:12.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.817+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe5dc0cdd70 con 0x7fe5dc0b3910 2024-09-21T05:46:12.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.821+0000 7fe5d1ffb640 1 -- 172.21.15.22:0/1392690247 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7fe5d8062460 con 0x7fe5dc0b3910 2024-09-21T05:46:12.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe5c0078b90 msgr2=0x7fe5c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe5c0078b90 0x7fe5c007b0a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc033000 tx=0x7fe5cc004730 comp rx=0 tx=0).stop 2024-09-21T05:46:12.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 msgr2=0x7fe5dc1595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 0x7fe5dc1595b0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c3c0 tx=0x7fe5d800c880 comp rx=0 tx=0).stop 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 shutdown_connections 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe5c0078b90 0x7fe5c007b0a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe5dc0d4230 0x7fe5dc0c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe5dc0cfbc0 0x7fe5dc15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 --2- 172.21.15.22:0/1392690247 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe5dc0b3910 0x7fe5dc1595b0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 >> 172.21.15.22:0/1392690247 conn(0x7fe5dc08bfc0 msgr2=0x7fe5dc06e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 shutdown_connections 2024-09-21T05:46:12.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:12.823+0000 7fe5e35a8640 1 -- 172.21.15.22:0/1392690247 wait complete. 2024-09-21T05:46:12.836 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:46:12.837 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:46:12.878 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:46:12.926 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:46:12.926 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:12.926 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:12.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:46:12.942 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:46:12.942 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:12.961 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:46:12.961 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:12.961 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:12.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:46:12.999 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:46:12.999 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:13.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:46:13.019 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:46:13.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:13.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5c605c640 1 -- 172.21.15.22:0/2988557624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac003fd0 msgr2=0x7ff5ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5c605c640 1 --2- 172.21.15.22:0/2988557624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5ac003fd0 0x7ff5ac004800 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00c8000 tx=0x7ff5ac005830 comp rx=0 tx=0).stop 2024-09-21T05:46:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5bffff640 1 -- 172.21.15.22:0/2988557624 reap_dead start 2024-09-21T05:46:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5c605c640 1 -- 172.21.15.22:0/2988557624 shutdown_connections 2024-09-21T05:46:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5c605c640 1 -- 172.21.15.22:0/2988557624 >> 172.21.15.22:0/2988557624 conn(0x7ff5c008bfc0 msgr2=0x7ff5c009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5c605c640 1 -- 172.21.15.22:0/2988557624 shutdown_connections 2024-09-21T05:46:13.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.155+0000 7ff5c605c640 1 -- 172.21.15.22:0/2988557624 wait complete. 2024-09-21T05:46:13.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.157+0000 7ff5c605c640 1 Processor -- start 2024-09-21T05:46:13.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.157+0000 7ff5c605c640 1 -- start start 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.158+0000 7ff5c605c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ac003fd0 0x7ff5c014d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5c605c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 0x7ff5c00c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5c605c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c00b46e0 0x7ff5c00ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5c605c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00d2740 con 0x7ff5c00b3f60 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5c605c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c00cfee0 con 0x7ff5ac003fd0 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5c605c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c00d28c0 con 0x7ff5c00b46e0 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 0x7ff5c00c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 0x7ff5c00c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59040/0 (socket says 172.21.15.22:59040) 2024-09-21T05:46:13.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 -- 172.21.15.22:0/3967070248 learned_addr learned my addr 172.21.15.22:0/3967070248 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5bf7fe640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ac003fd0 0x7ff5c014d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5bffff640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c00b46e0 0x7ff5c00ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 -- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c00b46e0 msgr2=0x7ff5c00ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c00b46e0 0x7ff5c00ca520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 -- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ac003fd0 msgr2=0x7ff5c014d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ac003fd0 0x7ff5c014d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 -- 172.21.15.22:0/3967070248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c00b6850 con 0x7ff5c00b3f60 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5beffd640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 0x7ff5c00c9fe0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c350 tx=0x7ff5a8001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8031040 con 0x7ff5c00b3f60 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a8003610 con 0x7ff5c00b3f60 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8003a40 con 0x7ff5c00b3f60 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.159+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c01f9aa0 con 0x7ff5c00b3f60 2024-09-21T05:46:13.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.160+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c01624f0 con 0x7ff5c00b3f60 2024-09-21T05:46:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.160+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c00d1220 con 0x7ff5c00b3f60 2024-09-21T05:46:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.161+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff5a8034070 con 0x7ff5c00b3f60 2024-09-21T05:46:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.162+0000 7ff5b7fff640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5a0078b90 0x7ff5a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.162+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7ff5a80ba930 con 0x7ff5c00b3f60 2024-09-21T05:46:13.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.162+0000 7ff5bf7fe640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5a0078b90 0x7ff5a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.163+0000 7ff5bf7fe640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5a0078b90 0x7ff5a007b0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009bb0 tx=0x7ff5b0005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:13.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.167+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff5a80830a0 con 0x7ff5c00b3f60 2024-09-21T05:46:13.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.390+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5c00c9c00 con 0x7ff5c00b3f60 2024-09-21T05:46:13.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.391+0000 7ff5b7fff640 1 -- 172.21.15.22:0/3967070248 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3911 (secure 0 0 0) 0x7ff5a80829a0 con 0x7ff5c00b3f60 2024-09-21T05:46:13.393 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:13.393 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":77,"btime":"2024-09-21T05:46:07:119869+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5849,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/11977846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":11977846},{"type":"v1","addr":"172.21.15.134:6836","nonce":11977846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5966,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2011143939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":2011143939},{"type":"v1","addr":"172.21.15.134:6837","nonce":2011143939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6705,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3380951073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3380951073},{"type":"v1","addr":"172.21.15.22:6835","nonce":3380951073}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56},{"gid":7395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/2909606811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":2909606811},{"type":"v1","addr":"172.21.15.22:6834","nonce":2909606811}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[]} 2024-09-21T05:46:13.393 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 77 2024-09-21T05:46:13.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5a0078b90 msgr2=0x7ff5a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5a0078b90 0x7ff5a007b0a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009bb0 tx=0x7ff5b0005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:13.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 msgr2=0x7ff5c00c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 0x7ff5c00c9fe0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800c350 tx=0x7ff5a8001d60 comp rx=0 tx=0).stop 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 shutdown_connections 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff5a0078b90 0x7ff5a007b0a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff5c00b46e0 0x7ff5c00ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff5c00b3f60 0x7ff5c00c9fe0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 --2- 172.21.15.22:0/3967070248 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff5ac003fd0 0x7ff5c014d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 >> 172.21.15.22:0/3967070248 conn(0x7ff5c008bfc0 msgr2=0x7ff5c00d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 shutdown_connections 2024-09-21T05:46:13.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.393+0000 7ff5c605c640 1 -- 172.21.15.22:0/3967070248 wait complete. 2024-09-21T05:46:13.406 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:46:13.407 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:46:13.407 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:46:13.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:46:13.407 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:46:13.407 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:46:13.407 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:46:13.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:46:13.408 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:46:13.408 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:46:13.408 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:46:13.408 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:46:13.408 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:46:13.408 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:46:13.408 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:46:13.409 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:46:13.416 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:46:13.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:46:13.419 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:46:13.419 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:46:13.420 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:46:13.420 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:46:13.421 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:46:13.422 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:46:13.460 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:46:13.461 DEBUG:teuthology.parallel:result is None 2024-09-21T05:46:13.462 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:46:13.462 DEBUG:teuthology.parallel:result is None 2024-09-21T05:46:13.464 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:46:13.464 DEBUG:teuthology.parallel:result is None 2024-09-21T05:46:13.466 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:46:13.466 DEBUG:teuthology.parallel:result is None 2024-09-21T05:46:13.466 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:46:13.551 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:46:13.551 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:46:13.558 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:46:13.558 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:46:13.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 -- 172.21.15.22:0/4022475956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 msgr2=0x7f43f40d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 --2- 172.21.15.22:0/4022475956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 0x7f43f40d0d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f43e8005fd0 tx=0x7f43e802a5d0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 -- 172.21.15.22:0/4022475956 shutdown_connections 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 --2- 172.21.15.22:0/4022475956 >> v2:172.21.15.134:3300/0 conn(0x7f43f40d12c0 0x7f43f40d16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 --2- 172.21.15.22:0/4022475956 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 0x7f43f40d0d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 -- 172.21.15.22:0/4022475956 >> 172.21.15.22:0/4022475956 conn(0x7f43f4072ae0 msgr2=0x7f43f4072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 -- 172.21.15.22:0/4022475956 shutdown_connections 2024-09-21T05:46:13.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.589+0000 7f43fa189640 1 -- 172.21.15.22:0/4022475956 wait complete. 2024-09-21T05:46:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.591+0000 7f43fa189640 1 Processor -- start 2024-09-21T05:46:13.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.591+0000 7f43fa189640 1 -- start start 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43fa189640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43f40b51c0 0x7f43f40cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43fa189640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 0x7f43f40b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43fa189640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 0x7f43f40a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43fa189640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40cf220 con 0x7f43f40d12c0 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43fa189640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f43f40cf0a0 con 0x7f43f40b51c0 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43fa189640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f43f40cf3a0 con 0x7f43f40ccbd0 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 0x7f43f40a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f37fe640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43f40b51c0 0x7f43f40cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 0x7f43f40a9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:59072/0 (socket says 172.21.15.22:59072) 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 -- 172.21.15.22:0/1525382820 learned_addr learned my addr 172.21.15.22:0/1525382820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f2ffd640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 0x7f43f40b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 -- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 msgr2=0x7f43f40b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 0x7f43f40b1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 -- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43f40b51c0 msgr2=0x7f43f40cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43f40b51c0 0x7f43f40cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 -- 172.21.15.22:0/1525382820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f40b19c0 con 0x7f43f40d12c0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43f3fff640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 0x7f43f40a9e40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00ec20 tx=0x7f43ec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec003b70 con 0x7f43f40d12c0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.593+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43ec003e00 con 0x7f43f40d12c0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.594+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f40c8300 con 0x7f43f40d12c0 2024-09-21T05:46:13.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.594+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec004280 con 0x7f43f40d12c0 2024-09-21T05:46:13.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.594+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f40c88c0 con 0x7f43f40d12c0 2024-09-21T05:46:13.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.595+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f40cf850 con 0x7f43f40d12c0 2024-09-21T05:46:13.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.595+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f43ec004420 con 0x7f43f40d12c0 2024-09-21T05:46:13.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.596+0000 7f43dbfff640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f43e0078b90 0x7f43e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:13.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.596+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5115+0+0 (secure 0 0 0) 0x7f43ec09b7c0 con 0x7f43f40d12c0 2024-09-21T05:46:13.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.596+0000 7f43f37fe640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f43e0078b90 0x7f43e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:13.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.597+0000 7f43f37fe640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f43e0078b90 0x7f43e007b0a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43e4001720 tx=0x7f43e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:13.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.600+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f43ec0638a0 con 0x7f43f40d12c0 2024-09-21T05:46:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:13.800+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f43f40cc790 con 0x7f43f40d12c0 2024-09-21T05:46:14.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.158+0000 7f43dbfff640 1 -- 172.21.15.22:0/1525382820 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) ==== 97+0+0 (secure 0 0 0) 0x7f43ec0682d0 con 0x7f43f40d12c0 2024-09-21T05:46:14.160 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f43e0078b90 msgr2=0x7f43e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f43e0078b90 0x7f43e007b0a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f43e4001720 tx=0x7f43e4009040 comp rx=0 tx=0).stop 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 msgr2=0x7f43f40a9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 0x7f43f40a9e40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00ec20 tx=0x7f43ec00c6a0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 shutdown_connections 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f43f40d12c0 0x7f43f40a9e40 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f43e0078b90 0x7f43e007b0a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f43f40ccbd0 0x7f43f40b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 --2- 172.21.15.22:0/1525382820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f43f40b51c0 0x7f43f40cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.160+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 >> 172.21.15.22:0/1525382820 conn(0x7f43f4072ae0 msgr2=0x7f43f415eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:14.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.161+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 shutdown_connections 2024-09-21T05:46:14.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.161+0000 7f43fa189640 1 -- 172.21.15.22:0/1525382820 wait complete. 2024-09-21T05:46:14.176 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:46:14.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc232b640 1 -- 172.21.15.22:0/2153133664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 msgr2=0x7fdfa4003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc232b640 1 --2- 172.21.15.22:0/2153133664 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfa4003ea0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006fd0 tx=0x7fdfa4005830 comp rx=0 tx=0).stop 2024-09-21T05:46:14.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc08c8640 1 -- 172.21.15.22:0/2153133664 reap_dead start 2024-09-21T05:46:14.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc232b640 1 -- 172.21.15.22:0/2153133664 shutdown_connections 2024-09-21T05:46:14.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc232b640 1 -- 172.21.15.22:0/2153133664 >> 172.21.15.22:0/2153133664 conn(0x7fdfbc06d7d0 msgr2=0x7fdfbc0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:14.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc232b640 1 -- 172.21.15.22:0/2153133664 shutdown_connections 2024-09-21T05:46:14.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.293+0000 7fdfc232b640 1 -- 172.21.15.22:0/2153133664 wait complete. 2024-09-21T05:46:14.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.295+0000 7fdfc232b640 1 Processor -- start 2024-09-21T05:46:14.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.295+0000 7fdfc232b640 1 -- start start 2024-09-21T05:46:14.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc232b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfbc153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc232b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc13ccb0 0x7fdfbc06ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc232b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc06f3d0 0x7fdfbc13f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc232b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc073830 con 0x7fdfa4003a30 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc232b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc0736b0 con 0x7fdfbc06f3d0 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc232b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdfbc0739b0 con 0x7fdfbc13ccb0 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfbc153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfbc153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54936/0 (socket says 172.21.15.22:54936) 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 -- 172.21.15.22:0/3444636510 learned_addr learned my addr 172.21.15.22:0/3444636510 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfc08c8640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc06f3d0 0x7fdfbc13f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc13ccb0 0x7fdfbc06ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 -- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc13ccb0 msgr2=0x7fdfbc06ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc13ccb0 0x7fdfbc06ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 -- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc06f3d0 msgr2=0x7fdfbc13f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc06f3d0 0x7fdfbc13f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 -- 172.21.15.22:0/3444636510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc15c0d0 con 0x7fdfa4003a30 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc13ccb0 0x7fdfbc06ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfbbfff640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfbc153fe0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02b0d0 tx=0x7fdfac006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.297+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac031040 con 0x7fdfa4003a30 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.298+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfac0035f0 con 0x7fdfa4003a30 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.298+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc06f7e0 con 0x7fdfa4003a30 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.298+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003a00 con 0x7fdfa4003a30 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.298+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc15b830 con 0x7fdfa4003a30 2024-09-21T05:46:14.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.298+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfbc0b39c0 con 0x7fdfa4003a30 2024-09-21T05:46:14.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.299+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfac034070 con 0x7fdfa4003a30 2024-09-21T05:46:14.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.300+0000 7fdfb8ff9640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078bc0 0x7fdfa807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.300+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7fdfac0ba820 con 0x7fdfa4003a30 2024-09-21T05:46:14.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.300+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078bc0 0x7fdfa807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.301+0000 7fdfbb7fe640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078bc0 0x7fdfa807b0d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc153c00 tx=0x7fdfb0006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:14.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.304+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfbc06f7e0 con 0x7fdfa4003a30 2024-09-21T05:46:14.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.562+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdfbc0aeb50 con 0x7fdfa4003a30 2024-09-21T05:46:14.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.562+0000 7fdfb8ff9640 1 -- 172.21.15.22:0/3444636510 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fdfac083040 con 0x7fdfa4003a30 2024-09-21T05:46:14.565 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:14.565 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:46:14.565 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:46:14.565 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.566 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:46:14.566 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:46:14.566 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.566 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.566 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:46:14.566 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:46:14.567 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.567 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.567 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.567 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.567 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:46:14.567 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:46:14.568 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.568 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.568 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:46:14.568 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:46:14.568 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.568 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.569 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:46:14.570 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:46:14.570 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.570 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.570 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.570 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.570 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:46:14.571 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:46:14.571 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.571 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.571 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:46:14.571 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:46:14.571 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.572 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.572 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.572 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.572 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:46:14.572 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:46:14.572 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.573 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.573 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.573 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.573 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.573 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.573 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.574 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:46:14.574 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:46:14.574 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.574 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.574 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.574 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.575 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.575 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.575 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.575 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:46:14.575 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:46:14.575 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.576 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:46:14.577 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:46:14.577 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.577 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.577 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.577 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.577 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.578 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.578 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.578 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:46:14.578 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:46:14.578 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.578 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.579 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.579 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.579 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.579 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.579 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.579 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.580 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.581 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.581 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:46:14.581 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:46:14.581 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.581 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.581 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.582 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.582 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.582 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.582 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.582 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:46:14.582 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:46:14.583 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.583 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:46:14.583 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:46:14.583 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.583 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.583 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.584 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.584 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:46:14.584 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:46:14.584 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.584 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.584 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow r", 2024-09-21T05:46:14.585 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:46:14.585 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.585 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.585 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.585 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:46:14.585 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:46:14.586 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.586 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.586 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:46:14.586 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:46:14.586 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:46:14.586 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.587 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.587 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.587 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:46:14.587 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:46:14.587 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.587 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.588 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:46:14.588 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:46:14.588 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:46:14.588 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.588 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.588 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:46:14.589 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:46:14.590 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.590 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.590 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.590 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:46:14.590 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:46:14.590 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.591 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:46:14.591 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:46:14.591 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:46:14.591 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.591 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.592 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.592 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.592 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:46:14.593 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:46:14.594 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.594 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:46:14.594 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:46:14.594 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.594 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.594 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:14.596 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.596 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:46:14.596 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:46:14.596 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:46:14.596 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:46:14.596 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:46:14.597 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:46:14.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.597 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.597 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:46:14.597 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078bc0 msgr2=0x7fdfa807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078bc0 0x7fdfa807b0d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc153c00 tx=0x7fdfb0006b70 comp rx=0 tx=0).stop 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 msgr2=0x7fdfbc153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfbc153fe0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02b0d0 tx=0x7fdfac006c80 comp rx=0 tx=0).stop 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 shutdown_connections 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdfbc06f3d0 0x7fdfbc13f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdfa8078bc0 0x7fdfa807b0d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdfbc13ccb0 0x7fdfbc06ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 --2- 172.21.15.22:0/3444636510 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdfa4003a30 0x7fdfbc153fe0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 >> 172.21.15.22:0/3444636510 conn(0x7fdfbc06d7d0 msgr2=0x7fdfbc150600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 shutdown_connections 2024-09-21T05:46:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.564+0000 7fdfc232b640 1 -- 172.21.15.22:0/3444636510 wait complete. 2024-09-21T05:46:14.601 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:46:14.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 -- 172.21.15.22:0/1270857798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 msgr2=0x7f77700d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 --2- 172.21.15.22:0/1270857798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 0x7f77700d1220 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7768005fd0 tx=0x7f776802aa90 comp rx=0 tx=0).stop 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 -- 172.21.15.22:0/1270857798 shutdown_connections 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 --2- 172.21.15.22:0/1270857798 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 0x7f77700d1220 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 --2- 172.21.15.22:0/1270857798 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f77700b3a60 0x7f77700cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 -- 172.21.15.22:0/1270857798 >> 172.21.15.22:0/1270857798 conn(0x7f777008bfc0 msgr2=0x7f777009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 -- 172.21.15.22:0/1270857798 shutdown_connections 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.716+0000 7f77769da640 1 -- 172.21.15.22:0/1270857798 wait complete. 2024-09-21T05:46:14.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.718+0000 7f77769da640 1 Processor -- start 2024-09-21T05:46:14.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.718+0000 7f77769da640 1 -- start start 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f77769da640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f777006e3c0 0x7f77701587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f77769da640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 0x7f7770158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f77769da640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 0x7f77700ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f77769da640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f77700d2cc0 con 0x7f77700b3a60 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f77769da640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f77700d2b40 con 0x7f77700d0e10 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f77769da640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f77700d2e40 con 0x7f777006e3c0 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.719+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 0x7f7770158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 0x7f7770158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54950/0 (socket says 172.21.15.22:54950) 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 -- 172.21.15.22:0/2884899346 learned_addr learned my addr 172.21.15.22:0/2884899346 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f7774f77640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 0x7f77700ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776ffff640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f777006e3c0 0x7f77701587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 -- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f777006e3c0 msgr2=0x7f77701587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f777006e3c0 0x7f77701587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 -- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 msgr2=0x7f77700ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 0x7f77700ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 -- 172.21.15.22:0/2884899346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77701593f0 con 0x7f77700b3a60 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776f7fe640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 0x7f7770158eb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f776400c3c0 tx=0x7f776400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764017070 con 0x7f77700b3a60 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77640053e0 con 0x7f77700b3a60 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77700cd9a0 con 0x7f77700b3a60 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764003850 con 0x7f77700b3a60 2024-09-21T05:46:14.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.720+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77700cdf60 con 0x7f77700b3a60 2024-09-21T05:46:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.721+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77700d32f0 con 0x7f77700b3a60 2024-09-21T05:46:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.722+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f776401a070 con 0x7f77700b3a60 2024-09-21T05:46:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.723+0000 7f776cff9640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f775c078bc0 0x7f775c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.723+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f7764099f70 con 0x7f77700b3a60 2024-09-21T05:46:14.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.723+0000 7f776ffff640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f775c078bc0 0x7f775c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:14.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.724+0000 7f776ffff640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f775c078bc0 0x7f775c07b0d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7760009b30 tx=0x7f7760005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:14.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.727+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7764021080 con 0x7f77700b3a60 2024-09-21T05:46:14.916 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.914+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f777015e870 con 0x7f77700b3a60 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.915+0000 7f776cff9640 1 -- 172.21.15.22:0/2884899346 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) ==== 84+0+65 (secure 0 0 0) 0x7f77701593f0 con 0x7f77700b3a60 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:14.917 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f775c078bc0 msgr2=0x7f775c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f775c078bc0 0x7f775c07b0d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7760009b30 tx=0x7f7760005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 msgr2=0x7f7770158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 0x7f7770158eb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f776400c3c0 tx=0x7f776400c880 comp rx=0 tx=0).stop 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 shutdown_connections 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f77700d0e10 0x7f77700ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.919 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f77700b3a60 0x7f7770158eb0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f775c078bc0 0x7f775c07b0d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 --2- 172.21.15.22:0/2884899346 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f777006e3c0 0x7f77701587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:14.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 >> 172.21.15.22:0/2884899346 conn(0x7f777008bfc0 msgr2=0x7f77700b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:14.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 shutdown_connections 2024-09-21T05:46:14.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:14.917+0000 7f77769da640 1 -- 172.21.15.22:0/2884899346 wait complete. 2024-09-21T05:46:14.931 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:46:14.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/2791671645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 msgr2=0x7f8a240b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/2791671645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b47d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c005fd0 tx=0x7f8a1c02a540 comp rx=0 tx=0).stop 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/2791671645 shutdown_connections 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/2791671645 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b47d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/2791671645 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a240b67e0 0x7f8a240b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/2791671645 >> 172.21.15.22:0/2791671645 conn(0x7f8a2409aa30 msgr2=0x7f8a240b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/2791671645 shutdown_connections 2024-09-21T05:46:15.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.050+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/2791671645 wait complete. 2024-09-21T05:46:15.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.052+0000 7f8a2b3c6640 1 Processor -- start 2024-09-21T05:46:15.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.052+0000 7f8a2b3c6640 1 -- start start 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.053+0000 7f8a2b3c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a2406cde0 0x7f8a240af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.053+0000 7f8a2b3c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.053+0000 7f8a2b3c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a240b67e0 0x7f8a240b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.053+0000 7f8a2b3c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b50e0 con 0x7f8a240b43c0 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.053+0000 7f8a2b3c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8a240b4f60 con 0x7f8a2406cde0 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.053+0000 7f8a2b3c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8a240b5260 con 0x7f8a240b67e0 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54970/0 (socket says 172.21.15.22:54970) 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 -- 172.21.15.22:0/3732586102 learned_addr learned my addr 172.21.15.22:0/3732586102 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a29963640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a240b67e0 0x7f8a240b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a29162640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a2406cde0 0x7f8a240af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 -- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a240b67e0 msgr2=0x7f8a240b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:15.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a240b67e0 0x7f8a240b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 -- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a2406cde0 msgr2=0x7f8a240af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a2406cde0 0x7f8a240af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 -- 172.21.15.22:0/3732586102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a240b1fa0 con 0x7f8a240b43c0 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a28961640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b1450 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c3a0 tx=0x7f8a1400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14017070 con 0x7f8a240b43c0 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a140053c0 con 0x7f8a240b43c0 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a241f5d50 con 0x7f8a240b43c0 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a14003850 con 0x7f8a240b43c0 2024-09-21T05:46:15.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.054+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a241f5ff0 con 0x7f8a240b43c0 2024-09-21T05:46:15.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.055+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a240cbcd0 con 0x7f8a240b43c0 2024-09-21T05:46:15.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.056+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a1401a070 con 0x7f8a240b43c0 2024-09-21T05:46:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.057+0000 7f8a19ffb640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a08078b90 0x7f8a0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.057+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 4857+0+0 (secure 0 0 0) 0x7f8a1409a0f0 con 0x7f8a240b43c0 2024-09-21T05:46:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.057+0000 7f8a29162640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a08078b90 0x7f8a0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:15.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.058+0000 7f8a29162640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a08078b90 0x7f8a0807b0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005fd0 tx=0x7f8a20009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:15.063 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.061+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8a240cbcd0 con 0x7f8a240b43c0 2024-09-21T05:46:15.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:15.249+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8a240b6d30 con 0x7f8a240b43c0 2024-09-21T05:46:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.163+0000 7f8a19ffb640 1 -- 172.21.15.22:0/3732586102 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v94) ==== 138+0+0 (secure 0 0 0) 0x7f8a14021020 con 0x7f8a240b43c0 2024-09-21T05:46:16.166 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:46:16.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a08078b90 msgr2=0x7f8a0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:16.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a08078b90 0x7f8a0807b0a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a20005fd0 tx=0x7f8a20009500 comp rx=0 tx=0).stop 2024-09-21T05:46:16.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 msgr2=0x7f8a240b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b1450 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c3a0 tx=0x7f8a1400c860 comp rx=0 tx=0).stop 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 shutdown_connections 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8a08078b90 0x7f8a0807b0a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8a240b67e0 0x7f8a240b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8a240b43c0 0x7f8a240b1450 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 --2- 172.21.15.22:0/3732586102 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8a2406cde0 0x7f8a240af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 >> 172.21.15.22:0/3732586102 conn(0x7f8a2409aa30 msgr2=0x7f8a2415bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 shutdown_connections 2024-09-21T05:46:16.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.166+0000 7f8a2b3c6640 1 -- 172.21.15.22:0/3732586102 wait complete. 2024-09-21T05:46:16.187 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:46:16.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 -- 172.21.15.22:0/3351227501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 msgr2=0x7f5fdc0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 --2- 172.21.15.22:0/3351227501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0d2380 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8005fd0 tx=0x7f5fd802aa10 comp rx=0 tx=0).stop 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 -- 172.21.15.22:0/3351227501 shutdown_connections 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 --2- 172.21.15.22:0/3351227501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fdc0d28c0 0x7f5fdc0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 --2- 172.21.15.22:0/3351227501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0d2380 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 -- 172.21.15.22:0/3351227501 >> 172.21.15.22:0/3351227501 conn(0x7f5fdc0d4530 msgr2=0x7f5fdc0d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 -- 172.21.15.22:0/3351227501 shutdown_connections 2024-09-21T05:46:16.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.317+0000 7f5fe3469640 1 -- 172.21.15.22:0/3351227501 wait complete. 2024-09-21T05:46:16.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.319+0000 7f5fe3469640 1 Processor -- start 2024-09-21T05:46:16.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.319+0000 7f5fe3469640 1 -- start start 2024-09-21T05:46:16.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.320+0000 7f5fe3469640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 0x7f5fdc161b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.320+0000 7f5fe3469640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fdc0d28c0 0x7f5fdc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.320+0000 7f5fe3469640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.320+0000 7f5fe3469640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc0b3770 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.320+0000 7f5fe3469640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc0b35f0 con 0x7f5fdc15eb70 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.320+0000 7f5fe3469640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5fdc0b38f0 con 0x7f5fdc0d28c0 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 0x7f5fdc161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 0x7f5fdc161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54974/0 (socket says 172.21.15.22:54974) 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 -- 172.21.15.22:0/136336278 learned_addr learned my addr 172.21.15.22:0/136336278 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe0a04640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fdc0d28c0 0x7f5fdc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1a06640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 -- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fdc0d28c0 msgr2=0x7f5fdc150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fdc0d28c0 0x7f5fdc150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 -- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 msgr2=0x7f5fdc0b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:16.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 -- 172.21.15.22:0/136336278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc162040 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1a06640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe1205640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 0x7f5fdc161b00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000f80 tx=0x7f5fd8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8033050 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5fd8004d40 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80044b0 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.321+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc15fef0 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.322+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1604b0 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.323+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc0d0440 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.324+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5fd8036070 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.324+0000 7f5fd1ffb640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5fc0078b90 0x7f5fc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:16.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.324+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5227+0+0 (secure 0 0 0) 0x7f5fd80bba30 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.325+0000 7f5fe0a04640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5fc0078b90 0x7f5fc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:16.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.325+0000 7f5fe0a04640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5fc0078b90 0x7f5fc007b0a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0095f0 tx=0x7f5fcc005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:16.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.329+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5fd80840e0 con 0x7f5fdc0b28c0 2024-09-21T05:46:16.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:16.517+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5fdc15e190 con 0x7f5fdc0b28c0 2024-09-21T05:46:17.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.170+0000 7f5fd1ffb640 1 -- 172.21.15.22:0/136336278 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v95) ==== 170+0+0 (secure 0 0 0) 0x7f5fd80839e0 con 0x7f5fdc0b28c0 2024-09-21T05:46:17.172 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:46:17.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5fc0078b90 msgr2=0x7f5fc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:17.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5fc0078b90 0x7f5fc007b0a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0095f0 tx=0x7f5fcc005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:17.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 msgr2=0x7f5fdc161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:17.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 0x7f5fdc161b00 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8000f80 tx=0x7f5fd8003f90 comp rx=0 tx=0).stop 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 shutdown_connections 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5fdc15eb70 0x7f5fdc0b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5fc0078b90 0x7f5fc007b0a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5fdc0d28c0 0x7f5fdc150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 --2- 172.21.15.22:0/136336278 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5fdc0b28c0 0x7f5fdc161b00 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.174+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 >> 172.21.15.22:0/136336278 conn(0x7f5fdc0d4530 msgr2=0x7f5fdc0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.175+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 shutdown_connections 2024-09-21T05:46:17.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.175+0000 7f5fe3469640 1 -- 172.21.15.22:0/136336278 wait complete. 2024-09-21T05:46:17.205 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 -- 172.21.15.22:0/2339825912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 msgr2=0x7f60e415c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/2339825912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e415c6e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f60d4005fd0 tx=0x7f60d402aa90 comp rx=0 tx=0).stop 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 -- 172.21.15.22:0/2339825912 shutdown_connections 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/2339825912 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40ccbd0 0x7f60e40d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/2339825912 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e415c6e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 -- 172.21.15.22:0/2339825912 >> 172.21.15.22:0/2339825912 conn(0x7f60e4072ae0 msgr2=0x7f60e4072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 -- 172.21.15.22:0/2339825912 shutdown_connections 2024-09-21T05:46:17.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.328+0000 7f60ea1c6640 1 -- 172.21.15.22:0/2339825912 wait complete. 2024-09-21T05:46:17.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.330+0000 7f60ea1c6640 1 Processor -- start 2024-09-21T05:46:17.332 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.330+0000 7f60ea1c6640 1 -- start start 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40ccbd0 0x7f60e41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40d12c0 0x7f60e4144d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f60e4160a60 con 0x7f60e40b51c0 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f60e40d1a30 con 0x7f60e40ccbd0 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f60e4160be0 con 0x7f60e40d12c0 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e4156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:54990/0 (socket says 172.21.15.22:54990) 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 -- 172.21.15.22:0/3877043186 learned_addr learned my addr 172.21.15.22:0/3877043186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 -- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40d12c0 msgr2=0x7f60e4144d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40ccbd0 0x7f60e41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:17.335 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e3fff640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40d12c0 0x7f60e4144d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40d12c0 0x7f60e4144d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 -- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40ccbd0 msgr2=0x7f60e41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40ccbd0 0x7f60e41447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 -- 172.21.15.22:0/3877043186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e4153bd0 con 0x7f60e40b51c0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e3fff640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40d12c0 0x7f60e4144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40ccbd0 0x7f60e41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60e37fe640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e4156d20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f60d4000f80 tx=0x7f60d4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4044070 con 0x7f60e40b51c0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e415e4f0 con 0x7f60e40b51c0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60d4004250 con 0x7f60e40b51c0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.332+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4004660 con 0x7f60e40b51c0 2024-09-21T05:46:17.336 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.333+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e415eab0 con 0x7f60e40b51c0 2024-09-21T05:46:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.334+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e4161090 con 0x7f60e40b51c0 2024-09-21T05:46:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.334+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f60d4036400 con 0x7f60e40b51c0 2024-09-21T05:46:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.335+0000 7f60dbfff640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60c4078bc0 0x7f60c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:17.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.335+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5609+0+0 (secure 0 0 0) 0x7f60d40bbbf0 con 0x7f60e40b51c0 2024-09-21T05:46:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.335+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60c4078bc0 0x7f60c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:17.338 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.336+0000 7f60e2ffd640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60c4078bc0 0x7f60c407b0d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f60e4153850 tx=0x7f60cc005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:17.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.340+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60e4153bd0 con 0x7f60e40b51c0 2024-09-21T05:46:17.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:17.566+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f60e40b1970 con 0x7f60e40b51c0 2024-09-21T05:46:18.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.178+0000 7f60dbfff640 1 -- 172.21.15.22:0/3877043186 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v80) ==== 174+0+0 (secure 0 0 0) 0x7f60d4032030 con 0x7f60e40b51c0 2024-09-21T05:46:18.181 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 14 and data pool 15 2024-09-21T05:46:18.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.189+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60c4078bc0 msgr2=0x7f60c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.189+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60c4078bc0 0x7f60c407b0d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f60e4153850 tx=0x7f60cc005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:18.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.189+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 msgr2=0x7f60e4156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.189+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e4156d20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f60d4000f80 tx=0x7f60d4003ce0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.192+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 shutdown_connections 2024-09-21T05:46:18.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.192+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f60c4078bc0 0x7f60c407b0d0 secure :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f60e4153850 tx=0x7f60cc005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:18.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.192+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f60e40d12c0 0x7f60e4144d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.192+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f60e40ccbd0 0x7f60e41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.192+0000 7f60ea1c6640 1 --2- 172.21.15.22:0/3877043186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f60e40b51c0 0x7f60e4156d20 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.192+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 >> 172.21.15.22:0/3877043186 conn(0x7f60e4072ae0 msgr2=0x7f60e40b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:18.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.196+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 shutdown_connections 2024-09-21T05:46:18.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.197+0000 7f60ea1c6640 1 -- 172.21.15.22:0/3877043186 wait complete. 2024-09-21T05:46:18.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:46:18.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 -- 172.21.15.22:0/1892183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0cf940 msgr2=0x7fed8c06fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 --2- 172.21.15.22:0/1892183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0cf940 0x7fed8c06fc80 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fed80005fd0 tx=0x7fed8002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 -- 172.21.15.22:0/1892183032 shutdown_connections 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 --2- 172.21.15.22:0/1892183032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0cf940 0x7fed8c06fc80 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 --2- 172.21.15.22:0/1892183032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed8c0b4d90 0x7fed8c072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 -- 172.21.15.22:0/1892183032 >> 172.21.15.22:0/1892183032 conn(0x7fed8c06d7d0 msgr2=0x7fed8c0b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 -- 172.21.15.22:0/1892183032 shutdown_connections 2024-09-21T05:46:18.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.342+0000 7fed92d6e640 1 -- 172.21.15.22:0/1892183032 wait complete. 2024-09-21T05:46:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.344+0000 7fed92d6e640 1 Processor -- start 2024-09-21T05:46:18.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.344+0000 7fed92d6e640 1 -- start start 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.345+0000 7fed92d6e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 0x7fed8c0acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.345+0000 7fed92d6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed8c0cf940 0x7fed8c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.345+0000 7fed92d6e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed8c151b10 0x7fed8c151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.345+0000 7fed92d6e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c0ccbc0 con 0x7fed8c0701c0 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.345+0000 7fed92d6e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c070980 con 0x7fed8c151b10 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.345+0000 7fed92d6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fed8c0ccd40 con 0x7fed8c0cf940 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 0x7fed8c0acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 0x7fed8c0acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55004/0 (socket says 172.21.15.22:55004) 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 -- 172.21.15.22:0/503963575 learned_addr learned my addr 172.21.15.22:0/503963575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed8bfff640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed8c0cf940 0x7fed8c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed9130b640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed8c151b10 0x7fed8c151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 -- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed8c0cf940 msgr2=0x7fed8c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed8c0cf940 0x7fed8c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 -- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed8c151b10 msgr2=0x7fed8c151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed8c151b10 0x7fed8c151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 -- 172.21.15.22:0/503963575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c0ad790 con 0x7fed8c0701c0 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed9130b640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed8c151b10 0x7fed8c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed90b0a640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 0x7fed8c0acf90 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00e790 tx=0x7fed7c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c00eea0 con 0x7fed8c0701c0 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed7c00cdb0 con 0x7fed8c0701c0 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7c010690 con 0x7fed8c0701c0 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.346+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c0ad4d0 con 0x7fed8c0701c0 2024-09-21T05:46:18.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.347+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c152d10 con 0x7fed8c0701c0 2024-09-21T05:46:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.348+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c0cd190 con 0x7fed8c0701c0 2024-09-21T05:46:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.348+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed7c010830 con 0x7fed8c0701c0 2024-09-21T05:46:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.349+0000 7fed897fa640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fed78078bc0 0x7fed7807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.349+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fed7c09b6d0 con 0x7fed8c0701c0 2024-09-21T05:46:18.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.349+0000 7fed8bfff640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fed78078bc0 0x7fed7807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.350+0000 7fed8bfff640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fed78078bc0 0x7fed7807b0d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed80033000 tx=0x7fed80004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:18.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.353+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed7c0634c0 con 0x7fed8c0701c0 2024-09-21T05:46:18.547 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.545+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed8c0b1f00 con 0x7fed8c0701c0 2024-09-21T05:46:18.548 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.546+0000 7fed897fa640 1 -- 172.21.15.22:0/503963575 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16669 (secure 0 0 0) 0x7fed7c067ef0 con 0x7fed8c0701c0 2024-09-21T05:46:18.548 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:18.549 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":96,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:46:17.567441+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:46:15.254551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T05:46:16.522802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.548+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fed78078bc0 msgr2=0x7fed7807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.548+0000 7fed92d6e640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fed78078bc0 0x7fed7807b0d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed80033000 tx=0x7fed80004730 comp rx=0 tx=0).stop 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.548+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 msgr2=0x7fed8c0acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.548+0000 7fed92d6e640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 0x7fed8c0acf90 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00e790 tx=0x7fed7c00ec50 comp rx=0 tx=0).stop 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 shutdown_connections 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fed8c151b10 0x7fed8c151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fed78078bc0 0x7fed7807b0d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fed8c0cf940 0x7fed8c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 --2- 172.21.15.22:0/503963575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fed8c0701c0 0x7fed8c0acf90 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 >> 172.21.15.22:0/503963575 conn(0x7fed8c06d7d0 msgr2=0x7fed8c15c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 shutdown_connections 2024-09-21T05:46:18.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.549+0000 7fed92d6e640 1 -- 172.21.15.22:0/503963575 wait complete. 2024-09-21T05:46:18.563 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d2320640 1 -- 172.21.15.22:0/4266921736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 msgr2=0x7fb4b4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d2320640 1 --2- 172.21.15.22:0/4266921736 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4b4004c10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000c290 tx=0x7fb4b4005830 comp rx=0 tx=0).stop 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d08bd640 1 -- 172.21.15.22:0/4266921736 reap_dead start 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d2320640 1 -- 172.21.15.22:0/4266921736 shutdown_connections 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d2320640 1 -- 172.21.15.22:0/4266921736 >> 172.21.15.22:0/4266921736 conn(0x7fb4cc0710b0 msgr2=0x7fb4cc0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d2320640 1 -- 172.21.15.22:0/4266921736 shutdown_connections 2024-09-21T05:46:18.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.677+0000 7fb4d2320640 1 -- 172.21.15.22:0/4266921736 wait complete. 2024-09-21T05:46:18.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.679+0000 7fb4d2320640 1 Processor -- start 2024-09-21T05:46:18.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.679+0000 7fb4d2320640 1 -- start start 2024-09-21T05:46:18.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4cc14cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.684 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14d4e0 0x7fb4cc147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc147fd0 0x7fb4cc1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.685 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc0d08a0 con 0x7fb4b4004800 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15ca40 con 0x7fb4cc14d4e0 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc0d0a20 con 0x7fb4cc147fd0 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4cc14cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4cc14cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55022/0 (socket says 172.21.15.22:55022) 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 -- 172.21.15.22:0/1669004271 learned_addr learned my addr 172.21.15.22:0/1669004271 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cb7fe640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14d4e0 0x7fb4cc147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d08bd640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc147fd0 0x7fb4cc1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 -- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc147fd0 msgr2=0x7fb4cc1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc147fd0 0x7fb4cc1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 -- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14d4e0 msgr2=0x7fb4cc147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14d4e0 0x7fb4cc147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 -- 172.21.15.22:0/1669004271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc15e260 con 0x7fb4b4004800 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4cbfff640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4cc14cfa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc011ae0 tx=0x7fb4bc009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:18.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc01e070 con 0x7fb4b4004800 2024-09-21T05:46:18.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4bc00fe70 con 0x7fb4b4004800 2024-09-21T05:46:18.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc14d8f0 con 0x7fb4b4004800 2024-09-21T05:46:18.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.681+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc004290 con 0x7fb4b4004800 2024-09-21T05:46:18.687 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.682+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc15e800 con 0x7fb4b4004800 2024-09-21T05:46:18.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.682+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc0d0fb0 con 0x7fb4b4004800 2024-09-21T05:46:18.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.683+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb4bc021070 con 0x7fb4b4004800 2024-09-21T05:46:18.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.684+0000 7fb4c8ff9640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb4b8078b90 0x7fb4b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:18.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.684+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fb4bc09b790 con 0x7fb4b4004800 2024-09-21T05:46:18.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.684+0000 7fb4cb7fe640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb4b8078b90 0x7fb4b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:18.688 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.685+0000 7fb4cb7fe640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb4b8078b90 0x7fb4b807b0a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0033000 tx=0x7fb4c0038040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:18.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.688+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4bc0636d0 con 0x7fb4b4004800 2024-09-21T05:46:18.878 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.876+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4cc14de00 con 0x7fb4b4004800 2024-09-21T05:46:18.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.877+0000 7fb4c8ff9640 1 -- 172.21.15.22:0/1669004271 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+16669 (secure 0 0 0) 0x7fb4bc068100 con 0x7fb4b4004800 2024-09-21T05:46:18.880 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:18.880 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":96,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:46:17.567441+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:46:15.254551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T05:46:16.522802+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb4b8078b90 msgr2=0x7fb4b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb4b8078b90 0x7fb4b807b0a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0033000 tx=0x7fb4c0038040 comp rx=0 tx=0).stop 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 msgr2=0x7fb4cc14cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4cc14cfa0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc011ae0 tx=0x7fb4bc009e90 comp rx=0 tx=0).stop 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 shutdown_connections 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb4b8078b90 0x7fb4b807b0a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4cc147fd0 0x7fb4cc1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4cc14d4e0 0x7fb4cc147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 --2- 172.21.15.22:0/1669004271 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4b4004800 0x7fb4cc14cfa0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.879+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 >> 172.21.15.22:0/1669004271 conn(0x7fb4cc0710b0 msgr2=0x7fb4cc15fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.880+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 shutdown_connections 2024-09-21T05:46:18.883 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:18.880+0000 7fb4d2320640 1 -- 172.21.15.22:0/1669004271 wait complete. 2024-09-21T05:46:18.894 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 -- 172.21.15.22:0/2320556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d07c0 msgr2=0x7fe3e806e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 --2- 172.21.15.22:0/2320556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d07c0 0x7fe3e806e950 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80700d0 tx=0x7fe3e002ab40 comp rx=0 tx=0).stop 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 -- 172.21.15.22:0/2320556775 shutdown_connections 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 --2- 172.21.15.22:0/2320556775 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d07c0 0x7fe3e806e950 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 --2- 172.21.15.22:0/2320556775 >> v2:172.21.15.134:3300/0 conn(0x7fe3e80d2240 0x7fe3e80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 -- 172.21.15.22:0/2320556775 >> 172.21.15.22:0/2320556775 conn(0x7fe3e80b3f10 msgr2=0x7fe3e80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 -- 172.21.15.22:0/2320556775 shutdown_connections 2024-09-21T05:46:19.014 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.011+0000 7fe3ee172640 1 -- 172.21.15.22:0/2320556775 wait complete. 2024-09-21T05:46:19.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.013+0000 7fe3ee172640 1 Processor -- start 2024-09-21T05:46:19.015 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.013+0000 7fe3ee172640 1 -- start start 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.014+0000 7fe3ee172640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e80d07c0 0x7fe3e814b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.014+0000 7fe3ee172640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 0x7fe3e80cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.014+0000 7fe3ee172640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e815eb00 0x7fe3e80cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.014+0000 7fe3ee172640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e806f220 con 0x7fe3e80d2240 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.014+0000 7fe3ee172640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e806f0a0 con 0x7fe3e815eb00 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.014+0000 7fe3ee172640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e806f3a0 con 0x7fe3e80d07c0 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 0x7fe3e80cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 0x7fe3e80cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55048/0 (socket says 172.21.15.22:55048) 2024-09-21T05:46:19.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 -- 172.21.15.22:0/192605672 learned_addr learned my addr 172.21.15.22:0/192605672 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e7fff640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e815eb00 0x7fe3e80cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e77fe640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e80d07c0 0x7fe3e814b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 -- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e80d07c0 msgr2=0x7fe3e814b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e80d07c0 0x7fe3e814b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 -- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e815eb00 msgr2=0x7fe3e80cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e815eb00 0x7fe3e80cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 -- 172.21.15.22:0/192605672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e80c85a0 con 0x7fe3e80d2240 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e7fff640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e815eb00 0x7fe3e80cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e77fe640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e80d07c0 0x7fe3e814b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3e6ffd640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 0x7fe3e80cd1e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00d740 tx=0x7fe3dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc017070 con 0x7fe3e80d2240 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3dc009d20 con 0x7fe3e80d2240 2024-09-21T05:46:19.019 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e80c8860 con 0x7fe3e80d2240 2024-09-21T05:46:19.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0106c0 con 0x7fe3e80d2240 2024-09-21T05:46:19.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.015+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e8154f50 con 0x7fe3e80d2240 2024-09-21T05:46:19.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.017+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e81f9f20 con 0x7fe3e80d2240 2024-09-21T05:46:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.017+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe3dc01a070 con 0x7fe3e80d2240 2024-09-21T05:46:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.018+0000 7fe3cffff640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe3d4078b90 0x7fe3d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.018+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5709+0+0 (secure 0 0 0) 0x7fe3dc09ae60 con 0x7fe3e80d2240 2024-09-21T05:46:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.018+0000 7fe3e77fe640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe3d4078b90 0x7fe3d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:19.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.019+0000 7fe3e77fe640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe3d4078b90 0x7fe3d407b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80016a0 tx=0x7fe3d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:19.024 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.023+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3dc063320 con 0x7fe3e80d2240 2024-09-21T05:46:19.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:19.256+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe3e80b6730 con 0x7fe3e80d2240 2024-09-21T05:46:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.203+0000 7fe3cffff640 1 -- 172.21.15.22:0/192605672 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7fe3e80b6730 con 0x7fe3e80d2240 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.206+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe3d4078b90 msgr2=0x7fe3d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.206+0000 7fe3ee172640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe3d4078b90 0x7fe3d407b0a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80016a0 tx=0x7fe3d8009040 comp rx=0 tx=0).stop 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.206+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 msgr2=0x7fe3e80cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.206+0000 7fe3ee172640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 0x7fe3e80cd1e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00d740 tx=0x7fe3dc00dc00 comp rx=0 tx=0).stop 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.206+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 shutdown_connections 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.206+0000 7fe3ee172640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe3e815eb00 0x7fe3e80cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.207+0000 7fe3ee172640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe3e80d2240 0x7fe3e80cd1e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.207+0000 7fe3ee172640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe3d4078b90 0x7fe3d407b0a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.207+0000 7fe3ee172640 1 --2- 172.21.15.22:0/192605672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe3e80d07c0 0x7fe3e814b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.207+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 >> 172.21.15.22:0/192605672 conn(0x7fe3e80b3f10 msgr2=0x7fe3e80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:20.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.207+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 shutdown_connections 2024-09-21T05:46:20.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.207+0000 7fe3ee172640 1 -- 172.21.15.22:0/192605672 wait complete. 2024-09-21T05:46:20.222 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:46:20.223 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:46:20.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 -- 172.21.15.22:0/1162543592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a50004800 msgr2=0x7f6a50004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:20.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1162543592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a50004800 0x7f6a50004c10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0016c0 tx=0x7f6a50005830 comp rx=0 tx=0).stop 2024-09-21T05:46:20.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 -- 172.21.15.22:0/1162543592 shutdown_connections 2024-09-21T05:46:20.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1162543592 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a50004800 0x7f6a50004c10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 -- 172.21.15.22:0/1162543592 >> 172.21.15.22:0/1162543592 conn(0x7f6a6809d830 msgr2=0x7f6a680d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:20.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 -- 172.21.15.22:0/1162543592 shutdown_connections 2024-09-21T05:46:20.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.348+0000 7f6a6d094640 1 -- 172.21.15.22:0/1162543592 wait complete. 2024-09-21T05:46:20.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.350+0000 7f6a6d094640 1 Processor -- start 2024-09-21T05:46:20.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.350+0000 7f6a6d094640 1 -- start start 2024-09-21T05:46:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.351+0000 7f6a6d094640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a50004800 0x7f6a680ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:20.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.351+0000 7f6a6d094640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a68159900 0x7f6a6814b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.351+0000 7f6a6d094640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 0x7f6a68160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.351+0000 7f6a6d094640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680b6120 con 0x7f6a6814bd20 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.351+0000 7f6a6d094640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680b5fa0 con 0x7f6a50004800 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.351+0000 7f6a6d094640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6a680b62a0 con 0x7f6a68159900 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 0x7f6a68160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 0x7f6a68160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55072/0 (socket says 172.21.15.22:55072) 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 -- 172.21.15.22:0/1285233215 learned_addr learned my addr 172.21.15.22:0/1285233215 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6659c640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a68159900 0x7f6a6814b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a66d9d640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a50004800 0x7f6a680ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 -- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a68159900 msgr2=0x7f6a6814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a68159900 0x7f6a6814b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 -- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a50004800 msgr2=0x7f6a680ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:20.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a50004800 0x7f6a680ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 -- 172.21.15.22:0/1285233215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6814c130 con 0x7f6a6814bd20 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6759e640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 0x7f6a68160fb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6a680d5590 tx=0x7f6a60011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000fca0 con 0x7f6a6814bd20 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a60004030 con 0x7f6a6814bd20 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6815f4d0 con 0x7f6a6814bd20 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600044c0 con 0x7f6a6814bd20 2024-09-21T05:46:20.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.352+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a680c9c90 con 0x7f6a6814bd20 2024-09-21T05:46:20.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.354+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6814c300 con 0x7f6a6814bd20 2024-09-21T05:46:20.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.354+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a600035f0 con 0x7f6a6814bd20 2024-09-21T05:46:20.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.354+0000 7f6a4f7fe640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a54078b90 0x7f6a5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:20.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.355+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f6a6009b9b0 con 0x7f6a6814bd20 2024-09-21T05:46:20.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.355+0000 7f6a66d9d640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a54078b90 0x7f6a5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:20.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.355+0000 7f6a66d9d640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a54078b90 0x7f6a5407b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6a580016d0 tx=0x7f6a58006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:20.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.359+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a60063700 con 0x7f6a6814bd20 2024-09-21T05:46:20.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:20.586+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6a680b6800 con 0x7f6a6814bd20 2024-09-21T05:46:21.213 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.211+0000 7f6a4f7fe640 1 -- 172.21.15.22:0/1285233215 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7f6a60068130 con 0x7f6a6814bd20 2024-09-21T05:46:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a54078b90 msgr2=0x7f6a5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a54078b90 0x7f6a5407b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f6a580016d0 tx=0x7f6a58006b70 comp rx=0 tx=0).stop 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 msgr2=0x7f6a68160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 0x7f6a68160fb0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6a680d5590 tx=0x7f6a60011e10 comp rx=0 tx=0).stop 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 shutdown_connections 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6a6814bd20 0x7f6a68160fb0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6a54078b90 0x7f6a5407b0a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6a68159900 0x7f6a6814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 --2- 172.21.15.22:0/1285233215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6a50004800 0x7f6a680ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 >> 172.21.15.22:0/1285233215 conn(0x7f6a6809d830 msgr2=0x7f6a6806e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 shutdown_connections 2024-09-21T05:46:21.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.214+0000 7f6a6d094640 1 -- 172.21.15.22:0/1285233215 wait complete. 2024-09-21T05:46:21.230 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:46:21.231 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 -- 172.21.15.22:0/1299181023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640b9b60 msgr2=0x7f69640d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1299181023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640b9b60 0x7f69640d4d80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6954009820 tx=0x7f695402a440 comp rx=0 tx=0).stop 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 -- 172.21.15.22:0/1299181023 shutdown_connections 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1299181023 >> v2:172.21.15.134:3300/0 conn(0x7f69640cf540 0x7f69640b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1299181023 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640b9b60 0x7f69640d4d80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 -- 172.21.15.22:0/1299181023 >> 172.21.15.22:0/1299181023 conn(0x7f696409d7f0 msgr2=0x7f69640d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 -- 172.21.15.22:0/1299181023 shutdown_connections 2024-09-21T05:46:21.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.352+0000 7f6968c9b640 1 -- 172.21.15.22:0/1299181023 wait complete. 2024-09-21T05:46:21.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.354+0000 7f6968c9b640 1 Processor -- start 2024-09-21T05:46:21.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.354+0000 7f6968c9b640 1 -- start start 2024-09-21T05:46:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6968c9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69640b64e0 0x7f696415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6968c9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69640b9b60 0x7f69641f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6968c9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 0x7f69640c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6968c9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f69640d2e40 con 0x7f69640cf540 2024-09-21T05:46:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6968c9b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f69640d2cc0 con 0x7f69640b64e0 2024-09-21T05:46:21.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6968c9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f69640d2fc0 con 0x7f69640b9b60 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 0x7f69640c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 0x7f69640c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55090/0 (socket says 172.21.15.22:55090) 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 -- 172.21.15.22:0/1493706444 learned_addr learned my addr 172.21.15.22:0/1493706444 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f696259c640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69640b64e0 0x7f696415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6961d9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69640b9b60 0x7f69641f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 -- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69640b9b60 msgr2=0x7f69641f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69640b9b60 0x7f69641f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 -- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69640b64e0 msgr2=0x7f696415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69640b64e0 0x7f696415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f6962d9d640 1 -- 172.21.15.22:0/1493706444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6964154100 con 0x7f69640cf540 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.356+0000 7f696259c640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69640b64e0 0x7f696415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.357+0000 7f6962d9d640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 0x7f69640c9800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c760 tx=0x7f695c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:21.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.357+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c003da0 con 0x7f69640cf540 2024-09-21T05:46:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.357+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f695c00ce10 con 0x7f69640cf540 2024-09-21T05:46:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.357+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69641543e0 con 0x7f69640cf540 2024-09-21T05:46:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.357+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c0052d0 con 0x7f69640cf540 2024-09-21T05:46:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.357+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696415e2d0 con 0x7f69640cf540 2024-09-21T05:46:21.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.358+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69640ceb70 con 0x7f69640cf540 2024-09-21T05:46:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.358+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f695c005470 con 0x7f69640cf540 2024-09-21T05:46:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.359+0000 7f6952ffd640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6944078bc0 0x7f694407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.359+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f695c09aae0 con 0x7f69640cf540 2024-09-21T05:46:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.359+0000 7f696259c640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6944078bc0 0x7f694407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.360+0000 7f696259c640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6944078bc0 0x7f694407b0d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6954034000 tx=0x7f6954004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:21.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.364+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69640ceb70 con 0x7f69640cf540 2024-09-21T05:46:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.590+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f696415e530 con 0x7f69640cf540 2024-09-21T05:46:21.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.590+0000 7f6952ffd640 1 -- 172.21.15.22:0/1493706444 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5231 (secure 0 0 0) 0x7f69640b68f0 con 0x7f69640cf540 2024-09-21T05:46:21.592 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:21.593 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"btime":"2024-09-21T05:46:21:204567+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:46:20.588210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7482":{"gid":7482,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1062944206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1062944206},{"type":"v1","addr":"172.21.15.22:6834","nonce":1062944206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7482,"qdb_cluster":[7482]},"id":7}]} 2024-09-21T05:46:21.593 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 84 2024-09-21T05:46:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6944078bc0 msgr2=0x7f694407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6944078bc0 0x7f694407b0d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f6954034000 tx=0x7f6954004730 comp rx=0 tx=0).stop 2024-09-21T05:46:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 msgr2=0x7f69640c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 0x7f69640c9800 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f695c00c760 tx=0x7f695c00cc20 comp rx=0 tx=0).stop 2024-09-21T05:46:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 shutdown_connections 2024-09-21T05:46:21.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f69640cf540 0x7f69640c9800 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6944078bc0 0x7f694407b0d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f69640b9b60 0x7f69641f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 --2- 172.21.15.22:0/1493706444 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f69640b64e0 0x7f696415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 >> 172.21.15.22:0/1493706444 conn(0x7f696409d7f0 msgr2=0x7f69640d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 shutdown_connections 2024-09-21T05:46:21.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.593+0000 7f6968c9b640 1 -- 172.21.15.22:0/1493706444 wait complete. 2024-09-21T05:46:21.607 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:46:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 -- 172.21.15.22:0/1660621580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24280cf540 msgr2=0x7f242806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 --2- 172.21.15.22:0/1660621580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24280cf540 0x7f242806eab0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f2420005fd0 tx=0x7f242002a980 comp rx=0 tx=0).stop 2024-09-21T05:46:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 -- 172.21.15.22:0/1660621580 shutdown_connections 2024-09-21T05:46:21.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 --2- 172.21.15.22:0/1660621580 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24280cf540 0x7f242806eab0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 --2- 172.21.15.22:0/1660621580 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24280b66b0 0x7f24280d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 -- 172.21.15.22:0/1660621580 >> 172.21.15.22:0/1660621580 conn(0x7f242809d7f0 msgr2=0x7f24280d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:21.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 -- 172.21.15.22:0/1660621580 shutdown_connections 2024-09-21T05:46:21.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.721+0000 7f242e77b640 1 -- 172.21.15.22:0/1660621580 wait complete. 2024-09-21T05:46:21.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.723+0000 7f242e77b640 1 Processor -- start 2024-09-21T05:46:21.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.723+0000 7f242e77b640 1 -- start start 2024-09-21T05:46:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242e77b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24280b9b60 0x7f24280ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242e77b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24280cf540 0x7f24281e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242e77b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 0x7f24281f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242e77b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f242806f5d0 con 0x7f24281f8870 2024-09-21T05:46:21.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242e77b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f242806f450 con 0x7f24280cf540 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242e77b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f242806f750 con 0x7f24280b9b60 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242cd18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 0x7f24281f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242cd18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 0x7f24281f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55108/0 (socket says 172.21.15.22:55108) 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f242cd18640 1 -- 172.21.15.22:0/2654202153 learned_addr learned my addr 172.21.15.22:0/2654202153 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f2427fff640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24280b9b60 0x7f24280ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.724+0000 7f24277fe640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24280cf540 0x7f24281e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242cd18640 1 -- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24280b9b60 msgr2=0x7f24280ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242cd18640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24280b9b60 0x7f24280ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242cd18640 1 -- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24280cf540 msgr2=0x7f24281e9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242cd18640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24280cf540 0x7f24281e9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242cd18640 1 -- 172.21.15.22:0/2654202153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24281f9710 con 0x7f24281f8870 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f24277fe640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24280cf540 0x7f24281e9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:21.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242cd18640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 0x7f24281f8c80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2420033040 tx=0x7f2420003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420034040 con 0x7f24281f8870 2024-09-21T05:46:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24200045e0 con 0x7f24281f8870 2024-09-21T05:46:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24281f91c0 con 0x7f24281f8870 2024-09-21T05:46:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420045480 con 0x7f24281f8870 2024-09-21T05:46:21.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.725+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24280c9500 con 0x7f24281f8870 2024-09-21T05:46:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.726+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242806fc70 con 0x7f24281f8870 2024-09-21T05:46:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.727+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2420037020 con 0x7f24281f8870 2024-09-21T05:46:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.727+0000 7f2424ff9640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2414078b80 0x7f241407b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.728+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f24200bc6a0 con 0x7f24281f8870 2024-09-21T05:46:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.728+0000 7f2427fff640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2414078b80 0x7f241407b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:21.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.728+0000 7f2427fff640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2414078b80 0x7f241407b090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2418000fd0 tx=0x7f2418009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:21.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.732+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2420084af0 con 0x7f24281f8870 2024-09-21T05:46:21.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.919+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24280d3fb0 con 0x7f24281f8870 2024-09-21T05:46:21.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.921+0000 7f2424ff9640 1 -- 172.21.15.22:0/2654202153 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+16656 (secure 0 0 0) 0x7f24200843f0 con 0x7f24281f8870 2024-09-21T05:46:21.923 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:21.923 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":97,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:46:19.179164+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:46:15.254551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T05:46:16.522802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:46:21.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2414078b80 msgr2=0x7f241407b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2414078b80 0x7f241407b090 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2418000fd0 tx=0x7f2418009040 comp rx=0 tx=0).stop 2024-09-21T05:46:21.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 msgr2=0x7f24281f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:21.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 0x7f24281f8c80 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f2420033040 tx=0x7f2420003f90 comp rx=0 tx=0).stop 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 shutdown_connections 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f24281f8870 0x7f24281f8c80 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f24280cf540 0x7f24281e9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2414078b80 0x7f241407b090 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 --2- 172.21.15.22:0/2654202153 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f24280b9b60 0x7f24280ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 >> 172.21.15.22:0/2654202153 conn(0x7f242809d7f0 msgr2=0x7f24280b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 shutdown_connections 2024-09-21T05:46:21.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:21.923+0000 7f242e77b640 1 -- 172.21.15.22:0/2654202153 wait complete. 2024-09-21T05:46:21.937 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:46:21.937 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:46:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 -- 172.21.15.22:0/2961621839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940706b0 msgr2=0x7fb4940cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 --2- 172.21.15.22:0/2961621839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940706b0 0x7fb4940cba00 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb488009830 tx=0x7fb48802a5c0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 -- 172.21.15.22:0/2961621839 shutdown_connections 2024-09-21T05:46:22.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 --2- 172.21.15.22:0/2961621839 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940706b0 0x7fb4940cba00 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 -- 172.21.15.22:0/2961621839 >> 172.21.15.22:0/2961621839 conn(0x7fb4940b1cb0 msgr2=0x7fb4940b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 -- 172.21.15.22:0/2961621839 shutdown_connections 2024-09-21T05:46:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.051+0000 7fb49b582640 1 -- 172.21.15.22:0/2961621839 wait complete. 2024-09-21T05:46:22.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.053+0000 7fb49b582640 1 Processor -- start 2024-09-21T05:46:22.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.053+0000 7fb49b582640 1 -- start start 2024-09-21T05:46:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49b582640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 0x7fb4941f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49b582640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4940cad30 0x7fb494158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49b582640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4941f42c0 0x7fb49415e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49b582640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb494074130 con 0x7fb4940ca800 2024-09-21T05:46:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49b582640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb494073fb0 con 0x7fb4941f42c0 2024-09-21T05:46:22.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49b582640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb4940742b0 con 0x7fb4940cad30 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 0x7fb4941f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 0x7fb4941f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55124/0 (socket says 172.21.15.22:55124) 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 -- 172.21.15.22:0/3075346156 learned_addr learned my addr 172.21.15.22:0/3075346156 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb499b1f640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4941f42c0 0x7fb49415e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb498b1d640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4940cad30 0x7fb494158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 -- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4940cad30 msgr2=0x7fb494158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4940cad30 0x7fb494158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 -- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4941f42c0 msgr2=0x7fb49415e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4941f42c0 0x7fb49415e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.054+0000 7fb49931e640 1 -- 172.21.15.22:0/3075346156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4941587d0 con 0x7fb4940ca800 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.055+0000 7fb49931e640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 0x7fb4941f7680 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb48400c3a0 tx=0x7fb48400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.055+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484017070 con 0x7fb4940ca800 2024-09-21T05:46:22.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.055+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4840053c0 con 0x7fb4940ca800 2024-09-21T05:46:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.055+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941f3820 con 0x7fb4940ca800 2024-09-21T05:46:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.055+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484003850 con 0x7fb4940ca800 2024-09-21T05:46:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.055+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941f3ac0 con 0x7fb4940ca800 2024-09-21T05:46:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.056+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4940b1290 con 0x7fb4940ca800 2024-09-21T05:46:22.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.057+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb48401a070 con 0x7fb4940ca800 2024-09-21T05:46:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.057+0000 7fb491ffb640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.058+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fb48409a170 con 0x7fb4940ca800 2024-09-21T05:46:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.058+0000 7fb498b1d640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.058+0000 7fb498b1d640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb488034000 tx=0x7fb488004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:22.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.062+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4940b1290 con 0x7fb4940ca800 2024-09-21T05:46:22.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.246+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb4940747d0 con 0x7fb480078bc0 2024-09-21T05:46:22.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.251+0000 7fb491ffb640 1 -- 172.21.15.22:0/3075346156 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7fb4940747d0 con 0x7fb480078bc0 2024-09-21T05:46:22.256 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:22.257 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 msgr2=0x7fb48007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb488034000 tx=0x7fb488004730 comp rx=0 tx=0).stop 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 msgr2=0x7fb4941f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 0x7fb4941f7680 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb48400c3a0 tx=0x7fb48400c860 comp rx=0 tx=0).stop 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 shutdown_connections 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb4941f42c0 0x7fb49415e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb480078bc0 0x7fb48007b0d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb4940cad30 0x7fb494158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 --2- 172.21.15.22:0/3075346156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb4940ca800 0x7fb4941f7680 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.254+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 >> 172.21.15.22:0/3075346156 conn(0x7fb4940b1cb0 msgr2=0x7fb4940b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:22.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.255+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 shutdown_connections 2024-09-21T05:46:22.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.255+0000 7fb49b582640 1 -- 172.21.15.22:0/3075346156 wait complete. 2024-09-21T05:46:22.268 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-21T05:46:21.851001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276700,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698020,"statfs":{"total":773094113280,"available":772810772480,"internally_reserved":0,"allocated":10788864,"data_stored":5299636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.681980"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.182932+0000","last_change":"2024-09-21T05:46:18.198277+0000","last_active":"2024-09-21T05:46:19.182932+0000","last_peered":"2024-09-21T05:46:19.182932+0000","last_clean":"2024-09-21T05:46:19.182932+0000","last_became_active":"2024-09-21T05:46:18.197795+0000","last_became_peered":"2024-09-21T05:46:18.197795+0000","last_unstale":"2024-09-21T05:46:19.182932+0000","last_undegraded":"2024-09-21T05:46:19.182932+0000","last_fullsized":"2024-09-21T05:46:19.182932+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:47.260759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199768+0000","last_change":"2024-09-21T05:46:18.195066+0000","last_active":"2024-09-21T05:46:19.199768+0000","last_peered":"2024-09-21T05:46:19.199768+0000","last_clean":"2024-09-21T05:46:19.199768+0000","last_became_active":"2024-09-21T05:46:18.194408+0000","last_became_peered":"2024-09-21T05:46:18.194408+0000","last_unstale":"2024-09-21T05:46:19.199768+0000","last_undegraded":"2024-09-21T05:46:19.199768+0000","last_fullsized":"2024-09-21T05:46:19.199768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:11.625271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183093+0000","last_change":"2024-09-21T05:46:18.198417+0000","last_active":"2024-09-21T05:46:19.183093+0000","last_peered":"2024-09-21T05:46:19.183093+0000","last_clean":"2024-09-21T05:46:19.183093+0000","last_became_active":"2024-09-21T05:46:18.197972+0000","last_became_peered":"2024-09-21T05:46:18.197972+0000","last_unstale":"2024-09-21T05:46:19.183093+0000","last_undegraded":"2024-09-21T05:46:19.183093+0000","last_fullsized":"2024-09-21T05:46:19.183093+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:35.955653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443250+0000","last_change":"2024-09-21T05:46:18.202921+0000","last_active":"2024-09-21T05:46:19.443250+0000","last_peered":"2024-09-21T05:46:19.443250+0000","last_clean":"2024-09-21T05:46:19.443250+0000","last_became_active":"2024-09-21T05:46:18.202580+0000","last_became_peered":"2024-09-21T05:46:18.202580+0000","last_unstale":"2024-09-21T05:46:19.443250+0000","last_undegraded":"2024-09-21T05:46:19.443250+0000","last_fullsized":"2024-09-21T05:46:19.443250+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:46.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187902+0000","last_change":"2024-09-21T05:46:18.199739+0000","last_active":"2024-09-21T05:46:19.187902+0000","last_peered":"2024-09-21T05:46:19.187902+0000","last_clean":"2024-09-21T05:46:19.187902+0000","last_became_active":"2024-09-21T05:46:18.199413+0000","last_became_peered":"2024-09-21T05:46:18.199413+0000","last_unstale":"2024-09-21T05:46:19.187902+0000","last_undegraded":"2024-09-21T05:46:19.187902+0000","last_fullsized":"2024-09-21T05:46:19.187902+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:33.816770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443836+0000","last_change":"2024-09-21T05:46:18.191732+0000","last_active":"2024-09-21T05:46:19.443836+0000","last_peered":"2024-09-21T05:46:19.443836+0000","last_clean":"2024-09-21T05:46:19.443836+0000","last_became_active":"2024-09-21T05:46:18.191276+0000","last_became_peered":"2024-09-21T05:46:18.191276+0000","last_unstale":"2024-09-21T05:46:19.443836+0000","last_undegraded":"2024-09-21T05:46:19.443836+0000","last_fullsized":"2024-09-21T05:46:19.443836+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:15.603230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443655+0000","last_change":"2024-09-21T05:46:18.193937+0000","last_active":"2024-09-21T05:46:19.443655+0000","last_peered":"2024-09-21T05:46:19.443655+0000","last_clean":"2024-09-21T05:46:19.443655+0000","last_became_active":"2024-09-21T05:46:18.193597+0000","last_became_peered":"2024-09-21T05:46:18.193597+0000","last_unstale":"2024-09-21T05:46:19.443655+0000","last_undegraded":"2024-09-21T05:46:19.443655+0000","last_fullsized":"2024-09-21T05:46:19.443655+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:36.619562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199003+0000","last_change":"2024-09-21T05:46:18.193242+0000","last_active":"2024-09-21T05:46:19.199003+0000","last_peered":"2024-09-21T05:46:19.199003+0000","last_clean":"2024-09-21T05:46:19.199003+0000","last_became_active":"2024-09-21T05:46:18.192900+0000","last_became_peered":"2024-09-21T05:46:18.192900+0000","last_unstale":"2024-09-21T05:46:19.199003+0000","last_undegraded":"2024-09-21T05:46:19.199003+0000","last_fullsized":"2024-09-21T05:46:19.199003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:22.274135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187656+0000","last_change":"2024-09-21T05:46:18.196117+0000","last_active":"2024-09-21T05:46:19.187656+0000","last_peered":"2024-09-21T05:46:19.187656+0000","last_clean":"2024-09-21T05:46:19.187656+0000","last_became_active":"2024-09-21T05:46:18.195812+0000","last_became_peered":"2024-09-21T05:46:18.195812+0000","last_unstale":"2024-09-21T05:46:19.187656+0000","last_undegraded":"2024-09-21T05:46:19.187656+0000","last_fullsized":"2024-09-21T05:46:19.187656+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:58.656688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186868+0000","last_change":"2024-09-21T05:46:18.201926+0000","last_active":"2024-09-21T05:46:19.186868+0000","last_peered":"2024-09-21T05:46:19.186868+0000","last_clean":"2024-09-21T05:46:19.186868+0000","last_became_active":"2024-09-21T05:46:18.201633+0000","last_became_peered":"2024-09-21T05:46:18.201633+0000","last_unstale":"2024-09-21T05:46:19.186868+0000","last_undegraded":"2024-09-21T05:46:19.186868+0000","last_fullsized":"2024-09-21T05:46:19.186868+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:23.952267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443652+0000","last_change":"2024-09-21T05:46:18.184560+0000","last_active":"2024-09-21T05:46:19.443652+0000","last_peered":"2024-09-21T05:46:19.443652+0000","last_clean":"2024-09-21T05:46:19.443652+0000","last_became_active":"2024-09-21T05:46:18.184134+0000","last_became_peered":"2024-09-21T05:46:18.184134+0000","last_unstale":"2024-09-21T05:46:19.443652+0000","last_undegraded":"2024-09-21T05:46:19.443652+0000","last_fullsized":"2024-09-21T05:46:19.443652+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:08.177819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188148+0000","last_change":"2024-09-21T05:46:18.193884+0000","last_active":"2024-09-21T05:46:19.188148+0000","last_peered":"2024-09-21T05:46:19.188148+0000","last_clean":"2024-09-21T05:46:19.188148+0000","last_became_active":"2024-09-21T05:46:18.193571+0000","last_became_peered":"2024-09-21T05:46:18.193571+0000","last_unstale":"2024-09-21T05:46:19.188148+0000","last_undegraded":"2024-09-21T05:46:19.188148+0000","last_fullsized":"2024-09-21T05:46:19.188148+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:41.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443177+0000","last_change":"2024-09-21T05:46:18.204644+0000","last_active":"2024-09-21T05:46:19.443177+0000","last_peered":"2024-09-21T05:46:19.443177+0000","last_clean":"2024-09-21T05:46:19.443177+0000","last_became_active":"2024-09-21T05:46:18.204331+0000","last_became_peered":"2024-09-21T05:46:18.204331+0000","last_unstale":"2024-09-21T05:46:19.443177+0000","last_undegraded":"2024-09-21T05:46:19.443177+0000","last_fullsized":"2024-09-21T05:46:19.443177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:12.799874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202086+0000","last_change":"2024-09-21T05:46:18.183826+0000","last_active":"2024-09-21T05:46:19.202086+0000","last_peered":"2024-09-21T05:46:19.202086+0000","last_clean":"2024-09-21T05:46:19.202086+0000","last_became_active":"2024-09-21T05:46:18.183495+0000","last_became_peered":"2024-09-21T05:46:18.183495+0000","last_unstale":"2024-09-21T05:46:19.202086+0000","last_undegraded":"2024-09-21T05:46:19.202086+0000","last_fullsized":"2024-09-21T05:46:19.202086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.749567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.442975+0000","last_change":"2024-09-21T05:46:18.206520+0000","last_active":"2024-09-21T05:46:19.442975+0000","last_peered":"2024-09-21T05:46:19.442975+0000","last_clean":"2024-09-21T05:46:19.442975+0000","last_became_active":"2024-09-21T05:46:18.206205+0000","last_became_peered":"2024-09-21T05:46:18.206205+0000","last_unstale":"2024-09-21T05:46:19.442975+0000","last_undegraded":"2024-09-21T05:46:19.442975+0000","last_fullsized":"2024-09-21T05:46:19.442975+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:33.075301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443471+0000","last_change":"2024-09-21T05:46:18.199208+0000","last_active":"2024-09-21T05:46:19.443471+0000","last_peered":"2024-09-21T05:46:19.443471+0000","last_clean":"2024-09-21T05:46:19.443471+0000","last_became_active":"2024-09-21T05:46:18.198902+0000","last_became_peered":"2024-09-21T05:46:18.198902+0000","last_unstale":"2024-09-21T05:46:19.443471+0000","last_undegraded":"2024-09-21T05:46:19.443471+0000","last_fullsized":"2024-09-21T05:46:19.443471+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:43.018538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199791+0000","last_change":"2024-09-21T05:46:18.195946+0000","last_active":"2024-09-21T05:46:19.199791+0000","last_peered":"2024-09-21T05:46:19.199791+0000","last_clean":"2024-09-21T05:46:19.199791+0000","last_became_active":"2024-09-21T05:46:18.195609+0000","last_became_peered":"2024-09-21T05:46:18.195609+0000","last_unstale":"2024-09-21T05:46:19.199791+0000","last_undegraded":"2024-09-21T05:46:19.199791+0000","last_fullsized":"2024-09-21T05:46:19.199791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:42.249069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443281+0000","last_change":"2024-09-21T05:46:18.202205+0000","last_active":"2024-09-21T05:46:19.443281+0000","last_peered":"2024-09-21T05:46:19.443281+0000","last_clean":"2024-09-21T05:46:19.443281+0000","last_became_active":"2024-09-21T05:46:18.201857+0000","last_became_peered":"2024-09-21T05:46:18.201857+0000","last_unstale":"2024-09-21T05:46:19.443281+0000","last_undegraded":"2024-09-21T05:46:19.443281+0000","last_fullsized":"2024-09-21T05:46:19.443281+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.661160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202445+0000","last_change":"2024-09-21T05:46:18.202860+0000","last_active":"2024-09-21T05:46:19.202445+0000","last_peered":"2024-09-21T05:46:19.202445+0000","last_clean":"2024-09-21T05:46:19.202445+0000","last_became_active":"2024-09-21T05:46:18.202526+0000","last_became_peered":"2024-09-21T05:46:18.202526+0000","last_unstale":"2024-09-21T05:46:19.202445+0000","last_undegraded":"2024-09-21T05:46:19.202445+0000","last_fullsized":"2024-09-21T05:46:19.202445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:08.335261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188188+0000","last_change":"2024-09-21T05:46:18.196489+0000","last_active":"2024-09-21T05:46:19.188188+0000","last_peered":"2024-09-21T05:46:19.188188+0000","last_clean":"2024-09-21T05:46:19.188188+0000","last_became_active":"2024-09-21T05:46:18.196169+0000","last_became_peered":"2024-09-21T05:46:18.196169+0000","last_unstale":"2024-09-21T05:46:19.188188+0000","last_undegraded":"2024-09-21T05:46:19.188188+0000","last_fullsized":"2024-09-21T05:46:19.188188+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:36.619562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202299+0000","last_change":"2024-09-21T05:46:18.192957+0000","last_active":"2024-09-21T05:46:19.202299+0000","last_peered":"2024-09-21T05:46:19.202299+0000","last_clean":"2024-09-21T05:46:19.202299+0000","last_became_active":"2024-09-21T05:46:18.192529+0000","last_became_peered":"2024-09-21T05:46:18.192529+0000","last_unstale":"2024-09-21T05:46:19.202299+0000","last_undegraded":"2024-09-21T05:46:19.202299+0000","last_fullsized":"2024-09-21T05:46:19.202299+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:19.102782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.198982+0000","last_change":"2024-09-21T05:46:18.195762+0000","last_active":"2024-09-21T05:46:19.198982+0000","last_peered":"2024-09-21T05:46:19.198982+0000","last_clean":"2024-09-21T05:46:19.198982+0000","last_became_active":"2024-09-21T05:46:18.195405+0000","last_became_peered":"2024-09-21T05:46:18.195405+0000","last_unstale":"2024-09-21T05:46:19.198982+0000","last_undegraded":"2024-09-21T05:46:19.198982+0000","last_fullsized":"2024-09-21T05:46:19.198982+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:25.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.198952+0000","last_change":"2024-09-21T05:46:18.193981+0000","last_active":"2024-09-21T05:46:19.198952+0000","last_peered":"2024-09-21T05:46:19.198952+0000","last_clean":"2024-09-21T05:46:19.198952+0000","last_became_active":"2024-09-21T05:46:18.193624+0000","last_became_peered":"2024-09-21T05:46:18.193624+0000","last_unstale":"2024-09-21T05:46:19.198952+0000","last_undegraded":"2024-09-21T05:46:19.198952+0000","last_fullsized":"2024-09-21T05:46:19.198952+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:49.104631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443392+0000","last_change":"2024-09-21T05:46:18.200577+0000","last_active":"2024-09-21T05:46:19.443392+0000","last_peered":"2024-09-21T05:46:19.443392+0000","last_clean":"2024-09-21T05:46:19.443392+0000","last_became_active":"2024-09-21T05:46:18.199923+0000","last_became_peered":"2024-09-21T05:46:18.199923+0000","last_unstale":"2024-09-21T05:46:19.443392+0000","last_undegraded":"2024-09-21T05:46:19.443392+0000","last_fullsized":"2024-09-21T05:46:19.443392+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:00.821372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186979+0000","last_change":"2024-09-21T05:46:18.197641+0000","last_active":"2024-09-21T05:46:19.186979+0000","last_peered":"2024-09-21T05:46:19.186979+0000","last_clean":"2024-09-21T05:46:19.186979+0000","last_became_active":"2024-09-21T05:46:18.197293+0000","last_became_peered":"2024-09-21T05:46:18.197293+0000","last_unstale":"2024-09-21T05:46:19.186979+0000","last_undegraded":"2024-09-21T05:46:19.186979+0000","last_fullsized":"2024-09-21T05:46:19.186979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:11.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199764+0000","last_change":"2024-09-21T05:46:18.200222+0000","last_active":"2024-09-21T05:46:19.199764+0000","last_peered":"2024-09-21T05:46:19.199764+0000","last_clean":"2024-09-21T05:46:19.199764+0000","last_became_active":"2024-09-21T05:46:18.199867+0000","last_became_peered":"2024-09-21T05:46:18.199867+0000","last_unstale":"2024-09-21T05:46:19.199764+0000","last_undegraded":"2024-09-21T05:46:19.199764+0000","last_fullsized":"2024-09-21T05:46:19.199764+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:00.234053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443913+0000","last_change":"2024-09-21T05:46:18.200719+0000","last_active":"2024-09-21T05:46:19.443913+0000","last_peered":"2024-09-21T05:46:19.443913+0000","last_clean":"2024-09-21T05:46:19.443913+0000","last_became_active":"2024-09-21T05:46:18.200358+0000","last_became_peered":"2024-09-21T05:46:18.200358+0000","last_unstale":"2024-09-21T05:46:19.443913+0000","last_undegraded":"2024-09-21T05:46:19.443913+0000","last_fullsized":"2024-09-21T05:46:19.443913+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:28.738432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188425+0000","last_change":"2024-09-21T05:46:18.195937+0000","last_active":"2024-09-21T05:46:19.188425+0000","last_peered":"2024-09-21T05:46:19.188425+0000","last_clean":"2024-09-21T05:46:19.188425+0000","last_became_active":"2024-09-21T05:46:18.195635+0000","last_became_peered":"2024-09-21T05:46:18.195635+0000","last_unstale":"2024-09-21T05:46:19.188425+0000","last_undegraded":"2024-09-21T05:46:19.188425+0000","last_fullsized":"2024-09-21T05:46:19.188425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:41.309563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199171+0000","last_change":"2024-09-21T05:46:18.196699+0000","last_active":"2024-09-21T05:46:19.199171+0000","last_peered":"2024-09-21T05:46:19.199171+0000","last_clean":"2024-09-21T05:46:19.199171+0000","last_became_active":"2024-09-21T05:46:18.196368+0000","last_became_peered":"2024-09-21T05:46:18.196368+0000","last_unstale":"2024-09-21T05:46:19.199171+0000","last_undegraded":"2024-09-21T05:46:19.199171+0000","last_fullsized":"2024-09-21T05:46:19.199171+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:26.119238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202350+0000","last_change":"2024-09-21T05:46:18.193817+0000","last_active":"2024-09-21T05:46:19.202350+0000","last_peered":"2024-09-21T05:46:19.202350+0000","last_clean":"2024-09-21T05:46:19.202350+0000","last_became_active":"2024-09-21T05:46:18.192992+0000","last_became_peered":"2024-09-21T05:46:18.192992+0000","last_unstale":"2024-09-21T05:46:19.202350+0000","last_undegraded":"2024-09-21T05:46:19.202350+0000","last_fullsized":"2024-09-21T05:46:19.202350+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:23.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443282+0000","last_change":"2024-09-21T05:46:18.197180+0000","last_active":"2024-09-21T05:46:19.443282+0000","last_peered":"2024-09-21T05:46:19.443282+0000","last_clean":"2024-09-21T05:46:19.443282+0000","last_became_active":"2024-09-21T05:46:18.196831+0000","last_became_peered":"2024-09-21T05:46:18.196831+0000","last_unstale":"2024-09-21T05:46:19.443282+0000","last_undegraded":"2024-09-21T05:46:19.443282+0000","last_fullsized":"2024-09-21T05:46:19.443282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.723228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183109+0000","last_change":"2024-09-21T05:46:18.197226+0000","last_active":"2024-09-21T05:46:19.183109+0000","last_peered":"2024-09-21T05:46:19.183109+0000","last_clean":"2024-09-21T05:46:19.183109+0000","last_became_active":"2024-09-21T05:46:18.196492+0000","last_became_peered":"2024-09-21T05:46:18.196492+0000","last_unstale":"2024-09-21T05:46:19.183109+0000","last_undegraded":"2024-09-21T05:46:19.183109+0000","last_fullsized":"2024-09-21T05:46:19.183109+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:46.715787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202686+0000","last_change":"2024-09-21T05:46:18.191966+0000","last_active":"2024-09-21T05:46:19.202686+0000","last_peered":"2024-09-21T05:46:19.202686+0000","last_clean":"2024-09-21T05:46:19.202686+0000","last_became_active":"2024-09-21T05:46:18.190860+0000","last_became_peered":"2024-09-21T05:46:18.190860+0000","last_unstale":"2024-09-21T05:46:19.202686+0000","last_undegraded":"2024-09-21T05:46:19.202686+0000","last_fullsized":"2024-09-21T05:46:19.202686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:48.756997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199710+0000","last_change":"2024-09-21T05:46:17.197005+0000","last_active":"2024-09-21T05:46:19.199710+0000","last_peered":"2024-09-21T05:46:19.199710+0000","last_clean":"2024-09-21T05:46:19.199710+0000","last_became_active":"2024-09-21T05:46:17.196527+0000","last_became_peered":"2024-09-21T05:46:17.196527+0000","last_unstale":"2024-09-21T05:46:19.199710+0000","last_undegraded":"2024-09-21T05:46:19.199710+0000","last_fullsized":"2024-09-21T05:46:19.199710+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:15.104091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443477+0000","last_change":"2024-09-21T05:46:18.190900+0000","last_active":"2024-09-21T05:46:19.443477+0000","last_peered":"2024-09-21T05:46:19.443477+0000","last_clean":"2024-09-21T05:46:19.443477+0000","last_became_active":"2024-09-21T05:46:18.190520+0000","last_became_peered":"2024-09-21T05:46:18.190520+0000","last_unstale":"2024-09-21T05:46:19.443477+0000","last_undegraded":"2024-09-21T05:46:19.443477+0000","last_fullsized":"2024-09-21T05:46:19.443477+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:22.634498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443423+0000","last_change":"2024-09-21T05:46:17.189404+0000","last_active":"2024-09-21T05:46:19.443423+0000","last_peered":"2024-09-21T05:46:19.443423+0000","last_clean":"2024-09-21T05:46:19.443423+0000","last_became_active":"2024-09-21T05:46:17.189086+0000","last_became_peered":"2024-09-21T05:46:17.189086+0000","last_unstale":"2024-09-21T05:46:19.443423+0000","last_undegraded":"2024-09-21T05:46:19.443423+0000","last_fullsized":"2024-09-21T05:46:19.443423+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:08.756865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183057+0000","last_change":"2024-09-21T05:46:18.188710+0000","last_active":"2024-09-21T05:46:19.183057+0000","last_peered":"2024-09-21T05:46:19.183057+0000","last_clean":"2024-09-21T05:46:19.183057+0000","last_became_active":"2024-09-21T05:46:18.188275+0000","last_became_peered":"2024-09-21T05:46:18.188275+0000","last_unstale":"2024-09-21T05:46:19.183057+0000","last_undegraded":"2024-09-21T05:46:19.183057+0000","last_fullsized":"2024-09-21T05:46:19.183057+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:07.835349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199551+0000","last_change":"2024-09-21T05:46:17.201433+0000","last_active":"2024-09-21T05:46:19.199551+0000","last_peered":"2024-09-21T05:46:19.199551+0000","last_clean":"2024-09-21T05:46:19.199551+0000","last_became_active":"2024-09-21T05:46:17.199344+0000","last_became_peered":"2024-09-21T05:46:17.199344+0000","last_unstale":"2024-09-21T05:46:19.199551+0000","last_undegraded":"2024-09-21T05:46:19.199551+0000","last_fullsized":"2024-09-21T05:46:19.199551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:48.069757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188132+0000","last_change":"2024-09-21T05:46:18.190968+0000","last_active":"2024-09-21T05:46:19.188132+0000","last_peered":"2024-09-21T05:46:19.188132+0000","last_clean":"2024-09-21T05:46:19.188132+0000","last_became_active":"2024-09-21T05:46:18.190665+0000","last_became_peered":"2024-09-21T05:46:18.190665+0000","last_unstale":"2024-09-21T05:46:19.188132+0000","last_undegraded":"2024-09-21T05:46:19.188132+0000","last_fullsized":"2024-09-21T05:46:19.188132+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:45.675703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199472+0000","last_change":"2024-09-21T05:46:17.199581+0000","last_active":"2024-09-21T05:46:19.199472+0000","last_peered":"2024-09-21T05:46:19.199472+0000","last_clean":"2024-09-21T05:46:19.199472+0000","last_became_active":"2024-09-21T05:46:17.199068+0000","last_became_peered":"2024-09-21T05:46:17.199068+0000","last_unstale":"2024-09-21T05:46:19.199472+0000","last_undegraded":"2024-09-21T05:46:19.199472+0000","last_fullsized":"2024-09-21T05:46:19.199472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:57.941291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444250+0000","last_change":"2024-09-21T05:46:18.194581+0000","last_active":"2024-09-21T05:46:19.444250+0000","last_peered":"2024-09-21T05:46:19.444250+0000","last_clean":"2024-09-21T05:46:19.444250+0000","last_became_active":"2024-09-21T05:46:18.194174+0000","last_became_peered":"2024-09-21T05:46:18.194174+0000","last_unstale":"2024-09-21T05:46:19.444250+0000","last_undegraded":"2024-09-21T05:46:19.444250+0000","last_fullsized":"2024-09-21T05:46:19.444250+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:27.838115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.182967+0000","last_change":"2024-09-21T05:46:17.191438+0000","last_active":"2024-09-21T05:46:19.182967+0000","last_peered":"2024-09-21T05:46:19.182967+0000","last_clean":"2024-09-21T05:46:19.182967+0000","last_became_active":"2024-09-21T05:46:17.191136+0000","last_became_peered":"2024-09-21T05:46:17.191136+0000","last_unstale":"2024-09-21T05:46:19.182967+0000","last_undegraded":"2024-09-21T05:46:19.182967+0000","last_fullsized":"2024-09-21T05:46:19.182967+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.389691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443397+0000","last_change":"2024-09-21T05:46:18.187130+0000","last_active":"2024-09-21T05:46:19.443397+0000","last_peered":"2024-09-21T05:46:19.443397+0000","last_clean":"2024-09-21T05:46:19.443397+0000","last_became_active":"2024-09-21T05:46:18.186802+0000","last_became_peered":"2024-09-21T05:46:18.186802+0000","last_unstale":"2024-09-21T05:46:19.443397+0000","last_undegraded":"2024-09-21T05:46:19.443397+0000","last_fullsized":"2024-09-21T05:46:19.443397+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.698448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199116+0000","last_change":"2024-09-21T05:46:17.196630+0000","last_active":"2024-09-21T05:46:19.199116+0000","last_peered":"2024-09-21T05:46:19.199116+0000","last_clean":"2024-09-21T05:46:19.199116+0000","last_became_active":"2024-09-21T05:46:17.196319+0000","last_became_peered":"2024-09-21T05:46:17.196319+0000","last_unstale":"2024-09-21T05:46:19.199116+0000","last_undegraded":"2024-09-21T05:46:19.199116+0000","last_fullsized":"2024-09-21T05:46:19.199116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:59.652406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.442885+0000","last_change":"2024-09-21T05:46:18.207857+0000","last_active":"2024-09-21T05:46:19.442885+0000","last_peered":"2024-09-21T05:46:19.442885+0000","last_clean":"2024-09-21T05:46:19.442885+0000","last_became_active":"2024-09-21T05:46:18.207300+0000","last_became_peered":"2024-09-21T05:46:18.207300+0000","last_unstale":"2024-09-21T05:46:19.442885+0000","last_undegraded":"2024-09-21T05:46:19.442885+0000","last_fullsized":"2024-09-21T05:46:19.442885+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:02.670528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187162+0000","last_change":"2024-09-21T05:46:17.198061+0000","last_active":"2024-09-21T05:46:19.187162+0000","last_peered":"2024-09-21T05:46:19.187162+0000","last_clean":"2024-09-21T05:46:19.187162+0000","last_became_active":"2024-09-21T05:46:17.197745+0000","last_became_peered":"2024-09-21T05:46:17.197745+0000","last_unstale":"2024-09-21T05:46:19.187162+0000","last_undegraded":"2024-09-21T05:46:19.187162+0000","last_fullsized":"2024-09-21T05:46:19.187162+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:26.832106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186574+0000","last_change":"2024-09-21T05:46:18.192709+0000","last_active":"2024-09-21T05:46:19.186574+0000","last_peered":"2024-09-21T05:46:19.186574+0000","last_clean":"2024-09-21T05:46:19.186574+0000","last_became_active":"2024-09-21T05:46:18.191982+0000","last_became_peered":"2024-09-21T05:46:18.191982+0000","last_unstale":"2024-09-21T05:46:19.186574+0000","last_undegraded":"2024-09-21T05:46:19.186574+0000","last_fullsized":"2024-09-21T05:46:19.186574+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:06.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183220+0000","last_change":"2024-09-21T05:46:17.186502+0000","last_active":"2024-09-21T05:46:19.183220+0000","last_peered":"2024-09-21T05:46:19.183220+0000","last_clean":"2024-09-21T05:46:19.183220+0000","last_became_active":"2024-09-21T05:46:17.186162+0000","last_became_peered":"2024-09-21T05:46:17.186162+0000","last_unstale":"2024-09-21T05:46:19.183220+0000","last_undegraded":"2024-09-21T05:46:19.183220+0000","last_fullsized":"2024-09-21T05:46:19.183220+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:31.895594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199978+0000","last_change":"2024-09-21T05:46:18.194734+0000","last_active":"2024-09-21T05:46:19.199978+0000","last_peered":"2024-09-21T05:46:19.199978+0000","last_clean":"2024-09-21T05:46:19.199978+0000","last_became_active":"2024-09-21T05:46:18.194374+0000","last_became_peered":"2024-09-21T05:46:18.194374+0000","last_unstale":"2024-09-21T05:46:19.199978+0000","last_undegraded":"2024-09-21T05:46:19.199978+0000","last_fullsized":"2024-09-21T05:46:19.199978+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:52.503589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183115+0000","last_change":"2024-09-21T05:46:17.184439+0000","last_active":"2024-09-21T05:46:19.183115+0000","last_peered":"2024-09-21T05:46:19.183115+0000","last_clean":"2024-09-21T05:46:19.183115+0000","last_became_active":"2024-09-21T05:46:17.184111+0000","last_became_peered":"2024-09-21T05:46:17.184111+0000","last_unstale":"2024-09-21T05:46:19.183115+0000","last_undegraded":"2024-09-21T05:46:19.183115+0000","last_fullsized":"2024-09-21T05:46:19.183115+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:19.945815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187912+0000","last_change":"2024-09-21T05:46:18.192051+0000","last_active":"2024-09-21T05:46:19.187912+0000","last_peered":"2024-09-21T05:46:19.187912+0000","last_clean":"2024-09-21T05:46:19.187912+0000","last_became_active":"2024-09-21T05:46:18.191747+0000","last_became_peered":"2024-09-21T05:46:18.191747+0000","last_unstale":"2024-09-21T05:46:19.187912+0000","last_undegraded":"2024-09-21T05:46:19.187912+0000","last_fullsized":"2024-09-21T05:46:19.187912+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.698448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443290+0000","last_change":"2024-09-21T05:46:17.200131+0000","last_active":"2024-09-21T05:46:19.443290+0000","last_peered":"2024-09-21T05:46:19.443290+0000","last_clean":"2024-09-21T05:46:19.443290+0000","last_became_active":"2024-09-21T05:46:17.199798+0000","last_became_peered":"2024-09-21T05:46:17.199798+0000","last_unstale":"2024-09-21T05:46:19.443290+0000","last_undegraded":"2024-09-21T05:46:19.443290+0000","last_fullsized":"2024-09-21T05:46:19.443290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:19.897956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188415+0000","last_change":"2024-09-21T05:46:18.197717+0000","last_active":"2024-09-21T05:46:19.188415+0000","last_peered":"2024-09-21T05:46:19.188415+0000","last_clean":"2024-09-21T05:46:19.188415+0000","last_became_active":"2024-09-21T05:46:18.197320+0000","last_became_peered":"2024-09-21T05:46:18.197320+0000","last_unstale":"2024-09-21T05:46:19.188415+0000","last_undegraded":"2024-09-21T05:46:19.188415+0000","last_fullsized":"2024-09-21T05:46:19.188415+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.723228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444256+0000","last_change":"2024-09-21T05:46:17.190720+0000","last_active":"2024-09-21T05:46:19.444256+0000","last_peered":"2024-09-21T05:46:19.444256+0000","last_clean":"2024-09-21T05:46:19.444256+0000","last_became_active":"2024-09-21T05:46:17.190431+0000","last_became_peered":"2024-09-21T05:46:17.190431+0000","last_unstale":"2024-09-21T05:46:19.444256+0000","last_undegraded":"2024-09-21T05:46:19.444256+0000","last_fullsized":"2024-09-21T05:46:19.444256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:21.544930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444170+0000","last_change":"2024-09-21T05:46:18.197885+0000","last_active":"2024-09-21T05:46:19.444170+0000","last_peered":"2024-09-21T05:46:19.444170+0000","last_clean":"2024-09-21T05:46:19.444170+0000","last_became_active":"2024-09-21T05:46:18.197569+0000","last_became_peered":"2024-09-21T05:46:18.197569+0000","last_unstale":"2024-09-21T05:46:19.444170+0000","last_undegraded":"2024-09-21T05:46:19.444170+0000","last_fullsized":"2024-09-21T05:46:19.444170+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:53.119749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":21,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188279+0000","last_change":"2024-09-21T05:46:17.189645+0000","last_active":"2024-09-21T05:46:19.188279+0000","last_peered":"2024-09-21T05:46:19.188279+0000","last_clean":"2024-09-21T05:46:19.188279+0000","last_became_active":"2024-09-21T05:46:17.189243+0000","last_became_peered":"2024-09-21T05:46:17.189243+0000","last_unstale":"2024-09-21T05:46:19.188279+0000","last_undegraded":"2024-09-21T05:46:19.188279+0000","last_fullsized":"2024-09-21T05:46:19.188279+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:38.660622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443159+0000","last_change":"2024-09-21T05:46:18.191157+0000","last_active":"2024-09-21T05:46:19.443159+0000","last_peered":"2024-09-21T05:46:19.443159+0000","last_clean":"2024-09-21T05:46:19.443159+0000","last_became_active":"2024-09-21T05:46:18.190646+0000","last_became_peered":"2024-09-21T05:46:18.190646+0000","last_unstale":"2024-09-21T05:46:19.443159+0000","last_undegraded":"2024-09-21T05:46:19.443159+0000","last_fullsized":"2024-09-21T05:46:19.443159+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:22.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202377+0000","last_change":"2024-09-21T05:46:17.189107+0000","last_active":"2024-09-21T05:46:19.202377+0000","last_peered":"2024-09-21T05:46:19.202377+0000","last_clean":"2024-09-21T05:46:19.202377+0000","last_became_active":"2024-09-21T05:46:17.188753+0000","last_became_peered":"2024-09-21T05:46:17.188753+0000","last_unstale":"2024-09-21T05:46:19.202377+0000","last_undegraded":"2024-09-21T05:46:19.202377+0000","last_fullsized":"2024-09-21T05:46:19.202377+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:28.618603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183142+0000","last_change":"2024-09-21T05:46:18.191886+0000","last_active":"2024-09-21T05:46:19.183142+0000","last_peered":"2024-09-21T05:46:19.183142+0000","last_clean":"2024-09-21T05:46:19.183142+0000","last_became_active":"2024-09-21T05:46:18.191569+0000","last_became_peered":"2024-09-21T05:46:18.191569+0000","last_unstale":"2024-09-21T05:46:19.183142+0000","last_undegraded":"2024-09-21T05:46:19.183142+0000","last_fullsized":"2024-09-21T05:46:19.183142+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:31.838595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202491+0000","last_change":"2024-09-21T05:46:17.193638+0000","last_active":"2024-09-21T05:46:19.202491+0000","last_peered":"2024-09-21T05:46:19.202491+0000","last_clean":"2024-09-21T05:46:19.202491+0000","last_became_active":"2024-09-21T05:46:17.193269+0000","last_became_peered":"2024-09-21T05:46:17.193269+0000","last_unstale":"2024-09-21T05:46:19.202491+0000","last_undegraded":"2024-09-21T05:46:19.202491+0000","last_fullsized":"2024-09-21T05:46:19.202491+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:08.277143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186500+0000","last_change":"2024-09-21T05:46:18.190319+0000","last_active":"2024-09-21T05:46:19.186500+0000","last_peered":"2024-09-21T05:46:19.186500+0000","last_clean":"2024-09-21T05:46:19.186500+0000","last_became_active":"2024-09-21T05:46:18.189963+0000","last_became_peered":"2024-09-21T05:46:18.189963+0000","last_unstale":"2024-09-21T05:46:19.186500+0000","last_undegraded":"2024-09-21T05:46:19.186500+0000","last_fullsized":"2024-09-21T05:46:19.186500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:05.463973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186646+0000","last_change":"2024-09-21T05:46:17.188129+0000","last_active":"2024-09-21T05:46:19.186646+0000","last_peered":"2024-09-21T05:46:19.186646+0000","last_clean":"2024-09-21T05:46:19.186646+0000","last_became_active":"2024-09-21T05:46:17.187315+0000","last_became_peered":"2024-09-21T05:46:17.187315+0000","last_unstale":"2024-09-21T05:46:19.186646+0000","last_undegraded":"2024-09-21T05:46:19.186646+0000","last_fullsized":"2024-09-21T05:46:19.186646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.478577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202077+0000","last_change":"2024-09-21T05:46:18.191336+0000","last_active":"2024-09-21T05:46:19.202077+0000","last_peered":"2024-09-21T05:46:19.202077+0000","last_clean":"2024-09-21T05:46:19.202077+0000","last_became_active":"2024-09-21T05:46:18.188921+0000","last_became_peered":"2024-09-21T05:46:18.188921+0000","last_unstale":"2024-09-21T05:46:19.202077+0000","last_undegraded":"2024-09-21T05:46:19.202077+0000","last_fullsized":"2024-09-21T05:46:19.202077+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:25.785814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183288+0000","last_change":"2024-09-21T05:46:17.187693+0000","last_active":"2024-09-21T05:46:19.183288+0000","last_peered":"2024-09-21T05:46:19.183288+0000","last_clean":"2024-09-21T05:46:19.183288+0000","last_became_active":"2024-09-21T05:46:17.187378+0000","last_became_peered":"2024-09-21T05:46:17.187378+0000","last_unstale":"2024-09-21T05:46:19.183288+0000","last_undegraded":"2024-09-21T05:46:19.183288+0000","last_fullsized":"2024-09-21T05:46:19.183288+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:09.816160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187230+0000","last_change":"2024-09-21T05:46:18.189439+0000","last_active":"2024-09-21T05:46:19.187230+0000","last_peered":"2024-09-21T05:46:19.187230+0000","last_clean":"2024-09-21T05:46:19.187230+0000","last_became_active":"2024-09-21T05:46:18.189119+0000","last_became_peered":"2024-09-21T05:46:18.189119+0000","last_unstale":"2024-09-21T05:46:19.187230+0000","last_undegraded":"2024-09-21T05:46:19.187230+0000","last_fullsized":"2024-09-21T05:46:19.187230+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:01.443807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187276+0000","last_change":"2024-09-21T05:46:17.189048+0000","last_active":"2024-09-21T05:46:19.187276+0000","last_peered":"2024-09-21T05:46:19.187276+0000","last_clean":"2024-09-21T05:46:19.187276+0000","last_became_active":"2024-09-21T05:46:17.188753+0000","last_became_peered":"2024-09-21T05:46:17.188753+0000","last_unstale":"2024-09-21T05:46:19.187276+0000","last_undegraded":"2024-09-21T05:46:19.187276+0000","last_fullsized":"2024-09-21T05:46:19.187276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:26.332525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443934+0000","last_change":"2024-09-21T05:46:18.197100+0000","last_active":"2024-09-21T05:46:19.443934+0000","last_peered":"2024-09-21T05:46:19.443934+0000","last_clean":"2024-09-21T05:46:19.443934+0000","last_became_active":"2024-09-21T05:46:18.196788+0000","last_became_peered":"2024-09-21T05:46:18.196788+0000","last_unstale":"2024-09-21T05:46:19.443934+0000","last_undegraded":"2024-09-21T05:46:19.443934+0000","last_fullsized":"2024-09-21T05:46:19.443934+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:26.287510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187326+0000","last_change":"2024-09-21T05:46:17.184825+0000","last_active":"2024-09-21T05:46:19.187326+0000","last_peered":"2024-09-21T05:46:19.187326+0000","last_clean":"2024-09-21T05:46:19.187326+0000","last_became_active":"2024-09-21T05:46:17.184487+0000","last_became_peered":"2024-09-21T05:46:17.184487+0000","last_unstale":"2024-09-21T05:46:19.187326+0000","last_undegraded":"2024-09-21T05:46:19.187326+0000","last_fullsized":"2024-09-21T05:46:19.187326+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:09.619020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444338+0000","last_change":"2024-09-21T05:46:18.189396+0000","last_active":"2024-09-21T05:46:19.444338+0000","last_peered":"2024-09-21T05:46:19.444338+0000","last_clean":"2024-09-21T05:46:19.444338+0000","last_became_active":"2024-09-21T05:46:18.189073+0000","last_became_peered":"2024-09-21T05:46:18.189073+0000","last_unstale":"2024-09-21T05:46:19.444338+0000","last_undegraded":"2024-09-21T05:46:19.444338+0000","last_fullsized":"2024-09-21T05:46:19.444338+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:35.928619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199508+0000","last_change":"2024-09-21T05:46:17.199865+0000","last_active":"2024-09-21T05:46:19.199508+0000","last_peered":"2024-09-21T05:46:19.199508+0000","last_clean":"2024-09-21T05:46:19.199508+0000","last_became_active":"2024-09-21T05:46:17.198799+0000","last_became_peered":"2024-09-21T05:46:17.198799+0000","last_unstale":"2024-09-21T05:46:19.199508+0000","last_undegraded":"2024-09-21T05:46:19.199508+0000","last_fullsized":"2024-09-21T05:46:19.199508+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:22.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202342+0000","last_change":"2024-09-21T05:46:18.194951+0000","last_active":"2024-09-21T05:46:19.202342+0000","last_peered":"2024-09-21T05:46:19.202342+0000","last_clean":"2024-09-21T05:46:19.202342+0000","last_became_active":"2024-09-21T05:46:18.194626+0000","last_became_peered":"2024-09-21T05:46:18.194626+0000","last_unstale":"2024-09-21T05:46:19.202342+0000","last_undegraded":"2024-09-21T05:46:19.202342+0000","last_fullsized":"2024-09-21T05:46:19.202342+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:34.154658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199283+0000","last_change":"2024-09-21T05:46:17.198720+0000","last_active":"2024-09-21T05:46:19.199283+0000","last_peered":"2024-09-21T05:46:19.199283+0000","last_clean":"2024-09-21T05:46:19.199283+0000","last_became_active":"2024-09-21T05:46:17.197128+0000","last_became_peered":"2024-09-21T05:46:17.197128+0000","last_unstale":"2024-09-21T05:46:19.199283+0000","last_undegraded":"2024-09-21T05:46:19.199283+0000","last_fullsized":"2024-09-21T05:46:19.199283+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:15.132838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202361+0000","last_change":"2024-09-21T05:46:18.182810+0000","last_active":"2024-09-21T05:46:19.202361+0000","last_peered":"2024-09-21T05:46:19.202361+0000","last_clean":"2024-09-21T05:46:19.202361+0000","last_became_active":"2024-09-21T05:46:18.182468+0000","last_became_peered":"2024-09-21T05:46:18.182468+0000","last_unstale":"2024-09-21T05:46:19.202361+0000","last_undegraded":"2024-09-21T05:46:19.202361+0000","last_fullsized":"2024-09-21T05:46:19.202361+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:27.025794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444170+0000","last_change":"2024-09-21T05:46:17.186773+0000","last_active":"2024-09-21T05:46:19.444170+0000","last_peered":"2024-09-21T05:46:19.444170+0000","last_clean":"2024-09-21T05:46:19.444170+0000","last_became_active":"2024-09-21T05:46:17.186333+0000","last_became_peered":"2024-09-21T05:46:17.186333+0000","last_unstale":"2024-09-21T05:46:19.444170+0000","last_undegraded":"2024-09-21T05:46:19.444170+0000","last_fullsized":"2024-09-21T05:46:19.444170+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:27.896430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202455+0000","last_change":"2024-09-21T05:46:18.201418+0000","last_active":"2024-09-21T05:46:19.202455+0000","last_peered":"2024-09-21T05:46:19.202455+0000","last_clean":"2024-09-21T05:46:19.202455+0000","last_became_active":"2024-09-21T05:46:18.201051+0000","last_became_peered":"2024-09-21T05:46:18.201051+0000","last_unstale":"2024-09-21T05:46:19.202455+0000","last_undegraded":"2024-09-21T05:46:19.202455+0000","last_fullsized":"2024-09-21T05:46:19.202455+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:36.520750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443657+0000","last_change":"2024-09-21T05:46:17.189743+0000","last_active":"2024-09-21T05:46:19.443657+0000","last_peered":"2024-09-21T05:46:19.443657+0000","last_clean":"2024-09-21T05:46:19.443657+0000","last_became_active":"2024-09-21T05:46:17.189441+0000","last_became_peered":"2024-09-21T05:46:17.189441+0000","last_unstale":"2024-09-21T05:46:19.443657+0000","last_undegraded":"2024-09-21T05:46:19.443657+0000","last_fullsized":"2024-09-21T05:46:19.443657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:44.778540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186788+0000","last_change":"2024-09-21T05:46:18.201920+0000","last_active":"2024-09-21T05:46:19.186788+0000","last_peered":"2024-09-21T05:46:19.186788+0000","last_clean":"2024-09-21T05:46:19.186788+0000","last_became_active":"2024-09-21T05:46:18.201556+0000","last_became_peered":"2024-09-21T05:46:18.201556+0000","last_unstale":"2024-09-21T05:46:19.186788+0000","last_undegraded":"2024-09-21T05:46:19.186788+0000","last_fullsized":"2024-09-21T05:46:19.186788+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:58.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.198893+0000","last_change":"2024-09-21T05:46:17.201319+0000","last_active":"2024-09-21T05:46:19.198893+0000","last_peered":"2024-09-21T05:46:19.198893+0000","last_clean":"2024-09-21T05:46:19.198893+0000","last_became_active":"2024-09-21T05:46:17.198548+0000","last_became_peered":"2024-09-21T05:46:17.198548+0000","last_unstale":"2024-09-21T05:46:19.198893+0000","last_undegraded":"2024-09-21T05:46:19.198893+0000","last_fullsized":"2024-09-21T05:46:19.198893+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:36.220460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.200006+0000","last_change":"2024-09-21T05:46:18.194698+0000","last_active":"2024-09-21T05:46:19.200006+0000","last_peered":"2024-09-21T05:46:19.200006+0000","last_clean":"2024-09-21T05:46:19.200006+0000","last_became_active":"2024-09-21T05:46:18.194007+0000","last_became_peered":"2024-09-21T05:46:18.194007+0000","last_unstale":"2024-09-21T05:46:19.200006+0000","last_undegraded":"2024-09-21T05:46:19.200006+0000","last_fullsized":"2024-09-21T05:46:19.200006+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:47.067006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444089+0000","last_change":"2024-09-21T05:46:17.188083+0000","last_active":"2024-09-21T05:46:19.444089+0000","last_peered":"2024-09-21T05:46:19.444089+0000","last_clean":"2024-09-21T05:46:19.444089+0000","last_became_active":"2024-09-21T05:46:17.187744+0000","last_became_peered":"2024-09-21T05:46:17.187744+0000","last_unstale":"2024-09-21T05:46:19.444089+0000","last_undegraded":"2024-09-21T05:46:19.444089+0000","last_fullsized":"2024-09-21T05:46:19.444089+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:28.100901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183188+0000","last_change":"2024-09-21T05:46:18.201490+0000","last_active":"2024-09-21T05:46:19.183188+0000","last_peered":"2024-09-21T05:46:19.183188+0000","last_clean":"2024-09-21T05:46:19.183188+0000","last_became_active":"2024-09-21T05:46:18.201183+0000","last_became_peered":"2024-09-21T05:46:18.201183+0000","last_unstale":"2024-09-21T05:46:19.183188+0000","last_undegraded":"2024-09-21T05:46:19.183188+0000","last_fullsized":"2024-09-21T05:46:19.183188+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:18.582096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.201547+0000","last_change":"2024-09-21T05:46:17.196627+0000","last_active":"2024-09-21T05:46:19.201547+0000","last_peered":"2024-09-21T05:46:19.201547+0000","last_clean":"2024-09-21T05:46:19.201547+0000","last_became_active":"2024-09-21T05:46:17.196294+0000","last_became_peered":"2024-09-21T05:46:17.196294+0000","last_unstale":"2024-09-21T05:46:19.201547+0000","last_undegraded":"2024-09-21T05:46:19.201547+0000","last_fullsized":"2024-09-21T05:46:19.201547+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:10.745490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183017+0000","last_change":"2024-09-21T05:46:18.194909+0000","last_active":"2024-09-21T05:46:19.183017+0000","last_peered":"2024-09-21T05:46:19.183017+0000","last_clean":"2024-09-21T05:46:19.183017+0000","last_became_active":"2024-09-21T05:46:18.194570+0000","last_became_peered":"2024-09-21T05:46:18.194570+0000","last_unstale":"2024-09-21T05:46:19.183017+0000","last_undegraded":"2024-09-21T05:46:19.183017+0000","last_fullsized":"2024-09-21T05:46:19.183017+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:35.950262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199589+0000","last_change":"2024-09-21T05:46:17.199645+0000","last_active":"2024-09-21T05:46:19.199589+0000","last_peered":"2024-09-21T05:46:19.199589+0000","last_clean":"2024-09-21T05:46:19.199589+0000","last_became_active":"2024-09-21T05:46:17.198926+0000","last_became_peered":"2024-09-21T05:46:17.198926+0000","last_unstale":"2024-09-21T05:46:19.199589+0000","last_undegraded":"2024-09-21T05:46:19.199589+0000","last_fullsized":"2024-09-21T05:46:19.199589+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:13.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187185+0000","last_change":"2024-09-21T05:46:18.195700+0000","last_active":"2024-09-21T05:46:19.187185+0000","last_peered":"2024-09-21T05:46:19.187185+0000","last_clean":"2024-09-21T05:46:19.187185+0000","last_became_active":"2024-09-21T05:46:18.195309+0000","last_became_peered":"2024-09-21T05:46:18.195309+0000","last_unstale":"2024-09-21T05:46:19.187185+0000","last_undegraded":"2024-09-21T05:46:19.187185+0000","last_fullsized":"2024-09-21T05:46:19.187185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:53.523226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188530+0000","last_change":"2024-09-21T05:46:17.186451+0000","last_active":"2024-09-21T05:46:19.188530+0000","last_peered":"2024-09-21T05:46:19.188530+0000","last_clean":"2024-09-21T05:46:19.188530+0000","last_became_active":"2024-09-21T05:46:17.186064+0000","last_became_peered":"2024-09-21T05:46:17.186064+0000","last_unstale":"2024-09-21T05:46:19.188530+0000","last_undegraded":"2024-09-21T05:46:19.188530+0000","last_fullsized":"2024-09-21T05:46:19.188530+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:41.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202552+0000","last_change":"2024-09-21T05:46:18.193241+0000","last_active":"2024-09-21T05:46:19.202552+0000","last_peered":"2024-09-21T05:46:19.202552+0000","last_clean":"2024-09-21T05:46:19.202552+0000","last_became_active":"2024-09-21T05:46:18.192433+0000","last_became_peered":"2024-09-21T05:46:18.192433+0000","last_unstale":"2024-09-21T05:46:19.202552+0000","last_undegraded":"2024-09-21T05:46:19.202552+0000","last_fullsized":"2024-09-21T05:46:19.202552+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:58.559819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199378+0000","last_change":"2024-09-21T05:46:17.196529+0000","last_active":"2024-09-21T05:46:19.199378+0000","last_peered":"2024-09-21T05:46:19.199378+0000","last_clean":"2024-09-21T05:46:19.199378+0000","last_became_active":"2024-09-21T05:46:17.196201+0000","last_became_peered":"2024-09-21T05:46:17.196201+0000","last_unstale":"2024-09-21T05:46:19.199378+0000","last_undegraded":"2024-09-21T05:46:19.199378+0000","last_fullsized":"2024-09-21T05:46:19.199378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:46.353796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186908+0000","last_change":"2024-09-21T05:46:18.188357+0000","last_active":"2024-09-21T05:46:19.186908+0000","last_peered":"2024-09-21T05:46:19.186908+0000","last_clean":"2024-09-21T05:46:19.186908+0000","last_became_active":"2024-09-21T05:46:18.186104+0000","last_became_peered":"2024-09-21T05:46:18.186104+0000","last_unstale":"2024-09-21T05:46:19.186908+0000","last_undegraded":"2024-09-21T05:46:19.186908+0000","last_fullsized":"2024-09-21T05:46:19.186908+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:56.560190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187040+0000","last_change":"2024-09-21T05:46:17.187748+0000","last_active":"2024-09-21T05:46:19.187040+0000","last_peered":"2024-09-21T05:46:19.187040+0000","last_clean":"2024-09-21T05:46:19.187040+0000","last_became_active":"2024-09-21T05:46:17.187414+0000","last_became_peered":"2024-09-21T05:46:17.187414+0000","last_unstale":"2024-09-21T05:46:19.187040+0000","last_undegraded":"2024-09-21T05:46:19.187040+0000","last_fullsized":"2024-09-21T05:46:19.187040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:25.281501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202096+0000","last_change":"2024-09-21T05:46:18.188393+0000","last_active":"2024-09-21T05:46:19.202096+0000","last_peered":"2024-09-21T05:46:19.202096+0000","last_clean":"2024-09-21T05:46:19.202096+0000","last_became_active":"2024-09-21T05:46:18.188084+0000","last_became_peered":"2024-09-21T05:46:18.188084+0000","last_unstale":"2024-09-21T05:46:19.202096+0000","last_undegraded":"2024-09-21T05:46:19.202096+0000","last_fullsized":"2024-09-21T05:46:19.202096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:38.245930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443129+0000","last_change":"2024-09-21T05:46:17.190702+0000","last_active":"2024-09-21T05:46:19.443129+0000","last_peered":"2024-09-21T05:46:19.443129+0000","last_clean":"2024-09-21T05:46:19.443129+0000","last_became_active":"2024-09-21T05:46:17.190334+0000","last_became_peered":"2024-09-21T05:46:17.190334+0000","last_unstale":"2024-09-21T05:46:19.443129+0000","last_undegraded":"2024-09-21T05:46:19.443129+0000","last_fullsized":"2024-09-21T05:46:19.443129+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:32.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187028+0000","last_change":"2024-09-21T05:46:18.196307+0000","last_active":"2024-09-21T05:46:19.187028+0000","last_peered":"2024-09-21T05:46:19.187028+0000","last_clean":"2024-09-21T05:46:19.187028+0000","last_became_active":"2024-09-21T05:46:18.195976+0000","last_became_peered":"2024-09-21T05:46:18.195976+0000","last_unstale":"2024-09-21T05:46:19.187028+0000","last_undegraded":"2024-09-21T05:46:19.187028+0000","last_fullsized":"2024-09-21T05:46:19.187028+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:47.788275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202526+0000","last_change":"2024-09-21T05:46:17.200565+0000","last_active":"2024-09-21T05:46:19.202526+0000","last_peered":"2024-09-21T05:46:19.202526+0000","last_clean":"2024-09-21T05:46:19.202526+0000","last_became_active":"2024-09-21T05:46:17.200201+0000","last_became_peered":"2024-09-21T05:46:17.200201+0000","last_unstale":"2024-09-21T05:46:19.202526+0000","last_undegraded":"2024-09-21T05:46:19.202526+0000","last_fullsized":"2024-09-21T05:46:19.202526+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:52.617508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202785+0000","last_change":"2024-09-21T05:46:18.195930+0000","last_active":"2024-09-21T05:46:19.202785+0000","last_peered":"2024-09-21T05:46:19.202785+0000","last_clean":"2024-09-21T05:46:19.202785+0000","last_became_active":"2024-09-21T05:46:18.195587+0000","last_became_peered":"2024-09-21T05:46:18.195587+0000","last_unstale":"2024-09-21T05:46:19.202785+0000","last_undegraded":"2024-09-21T05:46:19.202785+0000","last_fullsized":"2024-09-21T05:46:19.202785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:33.275939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443259+0000","last_change":"2024-09-21T05:46:17.191544+0000","last_active":"2024-09-21T05:46:19.443259+0000","last_peered":"2024-09-21T05:46:19.443259+0000","last_clean":"2024-09-21T05:46:19.443259+0000","last_became_active":"2024-09-21T05:46:17.191192+0000","last_became_peered":"2024-09-21T05:46:17.191192+0000","last_unstale":"2024-09-21T05:46:19.443259+0000","last_undegraded":"2024-09-21T05:46:19.443259+0000","last_fullsized":"2024-09-21T05:46:19.443259+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:09.759511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443574+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:46:19.443574+0000","last_peered":"2024-09-21T05:46:19.443574+0000","last_clean":"2024-09-21T05:46:19.443574+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:46:19.443574+0000","last_undegraded":"2024-09-21T05:46:19.443574+0000","last_fullsized":"2024-09-21T05:46:19.443574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1794048,"data_stored":1104740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1212416,"data_stored":515996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542233,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1200128,"data_stored":514912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542233,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":514330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36528,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335312,"statfs":{"total":96636764160,"available":96599359488,"internally_reserved":0,"allocated":1208320,"data_stored":515894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542235,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1196032,"data_stored":514294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36512,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335328,"statfs":{"total":96636764160,"available":96599375872,"internally_reserved":0,"allocated":1191936,"data_stored":514260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542233,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1789952,"data_stored":1105210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:46:22.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 -- 172.21.15.22:0/3773150014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc0b6100 msgr2=0x7f67dc0b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 --2- 172.21.15.22:0/3773150014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc0b6100 0x7f67dc0b1320 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0739d0 tx=0x7f67d002a5c0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 -- 172.21.15.22:0/3773150014 shutdown_connections 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 --2- 172.21.15.22:0/3773150014 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67dc161f50 0x7f67dc164370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 --2- 172.21.15.22:0/3773150014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc0b6100 0x7f67dc0b1320 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 --2- 172.21.15.22:0/3773150014 >> v2:172.21.15.134:3300/0 conn(0x7f67dc0b6680 0x7f67dc0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 -- 172.21.15.22:0/3773150014 >> 172.21.15.22:0/3773150014 conn(0x7f67dc0d12a0 msgr2=0x7f67dc0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:22.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 -- 172.21.15.22:0/3773150014 shutdown_connections 2024-09-21T05:46:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.423+0000 7f67e1e30640 1 -- 172.21.15.22:0/3773150014 wait complete. 2024-09-21T05:46:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.425+0000 7f67e1e30640 1 Processor -- start 2024-09-21T05:46:22.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.425+0000 7f67e1e30640 1 -- start start 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc0b6100 0x7f67dc1f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67dc0b6680 0x7f67dc09cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 0x7f67dc09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1648b0 con 0x7f67dc161f50 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc0b20d0 con 0x7f67dc0b6100 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f67dc164a30 con 0x7f67dc0b6680 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 0x7f67dc09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 0x7f67dc09d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55152/0 (socket says 172.21.15.22:55152) 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 -- 172.21.15.22:0/1470133602 learned_addr learned my addr 172.21.15.22:0/1470133602 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67daffd640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67dc0b6680 0x7f67dc09cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67db7fe640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc0b6100 0x7f67dc1f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 -- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67dc0b6680 msgr2=0x7f67dc09cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67dc0b6680 0x7f67dc09cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 -- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc0b6100 msgr2=0x7f67dc1f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc0b6100 0x7f67dc1f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 -- 172.21.15.22:0/1470133602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc1f6cc0 con 0x7f67dc161f50 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67dbfff640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 0x7f67dc09d4f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e990 tx=0x7f67d400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4004d20 con 0x7f67dc161f50 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67d400ca40 con 0x7f67dc161f50 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc1f6fa0 con 0x7f67dc161f50 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.427+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4005020 con 0x7f67dc161f50 2024-09-21T05:46:22.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.428+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1f7560 con 0x7f67dc161f50 2024-09-21T05:46:22.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.428+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc160fe0 con 0x7f67dc161f50 2024-09-21T05:46:22.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.429+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f67d40051c0 con 0x7f67dc161f50 2024-09-21T05:46:22.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.430+0000 7f67c3fff640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f67c8078bc0 0x7f67c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.430+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f67d409b5e0 con 0x7f67dc161f50 2024-09-21T05:46:22.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.430+0000 7f67db7fe640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f67c8078bc0 0x7f67c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.431+0000 7f67db7fe640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f67c8078bc0 0x7f67c807b0d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0ceda0 tx=0x7f67cc00b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:22.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.435+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f67d4016030 con 0x7f67dc161f50 2024-09-21T05:46:22.526 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:46:22.529 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:46:22.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.620+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67dc0b0de0 con 0x7f67c8078bc0 2024-09-21T05:46:22.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.625+0000 7f67c3fff640 1 -- 172.21.15.22:0/1470133602 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f67dc0b0de0 con 0x7f67c8078bc0 2024-09-21T05:46:22.629 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:22.630 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f67c8078bc0 msgr2=0x7f67c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f67c8078bc0 0x7f67c807b0d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f67dc0ceda0 tx=0x7f67cc00b260 comp rx=0 tx=0).stop 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 msgr2=0x7f67dc09d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 0x7f67dc09d4f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e990 tx=0x7f67d400ee50 comp rx=0 tx=0).stop 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 shutdown_connections 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f67dc161f50 0x7f67dc09d4f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f67c8078bc0 0x7f67c807b0d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f67dc0b6680 0x7f67dc09cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 --2- 172.21.15.22:0/1470133602 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f67dc0b6100 0x7f67dc1f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 >> 172.21.15.22:0/1470133602 conn(0x7f67dc0d12a0 msgr2=0x7f67dc0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:22.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 shutdown_connections 2024-09-21T05:46:22.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.628+0000 7f67e1e30640 1 -- 172.21.15.22:0/1470133602 wait complete. 2024-09-21T05:46:22.641 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-21T05:46:21.851001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276700,"kb_used_data":10536,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754698020,"statfs":{"total":773094113280,"available":772810772480,"internally_reserved":0,"allocated":10788864,"data_stored":5299636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.681980"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.182932+0000","last_change":"2024-09-21T05:46:18.198277+0000","last_active":"2024-09-21T05:46:19.182932+0000","last_peered":"2024-09-21T05:46:19.182932+0000","last_clean":"2024-09-21T05:46:19.182932+0000","last_became_active":"2024-09-21T05:46:18.197795+0000","last_became_peered":"2024-09-21T05:46:18.197795+0000","last_unstale":"2024-09-21T05:46:19.182932+0000","last_undegraded":"2024-09-21T05:46:19.182932+0000","last_fullsized":"2024-09-21T05:46:19.182932+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:47.260759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199768+0000","last_change":"2024-09-21T05:46:18.195066+0000","last_active":"2024-09-21T05:46:19.199768+0000","last_peered":"2024-09-21T05:46:19.199768+0000","last_clean":"2024-09-21T05:46:19.199768+0000","last_became_active":"2024-09-21T05:46:18.194408+0000","last_became_peered":"2024-09-21T05:46:18.194408+0000","last_unstale":"2024-09-21T05:46:19.199768+0000","last_undegraded":"2024-09-21T05:46:19.199768+0000","last_fullsized":"2024-09-21T05:46:19.199768+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:11.625271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183093+0000","last_change":"2024-09-21T05:46:18.198417+0000","last_active":"2024-09-21T05:46:19.183093+0000","last_peered":"2024-09-21T05:46:19.183093+0000","last_clean":"2024-09-21T05:46:19.183093+0000","last_became_active":"2024-09-21T05:46:18.197972+0000","last_became_peered":"2024-09-21T05:46:18.197972+0000","last_unstale":"2024-09-21T05:46:19.183093+0000","last_undegraded":"2024-09-21T05:46:19.183093+0000","last_fullsized":"2024-09-21T05:46:19.183093+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:35.955653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443250+0000","last_change":"2024-09-21T05:46:18.202921+0000","last_active":"2024-09-21T05:46:19.443250+0000","last_peered":"2024-09-21T05:46:19.443250+0000","last_clean":"2024-09-21T05:46:19.443250+0000","last_became_active":"2024-09-21T05:46:18.202580+0000","last_became_peered":"2024-09-21T05:46:18.202580+0000","last_unstale":"2024-09-21T05:46:19.443250+0000","last_undegraded":"2024-09-21T05:46:19.443250+0000","last_fullsized":"2024-09-21T05:46:19.443250+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:46.983760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187902+0000","last_change":"2024-09-21T05:46:18.199739+0000","last_active":"2024-09-21T05:46:19.187902+0000","last_peered":"2024-09-21T05:46:19.187902+0000","last_clean":"2024-09-21T05:46:19.187902+0000","last_became_active":"2024-09-21T05:46:18.199413+0000","last_became_peered":"2024-09-21T05:46:18.199413+0000","last_unstale":"2024-09-21T05:46:19.187902+0000","last_undegraded":"2024-09-21T05:46:19.187902+0000","last_fullsized":"2024-09-21T05:46:19.187902+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:36:33.816770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443836+0000","last_change":"2024-09-21T05:46:18.191732+0000","last_active":"2024-09-21T05:46:19.443836+0000","last_peered":"2024-09-21T05:46:19.443836+0000","last_clean":"2024-09-21T05:46:19.443836+0000","last_became_active":"2024-09-21T05:46:18.191276+0000","last_became_peered":"2024-09-21T05:46:18.191276+0000","last_unstale":"2024-09-21T05:46:19.443836+0000","last_undegraded":"2024-09-21T05:46:19.443836+0000","last_fullsized":"2024-09-21T05:46:19.443836+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:15.603230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443655+0000","last_change":"2024-09-21T05:46:18.193937+0000","last_active":"2024-09-21T05:46:19.443655+0000","last_peered":"2024-09-21T05:46:19.443655+0000","last_clean":"2024-09-21T05:46:19.443655+0000","last_became_active":"2024-09-21T05:46:18.193597+0000","last_became_peered":"2024-09-21T05:46:18.193597+0000","last_unstale":"2024-09-21T05:46:19.443655+0000","last_undegraded":"2024-09-21T05:46:19.443655+0000","last_fullsized":"2024-09-21T05:46:19.443655+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:36.619562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199003+0000","last_change":"2024-09-21T05:46:18.193242+0000","last_active":"2024-09-21T05:46:19.199003+0000","last_peered":"2024-09-21T05:46:19.199003+0000","last_clean":"2024-09-21T05:46:19.199003+0000","last_became_active":"2024-09-21T05:46:18.192900+0000","last_became_peered":"2024-09-21T05:46:18.192900+0000","last_unstale":"2024-09-21T05:46:19.199003+0000","last_undegraded":"2024-09-21T05:46:19.199003+0000","last_fullsized":"2024-09-21T05:46:19.199003+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:18:22.274135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187656+0000","last_change":"2024-09-21T05:46:18.196117+0000","last_active":"2024-09-21T05:46:19.187656+0000","last_peered":"2024-09-21T05:46:19.187656+0000","last_clean":"2024-09-21T05:46:19.187656+0000","last_became_active":"2024-09-21T05:46:18.195812+0000","last_became_peered":"2024-09-21T05:46:18.195812+0000","last_unstale":"2024-09-21T05:46:19.187656+0000","last_undegraded":"2024-09-21T05:46:19.187656+0000","last_fullsized":"2024-09-21T05:46:19.187656+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:58.656688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186868+0000","last_change":"2024-09-21T05:46:18.201926+0000","last_active":"2024-09-21T05:46:19.186868+0000","last_peered":"2024-09-21T05:46:19.186868+0000","last_clean":"2024-09-21T05:46:19.186868+0000","last_became_active":"2024-09-21T05:46:18.201633+0000","last_became_peered":"2024-09-21T05:46:18.201633+0000","last_unstale":"2024-09-21T05:46:19.186868+0000","last_undegraded":"2024-09-21T05:46:19.186868+0000","last_fullsized":"2024-09-21T05:46:19.186868+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:23.952267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443652+0000","last_change":"2024-09-21T05:46:18.184560+0000","last_active":"2024-09-21T05:46:19.443652+0000","last_peered":"2024-09-21T05:46:19.443652+0000","last_clean":"2024-09-21T05:46:19.443652+0000","last_became_active":"2024-09-21T05:46:18.184134+0000","last_became_peered":"2024-09-21T05:46:18.184134+0000","last_unstale":"2024-09-21T05:46:19.443652+0000","last_undegraded":"2024-09-21T05:46:19.443652+0000","last_fullsized":"2024-09-21T05:46:19.443652+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:08.177819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188148+0000","last_change":"2024-09-21T05:46:18.193884+0000","last_active":"2024-09-21T05:46:19.188148+0000","last_peered":"2024-09-21T05:46:19.188148+0000","last_clean":"2024-09-21T05:46:19.188148+0000","last_became_active":"2024-09-21T05:46:18.193571+0000","last_became_peered":"2024-09-21T05:46:18.193571+0000","last_unstale":"2024-09-21T05:46:19.188148+0000","last_undegraded":"2024-09-21T05:46:19.188148+0000","last_fullsized":"2024-09-21T05:46:19.188148+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:41.932041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443177+0000","last_change":"2024-09-21T05:46:18.204644+0000","last_active":"2024-09-21T05:46:19.443177+0000","last_peered":"2024-09-21T05:46:19.443177+0000","last_clean":"2024-09-21T05:46:19.443177+0000","last_became_active":"2024-09-21T05:46:18.204331+0000","last_became_peered":"2024-09-21T05:46:18.204331+0000","last_unstale":"2024-09-21T05:46:19.443177+0000","last_undegraded":"2024-09-21T05:46:19.443177+0000","last_fullsized":"2024-09-21T05:46:19.443177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:06:12.799874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202086+0000","last_change":"2024-09-21T05:46:18.183826+0000","last_active":"2024-09-21T05:46:19.202086+0000","last_peered":"2024-09-21T05:46:19.202086+0000","last_clean":"2024-09-21T05:46:19.202086+0000","last_became_active":"2024-09-21T05:46:18.183495+0000","last_became_peered":"2024-09-21T05:46:18.183495+0000","last_unstale":"2024-09-21T05:46:19.202086+0000","last_undegraded":"2024-09-21T05:46:19.202086+0000","last_fullsized":"2024-09-21T05:46:19.202086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:04:44.749567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.442975+0000","last_change":"2024-09-21T05:46:18.206520+0000","last_active":"2024-09-21T05:46:19.442975+0000","last_peered":"2024-09-21T05:46:19.442975+0000","last_clean":"2024-09-21T05:46:19.442975+0000","last_became_active":"2024-09-21T05:46:18.206205+0000","last_became_peered":"2024-09-21T05:46:18.206205+0000","last_unstale":"2024-09-21T05:46:19.442975+0000","last_undegraded":"2024-09-21T05:46:19.442975+0000","last_fullsized":"2024-09-21T05:46:19.442975+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:33.075301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443471+0000","last_change":"2024-09-21T05:46:18.199208+0000","last_active":"2024-09-21T05:46:19.443471+0000","last_peered":"2024-09-21T05:46:19.443471+0000","last_clean":"2024-09-21T05:46:19.443471+0000","last_became_active":"2024-09-21T05:46:18.198902+0000","last_became_peered":"2024-09-21T05:46:18.198902+0000","last_unstale":"2024-09-21T05:46:19.443471+0000","last_undegraded":"2024-09-21T05:46:19.443471+0000","last_fullsized":"2024-09-21T05:46:19.443471+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:20:43.018538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199791+0000","last_change":"2024-09-21T05:46:18.195946+0000","last_active":"2024-09-21T05:46:19.199791+0000","last_peered":"2024-09-21T05:46:19.199791+0000","last_clean":"2024-09-21T05:46:19.199791+0000","last_became_active":"2024-09-21T05:46:18.195609+0000","last_became_peered":"2024-09-21T05:46:18.195609+0000","last_unstale":"2024-09-21T05:46:19.199791+0000","last_undegraded":"2024-09-21T05:46:19.199791+0000","last_fullsized":"2024-09-21T05:46:19.199791+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:12:42.249069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443281+0000","last_change":"2024-09-21T05:46:18.202205+0000","last_active":"2024-09-21T05:46:19.443281+0000","last_peered":"2024-09-21T05:46:19.443281+0000","last_clean":"2024-09-21T05:46:19.443281+0000","last_became_active":"2024-09-21T05:46:18.201857+0000","last_became_peered":"2024-09-21T05:46:18.201857+0000","last_unstale":"2024-09-21T05:46:19.443281+0000","last_undegraded":"2024-09-21T05:46:19.443281+0000","last_fullsized":"2024-09-21T05:46:19.443281+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:09.661160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202445+0000","last_change":"2024-09-21T05:46:18.202860+0000","last_active":"2024-09-21T05:46:19.202445+0000","last_peered":"2024-09-21T05:46:19.202445+0000","last_clean":"2024-09-21T05:46:19.202445+0000","last_became_active":"2024-09-21T05:46:18.202526+0000","last_became_peered":"2024-09-21T05:46:18.202526+0000","last_unstale":"2024-09-21T05:46:19.202445+0000","last_undegraded":"2024-09-21T05:46:19.202445+0000","last_fullsized":"2024-09-21T05:46:19.202445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:08.335261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188188+0000","last_change":"2024-09-21T05:46:18.196489+0000","last_active":"2024-09-21T05:46:19.188188+0000","last_peered":"2024-09-21T05:46:19.188188+0000","last_clean":"2024-09-21T05:46:19.188188+0000","last_became_active":"2024-09-21T05:46:18.196169+0000","last_became_peered":"2024-09-21T05:46:18.196169+0000","last_unstale":"2024-09-21T05:46:19.188188+0000","last_undegraded":"2024-09-21T05:46:19.188188+0000","last_fullsized":"2024-09-21T05:46:19.188188+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:11:36.619562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202299+0000","last_change":"2024-09-21T05:46:18.192957+0000","last_active":"2024-09-21T05:46:19.202299+0000","last_peered":"2024-09-21T05:46:19.202299+0000","last_clean":"2024-09-21T05:46:19.202299+0000","last_became_active":"2024-09-21T05:46:18.192529+0000","last_became_peered":"2024-09-21T05:46:18.192529+0000","last_unstale":"2024-09-21T05:46:19.202299+0000","last_undegraded":"2024-09-21T05:46:19.202299+0000","last_fullsized":"2024-09-21T05:46:19.202299+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:19.102782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.198982+0000","last_change":"2024-09-21T05:46:18.195762+0000","last_active":"2024-09-21T05:46:19.198982+0000","last_peered":"2024-09-21T05:46:19.198982+0000","last_clean":"2024-09-21T05:46:19.198982+0000","last_became_active":"2024-09-21T05:46:18.195405+0000","last_became_peered":"2024-09-21T05:46:18.195405+0000","last_unstale":"2024-09-21T05:46:19.198982+0000","last_undegraded":"2024-09-21T05:46:19.198982+0000","last_fullsized":"2024-09-21T05:46:19.198982+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:25.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.198952+0000","last_change":"2024-09-21T05:46:18.193981+0000","last_active":"2024-09-21T05:46:19.198952+0000","last_peered":"2024-09-21T05:46:19.198952+0000","last_clean":"2024-09-21T05:46:19.198952+0000","last_became_active":"2024-09-21T05:46:18.193624+0000","last_became_peered":"2024-09-21T05:46:18.193624+0000","last_unstale":"2024-09-21T05:46:19.198952+0000","last_undegraded":"2024-09-21T05:46:19.198952+0000","last_fullsized":"2024-09-21T05:46:19.198952+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:49.104631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443392+0000","last_change":"2024-09-21T05:46:18.200577+0000","last_active":"2024-09-21T05:46:19.443392+0000","last_peered":"2024-09-21T05:46:19.443392+0000","last_clean":"2024-09-21T05:46:19.443392+0000","last_became_active":"2024-09-21T05:46:18.199923+0000","last_became_peered":"2024-09-21T05:46:18.199923+0000","last_unstale":"2024-09-21T05:46:19.443392+0000","last_undegraded":"2024-09-21T05:46:19.443392+0000","last_fullsized":"2024-09-21T05:46:19.443392+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:00.821372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186979+0000","last_change":"2024-09-21T05:46:18.197641+0000","last_active":"2024-09-21T05:46:19.186979+0000","last_peered":"2024-09-21T05:46:19.186979+0000","last_clean":"2024-09-21T05:46:19.186979+0000","last_became_active":"2024-09-21T05:46:18.197293+0000","last_became_peered":"2024-09-21T05:46:18.197293+0000","last_unstale":"2024-09-21T05:46:19.186979+0000","last_undegraded":"2024-09-21T05:46:19.186979+0000","last_fullsized":"2024-09-21T05:46:19.186979+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:11.751500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199764+0000","last_change":"2024-09-21T05:46:18.200222+0000","last_active":"2024-09-21T05:46:19.199764+0000","last_peered":"2024-09-21T05:46:19.199764+0000","last_clean":"2024-09-21T05:46:19.199764+0000","last_became_active":"2024-09-21T05:46:18.199867+0000","last_became_peered":"2024-09-21T05:46:18.199867+0000","last_unstale":"2024-09-21T05:46:19.199764+0000","last_undegraded":"2024-09-21T05:46:19.199764+0000","last_fullsized":"2024-09-21T05:46:19.199764+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:00.234053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443913+0000","last_change":"2024-09-21T05:46:18.200719+0000","last_active":"2024-09-21T05:46:19.443913+0000","last_peered":"2024-09-21T05:46:19.443913+0000","last_clean":"2024-09-21T05:46:19.443913+0000","last_became_active":"2024-09-21T05:46:18.200358+0000","last_became_peered":"2024-09-21T05:46:18.200358+0000","last_unstale":"2024-09-21T05:46:19.443913+0000","last_undegraded":"2024-09-21T05:46:19.443913+0000","last_fullsized":"2024-09-21T05:46:19.443913+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:28.738432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188425+0000","last_change":"2024-09-21T05:46:18.195937+0000","last_active":"2024-09-21T05:46:19.188425+0000","last_peered":"2024-09-21T05:46:19.188425+0000","last_clean":"2024-09-21T05:46:19.188425+0000","last_became_active":"2024-09-21T05:46:18.195635+0000","last_became_peered":"2024-09-21T05:46:18.195635+0000","last_unstale":"2024-09-21T05:46:19.188425+0000","last_undegraded":"2024-09-21T05:46:19.188425+0000","last_fullsized":"2024-09-21T05:46:19.188425+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:41.309563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199171+0000","last_change":"2024-09-21T05:46:18.196699+0000","last_active":"2024-09-21T05:46:19.199171+0000","last_peered":"2024-09-21T05:46:19.199171+0000","last_clean":"2024-09-21T05:46:19.199171+0000","last_became_active":"2024-09-21T05:46:18.196368+0000","last_became_peered":"2024-09-21T05:46:18.196368+0000","last_unstale":"2024-09-21T05:46:19.199171+0000","last_undegraded":"2024-09-21T05:46:19.199171+0000","last_fullsized":"2024-09-21T05:46:19.199171+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:26.119238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202350+0000","last_change":"2024-09-21T05:46:18.193817+0000","last_active":"2024-09-21T05:46:19.202350+0000","last_peered":"2024-09-21T05:46:19.202350+0000","last_clean":"2024-09-21T05:46:19.202350+0000","last_became_active":"2024-09-21T05:46:18.192992+0000","last_became_peered":"2024-09-21T05:46:18.192992+0000","last_unstale":"2024-09-21T05:46:19.202350+0000","last_undegraded":"2024-09-21T05:46:19.202350+0000","last_fullsized":"2024-09-21T05:46:19.202350+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:51:23.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443282+0000","last_change":"2024-09-21T05:46:18.197180+0000","last_active":"2024-09-21T05:46:19.443282+0000","last_peered":"2024-09-21T05:46:19.443282+0000","last_clean":"2024-09-21T05:46:19.443282+0000","last_became_active":"2024-09-21T05:46:18.196831+0000","last_became_peered":"2024-09-21T05:46:18.196831+0000","last_unstale":"2024-09-21T05:46:19.443282+0000","last_undegraded":"2024-09-21T05:46:19.443282+0000","last_fullsized":"2024-09-21T05:46:19.443282+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.723228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183109+0000","last_change":"2024-09-21T05:46:18.197226+0000","last_active":"2024-09-21T05:46:19.183109+0000","last_peered":"2024-09-21T05:46:19.183109+0000","last_clean":"2024-09-21T05:46:19.183109+0000","last_became_active":"2024-09-21T05:46:18.196492+0000","last_became_peered":"2024-09-21T05:46:18.196492+0000","last_unstale":"2024-09-21T05:46:19.183109+0000","last_undegraded":"2024-09-21T05:46:19.183109+0000","last_fullsized":"2024-09-21T05:46:19.183109+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:46.715787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202686+0000","last_change":"2024-09-21T05:46:18.191966+0000","last_active":"2024-09-21T05:46:19.202686+0000","last_peered":"2024-09-21T05:46:19.202686+0000","last_clean":"2024-09-21T05:46:19.202686+0000","last_became_active":"2024-09-21T05:46:18.190860+0000","last_became_peered":"2024-09-21T05:46:18.190860+0000","last_unstale":"2024-09-21T05:46:19.202686+0000","last_undegraded":"2024-09-21T05:46:19.202686+0000","last_fullsized":"2024-09-21T05:46:19.202686+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:48.756997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"96'4","reported_seq":19,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199710+0000","last_change":"2024-09-21T05:46:17.197005+0000","last_active":"2024-09-21T05:46:19.199710+0000","last_peered":"2024-09-21T05:46:19.199710+0000","last_clean":"2024-09-21T05:46:19.199710+0000","last_became_active":"2024-09-21T05:46:17.196527+0000","last_became_peered":"2024-09-21T05:46:17.196527+0000","last_unstale":"2024-09-21T05:46:19.199710+0000","last_undegraded":"2024-09-21T05:46:19.199710+0000","last_fullsized":"2024-09-21T05:46:19.199710+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:41:15.104091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443477+0000","last_change":"2024-09-21T05:46:18.190900+0000","last_active":"2024-09-21T05:46:19.443477+0000","last_peered":"2024-09-21T05:46:19.443477+0000","last_clean":"2024-09-21T05:46:19.443477+0000","last_became_active":"2024-09-21T05:46:18.190520+0000","last_became_peered":"2024-09-21T05:46:18.190520+0000","last_unstale":"2024-09-21T05:46:19.443477+0000","last_undegraded":"2024-09-21T05:46:19.443477+0000","last_fullsized":"2024-09-21T05:46:19.443477+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:58:22.634498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443423+0000","last_change":"2024-09-21T05:46:17.189404+0000","last_active":"2024-09-21T05:46:19.443423+0000","last_peered":"2024-09-21T05:46:19.443423+0000","last_clean":"2024-09-21T05:46:19.443423+0000","last_became_active":"2024-09-21T05:46:17.189086+0000","last_became_peered":"2024-09-21T05:46:17.189086+0000","last_unstale":"2024-09-21T05:46:19.443423+0000","last_undegraded":"2024-09-21T05:46:19.443423+0000","last_fullsized":"2024-09-21T05:46:19.443423+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:12:08.756865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183057+0000","last_change":"2024-09-21T05:46:18.188710+0000","last_active":"2024-09-21T05:46:19.183057+0000","last_peered":"2024-09-21T05:46:19.183057+0000","last_clean":"2024-09-21T05:46:19.183057+0000","last_became_active":"2024-09-21T05:46:18.188275+0000","last_became_peered":"2024-09-21T05:46:18.188275+0000","last_unstale":"2024-09-21T05:46:19.183057+0000","last_undegraded":"2024-09-21T05:46:19.183057+0000","last_fullsized":"2024-09-21T05:46:19.183057+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:51:07.835349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199551+0000","last_change":"2024-09-21T05:46:17.201433+0000","last_active":"2024-09-21T05:46:19.199551+0000","last_peered":"2024-09-21T05:46:19.199551+0000","last_clean":"2024-09-21T05:46:19.199551+0000","last_became_active":"2024-09-21T05:46:17.199344+0000","last_became_peered":"2024-09-21T05:46:17.199344+0000","last_unstale":"2024-09-21T05:46:19.199551+0000","last_undegraded":"2024-09-21T05:46:19.199551+0000","last_fullsized":"2024-09-21T05:46:19.199551+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:55:48.069757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188132+0000","last_change":"2024-09-21T05:46:18.190968+0000","last_active":"2024-09-21T05:46:19.188132+0000","last_peered":"2024-09-21T05:46:19.188132+0000","last_clean":"2024-09-21T05:46:19.188132+0000","last_became_active":"2024-09-21T05:46:18.190665+0000","last_became_peered":"2024-09-21T05:46:18.190665+0000","last_unstale":"2024-09-21T05:46:19.188132+0000","last_undegraded":"2024-09-21T05:46:19.188132+0000","last_fullsized":"2024-09-21T05:46:19.188132+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:45.675703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199472+0000","last_change":"2024-09-21T05:46:17.199581+0000","last_active":"2024-09-21T05:46:19.199472+0000","last_peered":"2024-09-21T05:46:19.199472+0000","last_clean":"2024-09-21T05:46:19.199472+0000","last_became_active":"2024-09-21T05:46:17.199068+0000","last_became_peered":"2024-09-21T05:46:17.199068+0000","last_unstale":"2024-09-21T05:46:19.199472+0000","last_undegraded":"2024-09-21T05:46:19.199472+0000","last_fullsized":"2024-09-21T05:46:19.199472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:57.941291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444250+0000","last_change":"2024-09-21T05:46:18.194581+0000","last_active":"2024-09-21T05:46:19.444250+0000","last_peered":"2024-09-21T05:46:19.444250+0000","last_clean":"2024-09-21T05:46:19.444250+0000","last_became_active":"2024-09-21T05:46:18.194174+0000","last_became_peered":"2024-09-21T05:46:18.194174+0000","last_unstale":"2024-09-21T05:46:19.444250+0000","last_undegraded":"2024-09-21T05:46:19.444250+0000","last_fullsized":"2024-09-21T05:46:19.444250+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:27.838115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.182967+0000","last_change":"2024-09-21T05:46:17.191438+0000","last_active":"2024-09-21T05:46:19.182967+0000","last_peered":"2024-09-21T05:46:19.182967+0000","last_clean":"2024-09-21T05:46:19.182967+0000","last_became_active":"2024-09-21T05:46:17.191136+0000","last_became_peered":"2024-09-21T05:46:17.191136+0000","last_unstale":"2024-09-21T05:46:19.182967+0000","last_undegraded":"2024-09-21T05:46:19.182967+0000","last_fullsized":"2024-09-21T05:46:19.182967+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:55.389691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443397+0000","last_change":"2024-09-21T05:46:18.187130+0000","last_active":"2024-09-21T05:46:19.443397+0000","last_peered":"2024-09-21T05:46:19.443397+0000","last_clean":"2024-09-21T05:46:19.443397+0000","last_became_active":"2024-09-21T05:46:18.186802+0000","last_became_peered":"2024-09-21T05:46:18.186802+0000","last_unstale":"2024-09-21T05:46:19.443397+0000","last_undegraded":"2024-09-21T05:46:19.443397+0000","last_fullsized":"2024-09-21T05:46:19.443397+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.698448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199116+0000","last_change":"2024-09-21T05:46:17.196630+0000","last_active":"2024-09-21T05:46:19.199116+0000","last_peered":"2024-09-21T05:46:19.199116+0000","last_clean":"2024-09-21T05:46:19.199116+0000","last_became_active":"2024-09-21T05:46:17.196319+0000","last_became_peered":"2024-09-21T05:46:17.196319+0000","last_unstale":"2024-09-21T05:46:19.199116+0000","last_undegraded":"2024-09-21T05:46:19.199116+0000","last_fullsized":"2024-09-21T05:46:19.199116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:34:59.652406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.442885+0000","last_change":"2024-09-21T05:46:18.207857+0000","last_active":"2024-09-21T05:46:19.442885+0000","last_peered":"2024-09-21T05:46:19.442885+0000","last_clean":"2024-09-21T05:46:19.442885+0000","last_became_active":"2024-09-21T05:46:18.207300+0000","last_became_peered":"2024-09-21T05:46:18.207300+0000","last_unstale":"2024-09-21T05:46:19.442885+0000","last_undegraded":"2024-09-21T05:46:19.442885+0000","last_fullsized":"2024-09-21T05:46:19.442885+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:17:02.670528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187162+0000","last_change":"2024-09-21T05:46:17.198061+0000","last_active":"2024-09-21T05:46:19.187162+0000","last_peered":"2024-09-21T05:46:19.187162+0000","last_clean":"2024-09-21T05:46:19.187162+0000","last_became_active":"2024-09-21T05:46:17.197745+0000","last_became_peered":"2024-09-21T05:46:17.197745+0000","last_unstale":"2024-09-21T05:46:19.187162+0000","last_undegraded":"2024-09-21T05:46:19.187162+0000","last_fullsized":"2024-09-21T05:46:19.187162+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:30:26.832106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186574+0000","last_change":"2024-09-21T05:46:18.192709+0000","last_active":"2024-09-21T05:46:19.186574+0000","last_peered":"2024-09-21T05:46:19.186574+0000","last_clean":"2024-09-21T05:46:19.186574+0000","last_became_active":"2024-09-21T05:46:18.191982+0000","last_became_peered":"2024-09-21T05:46:18.191982+0000","last_unstale":"2024-09-21T05:46:19.186574+0000","last_undegraded":"2024-09-21T05:46:19.186574+0000","last_fullsized":"2024-09-21T05:46:19.186574+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:06.117685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183220+0000","last_change":"2024-09-21T05:46:17.186502+0000","last_active":"2024-09-21T05:46:19.183220+0000","last_peered":"2024-09-21T05:46:19.183220+0000","last_clean":"2024-09-21T05:46:19.183220+0000","last_became_active":"2024-09-21T05:46:17.186162+0000","last_became_peered":"2024-09-21T05:46:17.186162+0000","last_unstale":"2024-09-21T05:46:19.183220+0000","last_undegraded":"2024-09-21T05:46:19.183220+0000","last_fullsized":"2024-09-21T05:46:19.183220+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:31.895594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199978+0000","last_change":"2024-09-21T05:46:18.194734+0000","last_active":"2024-09-21T05:46:19.199978+0000","last_peered":"2024-09-21T05:46:19.199978+0000","last_clean":"2024-09-21T05:46:19.199978+0000","last_became_active":"2024-09-21T05:46:18.194374+0000","last_became_peered":"2024-09-21T05:46:18.194374+0000","last_unstale":"2024-09-21T05:46:19.199978+0000","last_undegraded":"2024-09-21T05:46:19.199978+0000","last_fullsized":"2024-09-21T05:46:19.199978+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:52.503589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183115+0000","last_change":"2024-09-21T05:46:17.184439+0000","last_active":"2024-09-21T05:46:19.183115+0000","last_peered":"2024-09-21T05:46:19.183115+0000","last_clean":"2024-09-21T05:46:19.183115+0000","last_became_active":"2024-09-21T05:46:17.184111+0000","last_became_peered":"2024-09-21T05:46:17.184111+0000","last_unstale":"2024-09-21T05:46:19.183115+0000","last_undegraded":"2024-09-21T05:46:19.183115+0000","last_fullsized":"2024-09-21T05:46:19.183115+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:19.945815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187912+0000","last_change":"2024-09-21T05:46:18.192051+0000","last_active":"2024-09-21T05:46:19.187912+0000","last_peered":"2024-09-21T05:46:19.187912+0000","last_clean":"2024-09-21T05:46:19.187912+0000","last_became_active":"2024-09-21T05:46:18.191747+0000","last_became_peered":"2024-09-21T05:46:18.191747+0000","last_unstale":"2024-09-21T05:46:19.187912+0000","last_undegraded":"2024-09-21T05:46:19.187912+0000","last_fullsized":"2024-09-21T05:46:19.187912+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:44.698448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443290+0000","last_change":"2024-09-21T05:46:17.200131+0000","last_active":"2024-09-21T05:46:19.443290+0000","last_peered":"2024-09-21T05:46:19.443290+0000","last_clean":"2024-09-21T05:46:19.443290+0000","last_became_active":"2024-09-21T05:46:17.199798+0000","last_became_peered":"2024-09-21T05:46:17.199798+0000","last_unstale":"2024-09-21T05:46:19.443290+0000","last_undegraded":"2024-09-21T05:46:19.443290+0000","last_fullsized":"2024-09-21T05:46:19.443290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:54:19.897956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188415+0000","last_change":"2024-09-21T05:46:18.197717+0000","last_active":"2024-09-21T05:46:19.188415+0000","last_peered":"2024-09-21T05:46:19.188415+0000","last_clean":"2024-09-21T05:46:19.188415+0000","last_became_active":"2024-09-21T05:46:18.197320+0000","last_became_peered":"2024-09-21T05:46:18.197320+0000","last_unstale":"2024-09-21T05:46:19.188415+0000","last_undegraded":"2024-09-21T05:46:19.188415+0000","last_fullsized":"2024-09-21T05:46:19.188415+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:48:50.723228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"96'3","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444256+0000","last_change":"2024-09-21T05:46:17.190720+0000","last_active":"2024-09-21T05:46:19.444256+0000","last_peered":"2024-09-21T05:46:19.444256+0000","last_clean":"2024-09-21T05:46:19.444256+0000","last_became_active":"2024-09-21T05:46:17.190431+0000","last_became_peered":"2024-09-21T05:46:17.190431+0000","last_unstale":"2024-09-21T05:46:19.444256+0000","last_undegraded":"2024-09-21T05:46:19.444256+0000","last_fullsized":"2024-09-21T05:46:19.444256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:21.544930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444170+0000","last_change":"2024-09-21T05:46:18.197885+0000","last_active":"2024-09-21T05:46:19.444170+0000","last_peered":"2024-09-21T05:46:19.444170+0000","last_clean":"2024-09-21T05:46:19.444170+0000","last_became_active":"2024-09-21T05:46:18.197569+0000","last_became_peered":"2024-09-21T05:46:18.197569+0000","last_unstale":"2024-09-21T05:46:19.444170+0000","last_undegraded":"2024-09-21T05:46:19.444170+0000","last_fullsized":"2024-09-21T05:46:19.444170+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:53.119749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"96'6","reported_seq":21,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188279+0000","last_change":"2024-09-21T05:46:17.189645+0000","last_active":"2024-09-21T05:46:19.188279+0000","last_peered":"2024-09-21T05:46:19.188279+0000","last_clean":"2024-09-21T05:46:19.188279+0000","last_became_active":"2024-09-21T05:46:17.189243+0000","last_became_peered":"2024-09-21T05:46:17.189243+0000","last_unstale":"2024-09-21T05:46:19.188279+0000","last_undegraded":"2024-09-21T05:46:19.188279+0000","last_fullsized":"2024-09-21T05:46:19.188279+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:38.660622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443159+0000","last_change":"2024-09-21T05:46:18.191157+0000","last_active":"2024-09-21T05:46:19.443159+0000","last_peered":"2024-09-21T05:46:19.443159+0000","last_clean":"2024-09-21T05:46:19.443159+0000","last_became_active":"2024-09-21T05:46:18.190646+0000","last_became_peered":"2024-09-21T05:46:18.190646+0000","last_unstale":"2024-09-21T05:46:19.443159+0000","last_undegraded":"2024-09-21T05:46:19.443159+0000","last_fullsized":"2024-09-21T05:46:19.443159+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:22.425139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202377+0000","last_change":"2024-09-21T05:46:17.189107+0000","last_active":"2024-09-21T05:46:19.202377+0000","last_peered":"2024-09-21T05:46:19.202377+0000","last_clean":"2024-09-21T05:46:19.202377+0000","last_became_active":"2024-09-21T05:46:17.188753+0000","last_became_peered":"2024-09-21T05:46:17.188753+0000","last_unstale":"2024-09-21T05:46:19.202377+0000","last_undegraded":"2024-09-21T05:46:19.202377+0000","last_fullsized":"2024-09-21T05:46:19.202377+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:28.618603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183142+0000","last_change":"2024-09-21T05:46:18.191886+0000","last_active":"2024-09-21T05:46:19.183142+0000","last_peered":"2024-09-21T05:46:19.183142+0000","last_clean":"2024-09-21T05:46:19.183142+0000","last_became_active":"2024-09-21T05:46:18.191569+0000","last_became_peered":"2024-09-21T05:46:18.191569+0000","last_unstale":"2024-09-21T05:46:19.183142+0000","last_undegraded":"2024-09-21T05:46:19.183142+0000","last_fullsized":"2024-09-21T05:46:19.183142+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:31.838595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202491+0000","last_change":"2024-09-21T05:46:17.193638+0000","last_active":"2024-09-21T05:46:19.202491+0000","last_peered":"2024-09-21T05:46:19.202491+0000","last_clean":"2024-09-21T05:46:19.202491+0000","last_became_active":"2024-09-21T05:46:17.193269+0000","last_became_peered":"2024-09-21T05:46:17.193269+0000","last_unstale":"2024-09-21T05:46:19.202491+0000","last_undegraded":"2024-09-21T05:46:19.202491+0000","last_fullsized":"2024-09-21T05:46:19.202491+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:08.277143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186500+0000","last_change":"2024-09-21T05:46:18.190319+0000","last_active":"2024-09-21T05:46:19.186500+0000","last_peered":"2024-09-21T05:46:19.186500+0000","last_clean":"2024-09-21T05:46:19.186500+0000","last_became_active":"2024-09-21T05:46:18.189963+0000","last_became_peered":"2024-09-21T05:46:18.189963+0000","last_unstale":"2024-09-21T05:46:19.186500+0000","last_undegraded":"2024-09-21T05:46:19.186500+0000","last_fullsized":"2024-09-21T05:46:19.186500+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:05.463973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186646+0000","last_change":"2024-09-21T05:46:17.188129+0000","last_active":"2024-09-21T05:46:19.186646+0000","last_peered":"2024-09-21T05:46:19.186646+0000","last_clean":"2024-09-21T05:46:19.186646+0000","last_became_active":"2024-09-21T05:46:17.187315+0000","last_became_peered":"2024-09-21T05:46:17.187315+0000","last_unstale":"2024-09-21T05:46:19.186646+0000","last_undegraded":"2024-09-21T05:46:19.186646+0000","last_fullsized":"2024-09-21T05:46:19.186646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:14.478577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202077+0000","last_change":"2024-09-21T05:46:18.191336+0000","last_active":"2024-09-21T05:46:19.202077+0000","last_peered":"2024-09-21T05:46:19.202077+0000","last_clean":"2024-09-21T05:46:19.202077+0000","last_became_active":"2024-09-21T05:46:18.188921+0000","last_became_peered":"2024-09-21T05:46:18.188921+0000","last_unstale":"2024-09-21T05:46:19.202077+0000","last_undegraded":"2024-09-21T05:46:19.202077+0000","last_fullsized":"2024-09-21T05:46:19.202077+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:25.785814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"96'8","reported_seq":23,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183288+0000","last_change":"2024-09-21T05:46:17.187693+0000","last_active":"2024-09-21T05:46:19.183288+0000","last_peered":"2024-09-21T05:46:19.183288+0000","last_clean":"2024-09-21T05:46:19.183288+0000","last_became_active":"2024-09-21T05:46:17.187378+0000","last_became_peered":"2024-09-21T05:46:17.187378+0000","last_unstale":"2024-09-21T05:46:19.183288+0000","last_undegraded":"2024-09-21T05:46:19.183288+0000","last_fullsized":"2024-09-21T05:46:19.183288+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:56:09.816160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187230+0000","last_change":"2024-09-21T05:46:18.189439+0000","last_active":"2024-09-21T05:46:19.187230+0000","last_peered":"2024-09-21T05:46:19.187230+0000","last_clean":"2024-09-21T05:46:19.187230+0000","last_became_active":"2024-09-21T05:46:18.189119+0000","last_became_peered":"2024-09-21T05:46:18.189119+0000","last_unstale":"2024-09-21T05:46:19.187230+0000","last_undegraded":"2024-09-21T05:46:19.187230+0000","last_fullsized":"2024-09-21T05:46:19.187230+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:29:01.443807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187276+0000","last_change":"2024-09-21T05:46:17.189048+0000","last_active":"2024-09-21T05:46:19.187276+0000","last_peered":"2024-09-21T05:46:19.187276+0000","last_clean":"2024-09-21T05:46:19.187276+0000","last_became_active":"2024-09-21T05:46:17.188753+0000","last_became_peered":"2024-09-21T05:46:17.188753+0000","last_unstale":"2024-09-21T05:46:19.187276+0000","last_undegraded":"2024-09-21T05:46:19.187276+0000","last_fullsized":"2024-09-21T05:46:19.187276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:26.332525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443934+0000","last_change":"2024-09-21T05:46:18.197100+0000","last_active":"2024-09-21T05:46:19.443934+0000","last_peered":"2024-09-21T05:46:19.443934+0000","last_clean":"2024-09-21T05:46:19.443934+0000","last_became_active":"2024-09-21T05:46:18.196788+0000","last_became_peered":"2024-09-21T05:46:18.196788+0000","last_unstale":"2024-09-21T05:46:19.443934+0000","last_undegraded":"2024-09-21T05:46:19.443934+0000","last_fullsized":"2024-09-21T05:46:19.443934+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:26.287510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187326+0000","last_change":"2024-09-21T05:46:17.184825+0000","last_active":"2024-09-21T05:46:19.187326+0000","last_peered":"2024-09-21T05:46:19.187326+0000","last_clean":"2024-09-21T05:46:19.187326+0000","last_became_active":"2024-09-21T05:46:17.184487+0000","last_became_peered":"2024-09-21T05:46:17.184487+0000","last_unstale":"2024-09-21T05:46:19.187326+0000","last_undegraded":"2024-09-21T05:46:19.187326+0000","last_fullsized":"2024-09-21T05:46:19.187326+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:09.619020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444338+0000","last_change":"2024-09-21T05:46:18.189396+0000","last_active":"2024-09-21T05:46:19.444338+0000","last_peered":"2024-09-21T05:46:19.444338+0000","last_clean":"2024-09-21T05:46:19.444338+0000","last_became_active":"2024-09-21T05:46:18.189073+0000","last_became_peered":"2024-09-21T05:46:18.189073+0000","last_unstale":"2024-09-21T05:46:19.444338+0000","last_undegraded":"2024-09-21T05:46:19.444338+0000","last_fullsized":"2024-09-21T05:46:19.444338+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:35.928619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199508+0000","last_change":"2024-09-21T05:46:17.199865+0000","last_active":"2024-09-21T05:46:19.199508+0000","last_peered":"2024-09-21T05:46:19.199508+0000","last_clean":"2024-09-21T05:46:19.199508+0000","last_became_active":"2024-09-21T05:46:17.198799+0000","last_became_peered":"2024-09-21T05:46:17.198799+0000","last_unstale":"2024-09-21T05:46:19.199508+0000","last_undegraded":"2024-09-21T05:46:19.199508+0000","last_fullsized":"2024-09-21T05:46:19.199508+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:22.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202342+0000","last_change":"2024-09-21T05:46:18.194951+0000","last_active":"2024-09-21T05:46:19.202342+0000","last_peered":"2024-09-21T05:46:19.202342+0000","last_clean":"2024-09-21T05:46:19.202342+0000","last_became_active":"2024-09-21T05:46:18.194626+0000","last_became_peered":"2024-09-21T05:46:18.194626+0000","last_unstale":"2024-09-21T05:46:19.202342+0000","last_undegraded":"2024-09-21T05:46:19.202342+0000","last_fullsized":"2024-09-21T05:46:19.202342+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:34.154658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199283+0000","last_change":"2024-09-21T05:46:17.198720+0000","last_active":"2024-09-21T05:46:19.199283+0000","last_peered":"2024-09-21T05:46:19.199283+0000","last_clean":"2024-09-21T05:46:19.199283+0000","last_became_active":"2024-09-21T05:46:17.197128+0000","last_became_peered":"2024-09-21T05:46:17.197128+0000","last_unstale":"2024-09-21T05:46:19.199283+0000","last_undegraded":"2024-09-21T05:46:19.199283+0000","last_fullsized":"2024-09-21T05:46:19.199283+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:15.132838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202361+0000","last_change":"2024-09-21T05:46:18.182810+0000","last_active":"2024-09-21T05:46:19.202361+0000","last_peered":"2024-09-21T05:46:19.202361+0000","last_clean":"2024-09-21T05:46:19.202361+0000","last_became_active":"2024-09-21T05:46:18.182468+0000","last_became_peered":"2024-09-21T05:46:18.182468+0000","last_unstale":"2024-09-21T05:46:19.202361+0000","last_undegraded":"2024-09-21T05:46:19.202361+0000","last_fullsized":"2024-09-21T05:46:19.202361+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:27.025794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444170+0000","last_change":"2024-09-21T05:46:17.186773+0000","last_active":"2024-09-21T05:46:19.444170+0000","last_peered":"2024-09-21T05:46:19.444170+0000","last_clean":"2024-09-21T05:46:19.444170+0000","last_became_active":"2024-09-21T05:46:17.186333+0000","last_became_peered":"2024-09-21T05:46:17.186333+0000","last_unstale":"2024-09-21T05:46:19.444170+0000","last_undegraded":"2024-09-21T05:46:19.444170+0000","last_fullsized":"2024-09-21T05:46:19.444170+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:27.896430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202455+0000","last_change":"2024-09-21T05:46:18.201418+0000","last_active":"2024-09-21T05:46:19.202455+0000","last_peered":"2024-09-21T05:46:19.202455+0000","last_clean":"2024-09-21T05:46:19.202455+0000","last_became_active":"2024-09-21T05:46:18.201051+0000","last_became_peered":"2024-09-21T05:46:18.201051+0000","last_unstale":"2024-09-21T05:46:19.202455+0000","last_undegraded":"2024-09-21T05:46:19.202455+0000","last_fullsized":"2024-09-21T05:46:19.202455+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:36.520750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443657+0000","last_change":"2024-09-21T05:46:17.189743+0000","last_active":"2024-09-21T05:46:19.443657+0000","last_peered":"2024-09-21T05:46:19.443657+0000","last_clean":"2024-09-21T05:46:19.443657+0000","last_became_active":"2024-09-21T05:46:17.189441+0000","last_became_peered":"2024-09-21T05:46:17.189441+0000","last_unstale":"2024-09-21T05:46:19.443657+0000","last_undegraded":"2024-09-21T05:46:19.443657+0000","last_fullsized":"2024-09-21T05:46:19.443657+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:44.778540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186788+0000","last_change":"2024-09-21T05:46:18.201920+0000","last_active":"2024-09-21T05:46:19.186788+0000","last_peered":"2024-09-21T05:46:19.186788+0000","last_clean":"2024-09-21T05:46:19.186788+0000","last_became_active":"2024-09-21T05:46:18.201556+0000","last_became_peered":"2024-09-21T05:46:18.201556+0000","last_unstale":"2024-09-21T05:46:19.186788+0000","last_undegraded":"2024-09-21T05:46:19.186788+0000","last_fullsized":"2024-09-21T05:46:19.186788+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:58.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.198893+0000","last_change":"2024-09-21T05:46:17.201319+0000","last_active":"2024-09-21T05:46:19.198893+0000","last_peered":"2024-09-21T05:46:19.198893+0000","last_clean":"2024-09-21T05:46:19.198893+0000","last_became_active":"2024-09-21T05:46:17.198548+0000","last_became_peered":"2024-09-21T05:46:17.198548+0000","last_unstale":"2024-09-21T05:46:19.198893+0000","last_undegraded":"2024-09-21T05:46:19.198893+0000","last_fullsized":"2024-09-21T05:46:19.198893+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:36.220460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.200006+0000","last_change":"2024-09-21T05:46:18.194698+0000","last_active":"2024-09-21T05:46:19.200006+0000","last_peered":"2024-09-21T05:46:19.200006+0000","last_clean":"2024-09-21T05:46:19.200006+0000","last_became_active":"2024-09-21T05:46:18.194007+0000","last_became_peered":"2024-09-21T05:46:18.194007+0000","last_unstale":"2024-09-21T05:46:19.200006+0000","last_undegraded":"2024-09-21T05:46:19.200006+0000","last_fullsized":"2024-09-21T05:46:19.200006+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:22:47.067006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"96'2","reported_seq":17,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.444089+0000","last_change":"2024-09-21T05:46:17.188083+0000","last_active":"2024-09-21T05:46:19.444089+0000","last_peered":"2024-09-21T05:46:19.444089+0000","last_clean":"2024-09-21T05:46:19.444089+0000","last_became_active":"2024-09-21T05:46:17.187744+0000","last_became_peered":"2024-09-21T05:46:17.187744+0000","last_unstale":"2024-09-21T05:46:19.444089+0000","last_undegraded":"2024-09-21T05:46:19.444089+0000","last_fullsized":"2024-09-21T05:46:19.444089+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:28.100901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183188+0000","last_change":"2024-09-21T05:46:18.201490+0000","last_active":"2024-09-21T05:46:19.183188+0000","last_peered":"2024-09-21T05:46:19.183188+0000","last_clean":"2024-09-21T05:46:19.183188+0000","last_became_active":"2024-09-21T05:46:18.201183+0000","last_became_peered":"2024-09-21T05:46:18.201183+0000","last_unstale":"2024-09-21T05:46:19.183188+0000","last_undegraded":"2024-09-21T05:46:19.183188+0000","last_fullsized":"2024-09-21T05:46:19.183188+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:25:18.582096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"97'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.201547+0000","last_change":"2024-09-21T05:46:17.196627+0000","last_active":"2024-09-21T05:46:19.201547+0000","last_peered":"2024-09-21T05:46:19.201547+0000","last_clean":"2024-09-21T05:46:19.201547+0000","last_became_active":"2024-09-21T05:46:17.196294+0000","last_became_peered":"2024-09-21T05:46:17.196294+0000","last_unstale":"2024-09-21T05:46:19.201547+0000","last_undegraded":"2024-09-21T05:46:19.201547+0000","last_fullsized":"2024-09-21T05:46:19.201547+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:10.745490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.183017+0000","last_change":"2024-09-21T05:46:18.194909+0000","last_active":"2024-09-21T05:46:19.183017+0000","last_peered":"2024-09-21T05:46:19.183017+0000","last_clean":"2024-09-21T05:46:19.183017+0000","last_became_active":"2024-09-21T05:46:18.194570+0000","last_became_peered":"2024-09-21T05:46:18.194570+0000","last_unstale":"2024-09-21T05:46:19.183017+0000","last_undegraded":"2024-09-21T05:46:19.183017+0000","last_fullsized":"2024-09-21T05:46:19.183017+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:31:35.950262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199589+0000","last_change":"2024-09-21T05:46:17.199645+0000","last_active":"2024-09-21T05:46:19.199589+0000","last_peered":"2024-09-21T05:46:19.199589+0000","last_clean":"2024-09-21T05:46:19.199589+0000","last_became_active":"2024-09-21T05:46:17.198926+0000","last_became_peered":"2024-09-21T05:46:17.198926+0000","last_unstale":"2024-09-21T05:46:19.199589+0000","last_undegraded":"2024-09-21T05:46:19.199589+0000","last_fullsized":"2024-09-21T05:46:19.199589+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:13.824663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187185+0000","last_change":"2024-09-21T05:46:18.195700+0000","last_active":"2024-09-21T05:46:19.187185+0000","last_peered":"2024-09-21T05:46:19.187185+0000","last_clean":"2024-09-21T05:46:19.187185+0000","last_became_active":"2024-09-21T05:46:18.195309+0000","last_became_peered":"2024-09-21T05:46:18.195309+0000","last_unstale":"2024-09-21T05:46:19.187185+0000","last_undegraded":"2024-09-21T05:46:19.187185+0000","last_fullsized":"2024-09-21T05:46:19.187185+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:50:53.523226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.188530+0000","last_change":"2024-09-21T05:46:17.186451+0000","last_active":"2024-09-21T05:46:19.188530+0000","last_peered":"2024-09-21T05:46:19.188530+0000","last_clean":"2024-09-21T05:46:19.188530+0000","last_became_active":"2024-09-21T05:46:17.186064+0000","last_became_peered":"2024-09-21T05:46:17.186064+0000","last_unstale":"2024-09-21T05:46:19.188530+0000","last_undegraded":"2024-09-21T05:46:19.188530+0000","last_fullsized":"2024-09-21T05:46:19.188530+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:41.170106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202552+0000","last_change":"2024-09-21T05:46:18.193241+0000","last_active":"2024-09-21T05:46:19.202552+0000","last_peered":"2024-09-21T05:46:19.202552+0000","last_clean":"2024-09-21T05:46:19.202552+0000","last_became_active":"2024-09-21T05:46:18.192433+0000","last_became_peered":"2024-09-21T05:46:18.192433+0000","last_unstale":"2024-09-21T05:46:19.202552+0000","last_undegraded":"2024-09-21T05:46:19.202552+0000","last_fullsized":"2024-09-21T05:46:19.202552+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:00:58.559819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.199378+0000","last_change":"2024-09-21T05:46:17.196529+0000","last_active":"2024-09-21T05:46:19.199378+0000","last_peered":"2024-09-21T05:46:19.199378+0000","last_clean":"2024-09-21T05:46:19.199378+0000","last_became_active":"2024-09-21T05:46:17.196201+0000","last_became_peered":"2024-09-21T05:46:17.196201+0000","last_unstale":"2024-09-21T05:46:19.199378+0000","last_undegraded":"2024-09-21T05:46:19.199378+0000","last_fullsized":"2024-09-21T05:46:19.199378+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:08:46.353796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.186908+0000","last_change":"2024-09-21T05:46:18.188357+0000","last_active":"2024-09-21T05:46:19.186908+0000","last_peered":"2024-09-21T05:46:19.186908+0000","last_clean":"2024-09-21T05:46:19.186908+0000","last_became_active":"2024-09-21T05:46:18.186104+0000","last_became_peered":"2024-09-21T05:46:18.186104+0000","last_unstale":"2024-09-21T05:46:19.186908+0000","last_undegraded":"2024-09-21T05:46:19.186908+0000","last_fullsized":"2024-09-21T05:46:19.186908+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:56.560190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187040+0000","last_change":"2024-09-21T05:46:17.187748+0000","last_active":"2024-09-21T05:46:19.187040+0000","last_peered":"2024-09-21T05:46:19.187040+0000","last_clean":"2024-09-21T05:46:19.187040+0000","last_became_active":"2024-09-21T05:46:17.187414+0000","last_became_peered":"2024-09-21T05:46:17.187414+0000","last_unstale":"2024-09-21T05:46:19.187040+0000","last_undegraded":"2024-09-21T05:46:19.187040+0000","last_fullsized":"2024-09-21T05:46:19.187040+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:21:25.281501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202096+0000","last_change":"2024-09-21T05:46:18.188393+0000","last_active":"2024-09-21T05:46:19.202096+0000","last_peered":"2024-09-21T05:46:19.202096+0000","last_clean":"2024-09-21T05:46:19.202096+0000","last_became_active":"2024-09-21T05:46:18.188084+0000","last_became_peered":"2024-09-21T05:46:18.188084+0000","last_unstale":"2024-09-21T05:46:19.202096+0000","last_undegraded":"2024-09-21T05:46:19.202096+0000","last_fullsized":"2024-09-21T05:46:19.202096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:33:38.245930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443129+0000","last_change":"2024-09-21T05:46:17.190702+0000","last_active":"2024-09-21T05:46:19.443129+0000","last_peered":"2024-09-21T05:46:19.443129+0000","last_clean":"2024-09-21T05:46:19.443129+0000","last_became_active":"2024-09-21T05:46:17.190334+0000","last_became_peered":"2024-09-21T05:46:17.190334+0000","last_unstale":"2024-09-21T05:46:19.443129+0000","last_undegraded":"2024-09-21T05:46:19.443129+0000","last_fullsized":"2024-09-21T05:46:19.443129+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:49:32.993941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.187028+0000","last_change":"2024-09-21T05:46:18.196307+0000","last_active":"2024-09-21T05:46:19.187028+0000","last_peered":"2024-09-21T05:46:19.187028+0000","last_clean":"2024-09-21T05:46:19.187028+0000","last_became_active":"2024-09-21T05:46:18.195976+0000","last_became_peered":"2024-09-21T05:46:18.195976+0000","last_unstale":"2024-09-21T05:46:19.187028+0000","last_undegraded":"2024-09-21T05:46:19.187028+0000","last_fullsized":"2024-09-21T05:46:19.187028+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:47.788275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202526+0000","last_change":"2024-09-21T05:46:17.200565+0000","last_active":"2024-09-21T05:46:19.202526+0000","last_peered":"2024-09-21T05:46:19.202526+0000","last_clean":"2024-09-21T05:46:19.202526+0000","last_became_active":"2024-09-21T05:46:17.200201+0000","last_became_peered":"2024-09-21T05:46:17.200201+0000","last_unstale":"2024-09-21T05:46:19.202526+0000","last_undegraded":"2024-09-21T05:46:19.202526+0000","last_fullsized":"2024-09-21T05:46:19.202526+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:52.617508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.202785+0000","last_change":"2024-09-21T05:46:18.195930+0000","last_active":"2024-09-21T05:46:19.202785+0000","last_peered":"2024-09-21T05:46:19.202785+0000","last_clean":"2024-09-21T05:46:19.202785+0000","last_became_active":"2024-09-21T05:46:18.195587+0000","last_became_peered":"2024-09-21T05:46:18.195587+0000","last_unstale":"2024-09-21T05:46:19.202785+0000","last_undegraded":"2024-09-21T05:46:19.202785+0000","last_fullsized":"2024-09-21T05:46:19.202785+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:17.164496+0000","last_clean_scrub_stamp":"2024-09-21T05:46:17.164496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:10:33.275939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"96'1","reported_seq":16,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443259+0000","last_change":"2024-09-21T05:46:17.191544+0000","last_active":"2024-09-21T05:46:19.443259+0000","last_peered":"2024-09-21T05:46:19.443259+0000","last_clean":"2024-09-21T05:46:19.443259+0000","last_became_active":"2024-09-21T05:46:17.191192+0000","last_became_peered":"2024-09-21T05:46:17.191192+0000","last_unstale":"2024-09-21T05:46:19.443259+0000","last_undegraded":"2024-09-21T05:46:19.443259+0000","last_fullsized":"2024-09-21T05:46:19.443259+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:46:16.158487+0000","last_clean_scrub_stamp":"2024-09-21T05:46:16.158487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:09.759511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-21T05:46:19.443574+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:46:19.443574+0000","last_peered":"2024-09-21T05:46:19.443574+0000","last_clean":"2024-09-21T05:46:19.443574+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:46:19.443574+0000","last_undegraded":"2024-09-21T05:46:19.443574+0000","last_fullsized":"2024-09-21T05:46:19.443574+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32948,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338892,"statfs":{"total":96636764160,"available":96603025408,"internally_reserved":0,"allocated":1794048,"data_stored":1104740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509529,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1212416,"data_stored":515996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542233,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1200128,"data_stored":514912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542233,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":514330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36528,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335312,"statfs":{"total":96636764160,"available":96599359488,"internally_reserved":0,"allocated":1208320,"data_stored":515894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542235,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36516,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335324,"statfs":{"total":96636764160,"available":96599371776,"internally_reserved":0,"allocated":1196032,"data_stored":514294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542233,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36512,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335328,"statfs":{"total":96636764160,"available":96599375872,"internally_reserved":0,"allocated":1191936,"data_stored":514260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542233,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37100,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334740,"statfs":{"total":96636764160,"available":96598773760,"internally_reserved":0,"allocated":1789952,"data_stored":1105210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:46:22.645 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:46:22.645 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.804+0000 7f0eec4f9640 1 -- 172.21.15.22:0/888942169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed0005350 msgr2=0x7f0ed000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.804+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/888942169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed0005350 0x7f0ed000b9c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009ed0 tx=0x7f0ed0005830 comp rx=0 tx=0).stop 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.804+0000 7f0eec4f9640 1 -- 172.21.15.22:0/888942169 shutdown_connections 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.804+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/888942169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ed0005350 0x7f0ed000b9c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.804+0000 7f0eec4f9640 1 -- 172.21.15.22:0/888942169 >> 172.21.15.22:0/888942169 conn(0x7f0ee40b3f10 msgr2=0x7f0ee40b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.805+0000 7f0eec4f9640 1 -- 172.21.15.22:0/888942169 shutdown_connections 2024-09-21T05:46:22.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.805+0000 7f0eec4f9640 1 -- 172.21.15.22:0/888942169 wait complete. 2024-09-21T05:46:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.806+0000 7f0eec4f9640 1 Processor -- start 2024-09-21T05:46:22.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.806+0000 7f0eec4f9640 1 -- start start 2024-09-21T05:46:22.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eec4f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ed0005350 0x7f0ee40c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eec4f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ee4152c10 0x7f0ee4154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eec4f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 0x7f0ee41f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eec4f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee40b5210 con 0x7f0ee4155320 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eec4f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee40d2a40 con 0x7f0ee4152c10 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eec4f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee40b5390 con 0x7f0ed0005350 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 0x7f0ee41f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 0x7f0ee41f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55166/0 (socket says 172.21.15.22:55166) 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 -- 172.21.15.22:0/1569293923 learned_addr learned my addr 172.21.15.22:0/1569293923 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eea295640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ed0005350 0x7f0ee40c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0ee9a94640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ee4152c10 0x7f0ee4154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 -- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ed0005350 msgr2=0x7f0ee40c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ed0005350 0x7f0ee40c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 -- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ee4152c10 msgr2=0x7f0ee4154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ee4152c10 0x7f0ee4154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 -- 172.21.15.22:0/1569293923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee406ff00 con 0x7f0ee4155320 2024-09-21T05:46:22.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0ee9a94640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ee4152c10 0x7f0ee4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0eeaa96640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 0x7f0ee41f8c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009e20 tx=0x7f0edc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.808+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc004650 con 0x7f0ee4155320 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.809+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0edc03c480 con 0x7f0ee4155320 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.809+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc03c890 con 0x7f0ee4155320 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.809+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee4156e90 con 0x7f0ee4155320 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.809+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee4157450 con 0x7f0ee4155320 2024-09-21T05:46:22.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.809+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee40cf680 con 0x7f0ee4155320 2024-09-21T05:46:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.810+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0edc044020 con 0x7f0ee4155320 2024-09-21T05:46:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.811+0000 7f0edaffd640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0ec8078b90 0x7f0ec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.811+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f0edc0bbb90 con 0x7f0ee4155320 2024-09-21T05:46:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.811+0000 7f0eea295640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0ec8078b90 0x7f0ec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:22.814 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.812+0000 7f0eea295640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0ec8078b90 0x7f0ec807b0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0006fd0 tx=0x7f0ee0006d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:22.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:22.815+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0edc04b020 con 0x7f0ee4155320 2024-09-21T05:46:23.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.075+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0ee41f9b50 con 0x7f0ee4155320 2024-09-21T05:46:23.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.077+0000 7f0edaffd640 1 -- 172.21.15.22:0/1569293923 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f0edc0840f0 con 0x7f0ee4155320 2024-09-21T05:46:23.079 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:46:23.080 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:46:23.080 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:46:23.080 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:46:23.080 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:46:23.080 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0ec8078b90 msgr2=0x7f0ec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0ec8078b90 0x7f0ec807b0a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0ee0006fd0 tx=0x7f0ee0006d10 comp rx=0 tx=0).stop 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 msgr2=0x7f0ee41f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 0x7f0ee41f8c20 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009e20 tx=0x7f0edc004250 comp rx=0 tx=0).stop 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 shutdown_connections 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0ee4155320 0x7f0ee41f8c20 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0ee4152c10 0x7f0ee4154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0ec8078b90 0x7f0ec807b0a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 --2- 172.21.15.22:0/1569293923 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0ed0005350 0x7f0ee40c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 >> 172.21.15.22:0/1569293923 conn(0x7f0ee40b3f10 msgr2=0x7f0ee40b7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 shutdown_connections 2024-09-21T05:46:23.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.080+0000 7f0eec4f9640 1 -- 172.21.15.22:0/1569293923 wait complete. 2024-09-21T05:46:23.093 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 -- 172.21.15.22:0/204560312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa884155ce0 msgr2=0x7fa8840b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 --2- 172.21.15.22:0/204560312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa884155ce0 0x7fa8840b45a0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa88406eeb0 tx=0x7fa87402aac0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 -- 172.21.15.22:0/204560312 shutdown_connections 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 --2- 172.21.15.22:0/204560312 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa884155ce0 0x7fa8840b45a0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 --2- 172.21.15.22:0/204560312 >> v2:172.21.15.134:3300/0 conn(0x7fa884074420 0x7fa88406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 -- 172.21.15.22:0/204560312 >> 172.21.15.22:0/204560312 conn(0x7fa88406def0 msgr2=0x7fa88406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 -- 172.21.15.22:0/204560312 shutdown_connections 2024-09-21T05:46:23.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.207+0000 7fa88b807640 1 -- 172.21.15.22:0/204560312 wait complete. 2024-09-21T05:46:23.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.209+0000 7fa88b807640 1 Processor -- start 2024-09-21T05:46:23.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.209+0000 7fa88b807640 1 -- start start 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.210+0000 7fa88b807640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa884074420 0x7fa884149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.210+0000 7fa88b807640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa884155ce0 0x7fa884157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.210+0000 7fa88b807640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 0x7fa8840d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa88b807640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa8840b4fe0 con 0x7fa8840b28e0 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa88b807640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8840b4e60 con 0x7fa884155ce0 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa88b807640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa8840b5160 con 0x7fa884074420 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 0x7fa8840d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 0x7fa8840d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55182/0 (socket says 172.21.15.22:55182) 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 -- 172.21.15.22:0/1361949886 learned_addr learned my addr 172.21.15.22:0/1361949886 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa888da2640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa884155ce0 0x7fa884157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.214 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa8895a3640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa884074420 0x7fa884149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 -- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa884074420 msgr2=0x7fa884149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa884074420 0x7fa884149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 -- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa884155ce0 msgr2=0x7fa884157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa884155ce0 0x7fa884157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 -- 172.21.15.22:0/1361949886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8840b1e40 con 0x7fa8840b28e0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa889da4640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 0x7fa8840d01f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa87402b0b0 tx=0x7fa874003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874004250 con 0x7fa8840b28e0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8740043f0 con 0x7fa8840b28e0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8841588d0 con 0x7fa8840b28e0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.211+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874004840 con 0x7fa8840b28e0 2024-09-21T05:46:23.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.212+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8840b20e0 con 0x7fa8840b28e0 2024-09-21T05:46:23.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.213+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8840b5680 con 0x7fa8840b28e0 2024-09-21T05:46:23.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.213+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa874049430 con 0x7fa8840b28e0 2024-09-21T05:46:23.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.214+0000 7fa881ffb640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078bc0 0x7fa87007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.214+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fa8740bbb40 con 0x7fa8840b28e0 2024-09-21T05:46:23.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.214+0000 7fa8895a3640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078bc0 0x7fa87007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.215+0000 7fa8895a3640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078bc0 0x7fa87007b0d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00bb50 tx=0x7fa87c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:23.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.218+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa874032030 con 0x7fa8840b28e0 2024-09-21T05:46:23.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.444+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa88415fbd0 con 0x7fa8840b28e0 2024-09-21T05:46:23.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.445+0000 7fa881ffb640 1 -- 172.21.15.22:0/1361949886 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5231 (secure 0 0 0) 0x7fa874083920 con 0x7fa8840b28e0 2024-09-21T05:46:23.447 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:23.447 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"btime":"2024-09-21T05:46:21:204567+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:46:20.588210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7482":{"gid":7482,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1062944206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1062944206},{"type":"v1","addr":"172.21.15.22:6834","nonce":1062944206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7482,"qdb_cluster":[7482]},"id":7}]} 2024-09-21T05:46:23.447 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 84 2024-09-21T05:46:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078bc0 msgr2=0x7fa87007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078bc0 0x7fa87007b0d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00bb50 tx=0x7fa87c00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 msgr2=0x7fa8840d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 0x7fa8840d01f0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa87402b0b0 tx=0x7fa874003ce0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 shutdown_connections 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8840b28e0 0x7fa8840d01f0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa884155ce0 0x7fa884157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa870078bc0 0x7fa87007b0d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 --2- 172.21.15.22:0/1361949886 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa884074420 0x7fa884149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 >> 172.21.15.22:0/1361949886 conn(0x7fa88406def0 msgr2=0x7fa884154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 shutdown_connections 2024-09-21T05:46:23.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.447+0000 7fa88b807640 1 -- 172.21.15.22:0/1361949886 wait complete. 2024-09-21T05:46:23.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:46:17.567401+0000', 'modified': '2024-09-21T05:46:20.588210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7482': {'gid': 7482, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.22:6834/1062944206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 1062944206}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 1062944206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7482, 'qdb_cluster': [7482]} 2024-09-21T05:46:23.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:46:23.864 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T05:46:23.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/3323323791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40d4c20 msgr2=0x7f4db40b4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/3323323791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40d4c20 0x7f4db40b4040 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005fd0 tx=0x7f4dac02aa90 comp rx=0 tx=0).stop 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/3323323791 shutdown_connections 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/3323323791 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40d4c20 0x7f4db40b4040 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/3323323791 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db40d0ce0 0x7f4db40b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/3323323791 >> 172.21.15.22:0/3323323791 conn(0x7f4db408bfc0 msgr2=0x7f4db409d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/3323323791 shutdown_connections 2024-09-21T05:46:23.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.986+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/3323323791 wait complete. 2024-09-21T05:46:23.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.988+0000 7f4dbc6d5640 1 Processor -- start 2024-09-21T05:46:23.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.988+0000 7f4dbc6d5640 1 -- start start 2024-09-21T05:46:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbc6d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 0x7f4db415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbc6d5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db40d0ce0 0x7f4db415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbc6d5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db40cdd30 0x7f4db415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbc6d5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4db40b4ab0 con 0x7f4db40cf0b0 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbc6d5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4db40b4930 con 0x7f4db40d0ce0 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbc6d5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4db40b4c30 con 0x7f4db40cdd30 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 0x7f4db415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 0x7f4db415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55204/0 (socket says 172.21.15.22:55204) 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 -- 172.21.15.22:0/2890005386 learned_addr learned my addr 172.21.15.22:0/2890005386 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db40d0ce0 0x7f4db415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dbac72640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db40cdd30 0x7f4db415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 -- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db40cdd30 msgr2=0x7f4db415c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db40cdd30 0x7f4db415c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 -- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db40d0ce0 msgr2=0x7f4db415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.989+0000 7f4dba471640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db40d0ce0 0x7f4db415bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4dba471640 1 -- 172.21.15.22:0/2890005386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db4160940 con 0x7f4db40cf0b0 2024-09-21T05:46:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4dba471640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 0x7f4db415d4c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e790 tx=0x7f4db000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4daaffd640 1 -- 172.21.15.22:0/2890005386 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db000eea0 con 0x7f4db40cf0b0 2024-09-21T05:46:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4daaffd640 1 -- 172.21.15.22:0/2890005386 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4db000cdb0 con 0x7f4db40cf0b0 2024-09-21T05:46:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4daaffd640 1 -- 172.21.15.22:0/2890005386 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0010690 con 0x7f4db40cf0b0 2024-09-21T05:46:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db4160e10 con 0x7f4db40cf0b0 2024-09-21T05:46:23.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.990+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db40cd530 con 0x7f4db40cf0b0 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.992+0000 7f4daaffd640 1 -- 172.21.15.22:0/2890005386 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4db0010830 con 0x7f4db40cf0b0 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.992+0000 7f4daaffd640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4d98078b90 0x7f4d9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.993+0000 7f4daaffd640 1 -- 172.21.15.22:0/2890005386 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f4db009b750 con 0x7f4db40cf0b0 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.993+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4d98078b90 0x7f4d9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.993+0000 7f4dc3641740 1 Processor -- start 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.993+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4d98078b90 0x7f4d9807b0a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4da4007fd0 tx=0x7f4da4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:23.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.993+0000 7f4dc3641740 1 -- start start 2024-09-21T05:46:23.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.994+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x56347585cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.994+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347585d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x56347587a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4dc3641740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5634757a5820 con 0x5634754623b0 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563475794c50 con 0x56347546a590 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563475798a60 con 0x56347548b270 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347585d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347585d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55218/0 (socket says 172.21.15.22:55218) 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 -- 172.21.15.22:0/3516005500 learned_addr learned my addr 172.21.15.22:0/3516005500 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4dba471640 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x56347585cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4dbac72640 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x56347587a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 -- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 msgr2=0x56347585cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x56347585cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 -- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 msgr2=0x56347587a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:23.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x56347587a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 -- 172.21.15.22:0/3516005500 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634754f6850 con 0x5634754623b0 2024-09-21T05:46:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4db9c70640 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347585d4e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005240 tx=0x7f4da4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dc77a7b0f6a675 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4d97fff640 1 -- 172.21.15.22:0/3516005500 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da400e500 con 0x5634754623b0 2024-09-21T05:46:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4d97fff640 1 -- 172.21.15.22:0/3516005500 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da400e6a0 con 0x5634754623b0 2024-09-21T05:46:23.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.995+0000 7f4d97fff640 1 -- 172.21.15.22:0/3516005500 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da400eab0 con 0x5634754623b0 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 -- 172.21.15.22:0/3516005500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 msgr2=0x56347585d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347585d4e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005240 tx=0x7f4da4011000 comp rx=0 tx=0).stop 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 -- 172.21.15.22:0/3516005500 shutdown_connections 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x56347587a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347585d4e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 --2- 172.21.15.22:0/3516005500 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x56347585cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 -- 172.21.15.22:0/3516005500 >> 172.21.15.22:0/3516005500 conn(0x56347547f3c0 msgr2=0x563475833260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 -- 172.21.15.22:0/3516005500 shutdown_connections 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.996+0000 7f4dc3641740 1 -- 172.21.15.22:0/3516005500 wait complete. 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.997+0000 7f4dc3641740 1 -- start start 2024-09-21T05:46:24.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.997+0000 7f4dc3641740 1 Processor -- start 2024-09-21T05:46:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5634754623b0 0x5634758714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 0x56347585a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347548b270 0x563475856da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563475505620 con 0x56347546a590 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56347573a5f0 con 0x56347548b270 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56347585da20 con 0x5634754623b0 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dc3641740 10 client.0 _ll_register_callbacks cb 0x56347573e0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 0x56347585a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 0x56347585a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55234/0 (socket says 172.21.15.22:55234) 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 -- 172.21.15.22:0/811258439 learned_addr learned my addr 172.21.15.22:0/811258439 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dba471640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5634754623b0 0x5634758714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4dbac72640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347548b270 0x563475856da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 -- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5634754623b0 msgr2=0x5634758714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5634754623b0 0x5634758714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 -- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347548b270 msgr2=0x563475856da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347548b270 0x563475856da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 -- 172.21.15.22:0/811258439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347556d6b0 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4db9c70640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 0x56347585a470 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4da4014000 tx=0x7f4da400fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7800ce0d1b5e14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:23.999+0000 7f4d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4d957fa640 1 -- 172.21.15.22:0/811258439 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4021860 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4d957fa640 1 -- 172.21.15.22:0/811258439 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f4da4021a00 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da400e6a0 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4d957fa640 1 -- 172.21.15.22:0/811258439 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4da4032050 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 10 client.7614 fetch_fsmap learned FSMap version 84 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x5634758559c0 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4d957fa640 1 -- 172.21.15.22:0/811258439 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da400e6a0 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4d957fa640 1 -- 172.21.15.22:0/811258439 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f4da4035020 con 0x56347546a590 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 10 client.7614 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 10 client.7614 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 20 client.7614 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475449e40 0x56347588a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 4 client.7614 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:46:24.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634757a7770 con 0x563475449e40 2024-09-21T05:46:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.000+0000 7f4dba471640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475449e40 0x56347588a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.001+0000 7f4d957fa640 10 client.7614 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:24.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.001+0000 7f4dba471640 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475449e40 0x56347588a600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e610ea06a8a8b781 server_cookie=2a4b0c91a120c53d in_seq=0 out_seq=0 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.001+0000 7f4d957fa640 1 -- 172.21.15.22:0/811258439 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5634757a7770 con 0x563475449e40 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.001+0000 7f4d957fa640 10 client.7614 handle_command_reply: tid=0 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.001+0000 7f4dc3641740 1 client.7614 shutdown 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 msgr2=0x56347585a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 0x56347585a470 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4da4014000 tx=0x7f4da400fa90 comp rx=0 tx=0).stop 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 shutdown_connections 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475449e40 0x56347588a600 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347548b270 0x563475856da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.009 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347546a590 0x56347585a470 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 --2- 172.21.15.22:0/811258439 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5634754623b0 0x5634758714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4d957fa640 0 client.7614 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 >> 172.21.15.22:0/811258439 conn(0x56347547f3c0 msgr2=0x56347582c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 shutdown_connections 2024-09-21T05:46:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 1 -- 172.21.15.22:0/811258439 wait complete. 2024-09-21T05:46:24.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.002+0000 7f4dc3641740 20 client.7614 trim_cache size 0 max 16384 2024-09-21T05:46:24.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.020+0000 7f4dc3641740 1 Processor -- start 2024-09-21T05:46:24.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.020+0000 7f4dc3641740 1 -- start start 2024-09-21T05:46:24.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634754623b0 0x563475878600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347546a590 0x563475870880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 0x563475854730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dc3641740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4201f90 con 0x56347548b270 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4db4200f70 con 0x5634754623b0 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4db414e640 con 0x56347546a590 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 0x563475854730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 0x563475854730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55236/0 (socket says 172.21.15.22:55236) 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 -- 172.21.15.22:0/1873413556 learned_addr learned my addr 172.21.15.22:0/1873413556 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dba471640 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634754623b0 0x563475878600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4db9c70640 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347546a590 0x563475870880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 -- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347546a590 msgr2=0x563475870880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347546a590 0x563475870880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 -- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634754623b0 msgr2=0x563475878600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:24.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634754623b0 0x563475878600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 -- 172.21.15.22:0/1873413556 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563475870640 con 0x56347548b270 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dba471640 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634754623b0 0x563475878600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4dbac72640 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 0x563475854730 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009d90 tx=0x7f4dac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afdf46f217ec89b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4d977fe640 1 -- 172.21.15.22:0/1873413556 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac046070 con 0x56347548b270 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4d977fe640 1 -- 172.21.15.22:0/1873413556 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dac02eae0 con 0x56347548b270 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.022+0000 7f4d977fe640 1 -- 172.21.15.22:0/1873413556 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac003970 con 0x56347548b270 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 -- 172.21.15.22:0/1873413556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 msgr2=0x563475854730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 0x563475854730 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009d90 tx=0x7f4dac004ab0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 -- 172.21.15.22:0/1873413556 shutdown_connections 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56347548b270 0x563475854730 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347546a590 0x563475870880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 --2- 172.21.15.22:0/1873413556 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5634754623b0 0x563475878600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 -- 172.21.15.22:0/1873413556 >> 172.21.15.22:0/1873413556 conn(0x56347547f3c0 msgr2=0x563475887560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 -- 172.21.15.22:0/1873413556 shutdown_connections 2024-09-21T05:46:24.028 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.023+0000 7f4dc3641740 1 -- 172.21.15.22:0/1873413556 wait complete. 2024-09-21T05:46:24.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.024+0000 7f4dc3641740 1 -- start start 2024-09-21T05:46:24.031 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.024+0000 7f4dc3641740 1 Processor -- start 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347584b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x5634758fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x5634758f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56347586eda0 con 0x5634754623b0 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563475855770 con 0x56347546a590 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563475887fb0 con 0x56347548b270 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dc3641740 10 client.0 _ll_register_callbacks cb 0x7f4db0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347584b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347584b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55240/0 (socket says 172.21.15.22:55240) 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 -- 172.21.15.22:0/2784611274 learned_addr learned my addr 172.21.15.22:0/2784611274 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dbac72640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x5634758f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.032 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x5634758fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 -- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 msgr2=0x5634758f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x5634758f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 -- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 msgr2=0x5634758fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x5634758fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 -- 172.21.15.22:0/2784611274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634758f6d70 con 0x5634754623b0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x5634758fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dbac72640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x5634758f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4dba471640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347584b920 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4db000c6a0 tx=0x7f4db0023040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32d16953d997aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.026+0000 7f4d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4d957fa640 1 -- 172.21.15.22:0/2784611274 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db00628f0 con 0x5634754623b0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4d957fa640 1 -- 172.21.15.22:0/2784611274 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f4db0010b20 con 0x5634754623b0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dac02eae0 con 0x5634754623b0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4d957fa640 1 -- 172.21.15.22:0/2784611274 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4db0010d20 con 0x5634754623b0 2024-09-21T05:46:24.033 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 10 client.7620 fetch_fsmap learned FSMap version 84 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x56347586e2b0 con 0x5634754623b0 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4d957fa640 1 -- 172.21.15.22:0/2784611274 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4db006f030 con 0x5634754623b0 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4d957fa640 1 -- 172.21.15.22:0/2784611274 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f4db006c040 con 0x5634754623b0 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 10 client.7620 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 10 client.7620 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 20 client.7620 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475854730 0x56347586f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 4 client.7620 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "status"}) -- 0x563475828f80 con 0x563475854730 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475854730 0x56347586f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.027+0000 7f4d957fa640 10 client.7620 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4db9c70640 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475854730 0x56347586f6b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3075cbd2495ca64 server_cookie=12be90046ec843ae in_seq=0 out_seq=0 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4d957fa640 1 -- 172.21.15.22:0/2784611274 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563475828f80 con 0x563475854730 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4d957fa640 10 client.7620 handle_command_reply: tid=0 2024-09-21T05:46:24.034 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 client.7620 shutdown 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 msgr2=0x56347584b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347584b920 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4db000c6a0 tx=0x7f4db0023040 comp rx=0 tx=0).stop 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 shutdown_connections 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x563475854730 0x56347586f6b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56347548b270 0x5634758f6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56347546a590 0x5634758fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4d957fa640 0 client.7620 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 --2- 172.21.15.22:0/2784611274 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5634754623b0 0x56347584b920 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 >> 172.21.15.22:0/2784611274 conn(0x56347547f3c0 msgr2=0x563475887560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 shutdown_connections 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 1 -- 172.21.15.22:0/2784611274 wait complete. 2024-09-21T05:46:24.035 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.028+0000 7f4dc3641740 20 client.7620 trim_cache size 0 max 16384 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 7482, 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 5.8471100690000002, 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 84, 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 97, 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 97, 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 10.477480272999999 2024-09-21T05:46:24.036 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:46:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4d98078b90 msgr2=0x7f4d9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4d98078b90 0x7f4d9807b0a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4da4007fd0 tx=0x7f4da4009040 comp rx=0 tx=0).stop 2024-09-21T05:46:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 msgr2=0x7f4db415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 0x7f4db415d4c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4db000e790 tx=0x7f4db000ec50 comp rx=0 tx=0).stop 2024-09-21T05:46:24.037 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 shutdown_connections 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4d98078b90 0x7f4d9807b0a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4db40cdd30 0x7f4db415c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4db40d0ce0 0x7f4db415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 --2- 172.21.15.22:0/2890005386 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4db40cf0b0 0x7f4db415d4c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 >> 172.21.15.22:0/2890005386 conn(0x7f4db408bfc0 msgr2=0x7f4db4161d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 shutdown_connections 2024-09-21T05:46:24.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.029+0000 7f4dbc6d5640 1 -- 172.21.15.22:0/2890005386 wait complete. 2024-09-21T05:46:24.042 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:46:24.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 -- 172.21.15.22:0/2164184676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c0806e0e0 msgr2=0x7f2c0806e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/2164184676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c0806e0e0 0x7f2c0806e4f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005fd0 tx=0x7f2c0002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 -- 172.21.15.22:0/2164184676 shutdown_connections 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/2164184676 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c0806e0e0 0x7f2c0806e4f0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/2164184676 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c080b66b0 0x7f2c080b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 -- 172.21.15.22:0/2164184676 >> 172.21.15.22:0/2164184676 conn(0x7f2c0809d7f0 msgr2=0x7f2c080d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 -- 172.21.15.22:0/2164184676 shutdown_connections 2024-09-21T05:46:24.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.156+0000 7f2c0d49b640 1 -- 172.21.15.22:0/2164184676 wait complete. 2024-09-21T05:46:24.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.158+0000 7f2c0d49b640 1 Processor -- start 2024-09-21T05:46:24.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.158+0000 7f2c0d49b640 1 -- start start 2024-09-21T05:46:24.160 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f81eb182640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c0d49b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c0806e0e0 0x7f2c0814dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c0d49b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 0x7f2c081506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c0d49b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c0815c6e0 0x7f2c08150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c0d49b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0806ea30 con 0x7f2c080b9970 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c0d49b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080d5050 con 0x7f2c0806e0e0 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c0d49b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f2c0806ebb0 con 0x7f2c0815c6e0 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.159+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 0x7f2c081506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 0x7f2c081506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55254/0 (socket says 172.21.15.22:55254) 2024-09-21T05:46:24.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 -- 172.21.15.22:0/186562749 learned_addr learned my addr 172.21.15.22:0/186562749 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c06ffd640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c0806e0e0 0x7f2c0814dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c077fe640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c0815c6e0 0x7f2c08150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 -- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c0815c6e0 msgr2=0x7f2c08150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c0815c6e0 0x7f2c08150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 -- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c0806e0e0 msgr2=0x7f2c0814dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c0806e0e0 0x7f2c0814dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 -- 172.21.15.22:0/186562749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c081e4340 con 0x7f2c080b9970 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c067fc640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 0x7f2c081506c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00d6d0 tx=0x7f2bfc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc017070 con 0x7f2c080b9970 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bfc005330 con 0x7f2c080b9970 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc009820 con 0x7f2c080b9970 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c08151290 con 0x7f2c080b9970 2024-09-21T05:46:24.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.160+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0814e2e0 con 0x7f2c080b9970 2024-09-21T05:46:24.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.161+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c080cb940 con 0x7f2c080b9970 2024-09-21T05:46:24.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.162+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2bfc009bd0 con 0x7f2c080b9970 2024-09-21T05:46:24.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.163+0000 7f2bef7fe640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2bf4078b90 0x7f2bf407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:24.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.163+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f2bfc09b070 con 0x7f2c080b9970 2024-09-21T05:46:24.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.163+0000 7f2c06ffd640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2bf4078b90 0x7f2bf407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:24.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.164+0000 7f2c06ffd640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2bf4078b90 0x7f2bf407b0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8007fd0 tx=0x7f2bf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:24.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.168+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2bfc062dc0 con 0x7f2c080b9970 2024-09-21T05:46:24.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.398+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2c0806f080 con 0x7f2c080b9970 2024-09-21T05:46:24.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.399+0000 7f2bef7fe640 1 -- 172.21.15.22:0/186562749 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f2bfc0677f0 con 0x7f2c080b9970 2024-09-21T05:46:24.401 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:46:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2bf4078b90 msgr2=0x7f2bf407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2bf4078b90 0x7f2bf407b0a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8007fd0 tx=0x7f2bf8009040 comp rx=0 tx=0).stop 2024-09-21T05:46:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 msgr2=0x7f2c081506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:24.403 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 0x7f2c081506c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00d6d0 tx=0x7f2bfc00db90 comp rx=0 tx=0).stop 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 shutdown_connections 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2bf4078b90 0x7f2bf407b0a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f2c0815c6e0 0x7f2c08150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2c080b9970 0x7f2c081506c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 --2- 172.21.15.22:0/186562749 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f2c0806e0e0 0x7f2c0814dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 >> 172.21.15.22:0/186562749 conn(0x7f2c0809d7f0 msgr2=0x7f2c080b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 shutdown_connections 2024-09-21T05:46:24.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24.401+0000 7f2c0d49b640 1 -- 172.21.15.22:0/186562749 wait complete. 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:46:24.415 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:46:24.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.416 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:24.432 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42359sec preferred_lft 42359sec 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:46:24.433 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:46:24.434 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:24.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.435 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:46:24.496 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:46:24.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.498 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:46:24.554 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:46:24.554 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:46:24.555 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:46:24.555 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:46:24.555 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:46:24.555 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:46:24.555 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:46:24.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:46:24.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:46:24.555 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:46:24.555 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:46:24.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:46:24.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:24.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.684 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:46:24.700 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:46:24.700 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:46:24.700 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:46:24.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:46:24.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:46:24.766 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:24.822 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:46:24.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:46:24.822 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:24.878 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:46:24.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.878 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:24.973 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:24.973 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:46:24.973 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:46:24.984 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:24.984 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:46:24.984 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:46:24.984 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:46:24.984 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:46:24.989 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:24.989 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:46:24.990 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:46:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:46:24.991 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:46:24.992 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:46:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:24.992 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:25.018 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:46:25.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:46:25.115 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:25.121 INFO:teuthology.orchestra.run.smithi022.stdout:client7629 2024-09-21T05:46:25.121 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:25.121 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:25.138 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:25.138 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:25.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:25.195 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-21T05:46:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:6+0 records in 2024-09-21T05:46:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:6+0 records out 2024-09-21T05:46:25.291 INFO:teuthology.orchestra.run.smithi022.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.034972 s, 180 MB/s 2024-09-21T05:46:25.292 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:25.292 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:25.308 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:25.308 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:25.364 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:46:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 -- 172.21.15.22:0/2911364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 msgr2=0x7f91140ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 --2- 172.21.15.22:0/2911364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91140ceb50 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f9108005fd0 tx=0x7f910802a5d0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 -- 172.21.15.22:0/2911364967 shutdown_connections 2024-09-21T05:46:25.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 --2- 172.21.15.22:0/2911364967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91140ceb50 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 --2- 172.21.15.22:0/2911364967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91140d1d40 0x7f91140b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 -- 172.21.15.22:0/2911364967 >> 172.21.15.22:0/2911364967 conn(0x7f91140b3f10 msgr2=0x7f91140b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:25.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 -- 172.21.15.22:0/2911364967 shutdown_connections 2024-09-21T05:46:25.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.520+0000 7f911aebd640 1 -- 172.21.15.22:0/2911364967 wait complete. 2024-09-21T05:46:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.522+0000 7f911aebd640 1 Processor -- start 2024-09-21T05:46:25.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.522+0000 7f911aebd640 1 -- start start 2024-09-21T05:46:25.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911aebd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91140ccba0 0x7f91141f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:25.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911aebd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140d1d40 0x7f91141f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911aebd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91141f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911aebd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91140d12d0 con 0x7f911415c6f0 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911aebd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91140d1150 con 0x7f91140d1d40 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911aebd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f91140d1450 con 0x7f91140ccba0 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911945a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91141f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911945a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91141f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46476/0 (socket says 172.21.15.22:46476) 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f911945a640 1 -- 172.21.15.22:0/3924917489 learned_addr learned my addr 172.21.15.22:0/3924917489 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f9113fff640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140d1d40 0x7f91141f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.523+0000 7f9118c59640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91140ccba0 0x7f91141f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911945a640 1 -- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91140ccba0 msgr2=0x7f91141f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911945a640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91140ccba0 0x7f91141f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911945a640 1 -- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140d1d40 msgr2=0x7f91141f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:25.527 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911945a640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140d1d40 0x7f91141f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911945a640 1 -- 172.21.15.22:0/3924917489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911415dea0 con 0x7f911415c6f0 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911945a640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91141f3620 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f910c002230 tx=0x7f910c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c003b70 con 0x7f911415c6f0 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f910c003d10 con 0x7f911415c6f0 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911415e180 con 0x7f911415c6f0 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c005330 con 0x7f911415c6f0 2024-09-21T05:46:25.528 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.524+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911415e740 con 0x7f911415c6f0 2024-09-21T05:46:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.525+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91140d1970 con 0x7f911415c6f0 2024-09-21T05:46:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.526+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f910c0056e0 con 0x7f911415c6f0 2024-09-21T05:46:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.527+0000 7f91117fa640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9100078b90 0x7f910007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.527+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f910c09b790 con 0x7f911415c6f0 2024-09-21T05:46:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.527+0000 7f9118c59640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9100078b90 0x7f910007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:25.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.528+0000 7f9118c59640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9100078b90 0x7f910007b0a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9104009b30 tx=0x7f9104005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:25.532 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.530+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f910c063560 con 0x7f911415c6f0 2024-09-21T05:46:25.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.730+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91140b3b30 con 0x7f911415c6f0 2024-09-21T05:46:25.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.731+0000 7f91117fa640 1 -- 172.21.15.22:0/3924917489 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7f910c067f90 con 0x7f911415c6f0 2024-09-21T05:46:25.733 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9100078b90 msgr2=0x7f910007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9100078b90 0x7f910007b0a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9104009b30 tx=0x7f9104005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 msgr2=0x7f91141f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91141f3620 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f910c002230 tx=0x7f910c00cb90 comp rx=0 tx=0).stop 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 shutdown_connections 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f911415c6f0 0x7f91141f3620 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91140d1d40 0x7f91141f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9100078b90 0x7f910007b0a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 --2- 172.21.15.22:0/3924917489 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91140ccba0 0x7f91141f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:25.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 >> 172.21.15.22:0/3924917489 conn(0x7f91140b3f10 msgr2=0x7f91140b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:25.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 shutdown_connections 2024-09-21T05:46:25.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.733+0000 7f911aebd640 1 -- 172.21.15.22:0/3924917489 wait complete. 2024-09-21T05:46:25.746 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:46:25.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:25.747 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:25.780 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:46:25.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:25.780 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:46:25.836 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a19350640 1 -- 172.21.15.22:0/3597033706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a00004800 msgr2=0x7f1a00004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a19350640 1 --2- 172.21.15.22:0/3597033706 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a00004800 0x7f1a00004c10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f19fc009b50 tx=0x7f1a00005830 comp rx=0 tx=0).stop 2024-09-21T05:46:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a137fe640 1 -- 172.21.15.22:0/3597033706 reap_dead start 2024-09-21T05:46:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a19350640 1 -- 172.21.15.22:0/3597033706 shutdown_connections 2024-09-21T05:46:25.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a19350640 1 -- 172.21.15.22:0/3597033706 >> 172.21.15.22:0/3597033706 conn(0x7f1a14072ae0 msgr2=0x7f1a14072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:25.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a19350640 1 -- 172.21.15.22:0/3597033706 shutdown_connections 2024-09-21T05:46:25.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.991+0000 7f1a19350640 1 -- 172.21.15.22:0/3597033706 wait complete. 2024-09-21T05:46:25.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.993+0000 7f1a19350640 1 Processor -- start 2024-09-21T05:46:25.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.993+0000 7f1a19350640 1 -- start start 2024-09-21T05:46:26.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a19350640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a00004800 0x7f1a140b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a19350640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a1414b630 0x7f1a1414ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a19350640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 0x7f1a140cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a19350640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1a140ced70 con 0x7f1a1414bf80 2024-09-21T05:46:26.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a19350640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1a140cebf0 con 0x7f1a1414b630 2024-09-21T05:46:26.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a19350640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1a140ceef0 con 0x7f1a00004800 2024-09-21T05:46:26.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 0x7f1a140cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 0x7f1a140cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46486/0 (socket says 172.21.15.22:46486) 2024-09-21T05:46:26.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a137fe640 1 -- 172.21.15.22:0/646952131 learned_addr learned my addr 172.21.15.22:0/646952131 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:26.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.994+0000 7f1a137fe640 1 -- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a00004800 msgr2=0x7f1a140b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a12ffd640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a00004800 0x7f1a140b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a127fc640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a1414b630 0x7f1a1414ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a137fe640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a00004800 0x7f1a140b1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a137fe640 1 -- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a1414b630 msgr2=0x7f1a1414ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a137fe640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a1414b630 0x7f1a1414ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a137fe640 1 -- 172.21.15.22:0/646952131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a140c7b80 con 0x7f1a1414bf80 2024-09-21T05:46:26.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a137fe640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 0x7f1a140cb150 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1a140cd940 tx=0x7f1a0c011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c01f070 con 0x7f1a1414bf80 2024-09-21T05:46:26.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a0c00fb70 con 0x7f1a1414bf80 2024-09-21T05:46:26.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c012450 con 0x7f1a1414bf80 2024-09-21T05:46:26.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a140c7e60 con 0x7f1a1414bf80 2024-09-21T05:46:26.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.995+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a140c8420 con 0x7f1a1414bf80 2024-09-21T05:46:26.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.996+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a14151ad0 con 0x7f1a1414bf80 2024-09-21T05:46:26.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.997+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1a0c022070 con 0x7f1a1414bf80 2024-09-21T05:46:26.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.997+0000 7f1a0b7fe640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19f4078b90 0x7f19f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.997+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f1a0c09b740 con 0x7f1a1414bf80 2024-09-21T05:46:26.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.998+0000 7f1a12ffd640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19f4078b90 0x7f19f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:25.998+0000 7f1a12ffd640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19f4078b90 0x7f19f407b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1a04004ba0 tx=0x7f1a040018c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.002+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1a0c063ae0 con 0x7f1a1414bf80 2024-09-21T05:46:26.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.232+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a140d0d80 con 0x7f1a1414bf80 2024-09-21T05:46:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.233+0000 7f1a0b7fe640 1 -- 172.21.15.22:0/646952131 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5231 (secure 0 0 0) 0x7f1a0c0633e0 con 0x7f1a1414bf80 2024-09-21T05:46:26.235 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:26.235 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"btime":"2024-09-21T05:46:21:204567+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:46:20.588210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7482":{"gid":7482,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1062944206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1062944206},{"type":"v1","addr":"172.21.15.22:6834","nonce":1062944206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7482,"qdb_cluster":[7482]},"id":7}]} 2024-09-21T05:46:26.235 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 84 2024-09-21T05:46:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19f4078b90 msgr2=0x7f19f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19f4078b90 0x7f19f407b0a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1a04004ba0 tx=0x7f1a040018c0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 msgr2=0x7f1a140cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 0x7f1a140cb150 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1a140cd940 tx=0x7f1a0c011b10 comp rx=0 tx=0).stop 2024-09-21T05:46:26.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 shutdown_connections 2024-09-21T05:46:26.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1a1414bf80 0x7f1a140cb150 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1a1414b630 0x7f1a1414ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f19f4078b90 0x7f19f407b0a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 --2- 172.21.15.22:0/646952131 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1a00004800 0x7f1a140b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 >> 172.21.15.22:0/646952131 conn(0x7f1a14072ae0 msgr2=0x7f1a140b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:26.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 shutdown_connections 2024-09-21T05:46:26.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.235+0000 7f1a19350640 1 -- 172.21.15.22:0/646952131 wait complete. 2024-09-21T05:46:26.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T05:46:26.389 INFO:teuthology.orchestra.run.smithi022.stdout:{"message":"","return_code":0} 2024-09-21T05:46:26.389 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:46:26.390 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:26.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 -- 172.21.15.22:0/740783450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc0d2630 msgr2=0x7faecc15d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 --2- 172.21.15.22:0/740783450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc0d2630 0x7faecc15d130 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7faec0009830 tx=0x7faec002a510 comp rx=0 tx=0).stop 2024-09-21T05:46:26.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 -- 172.21.15.22:0/740783450 shutdown_connections 2024-09-21T05:46:26.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 --2- 172.21.15.22:0/740783450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc0d2630 0x7faecc15d130 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 -- 172.21.15.22:0/740783450 >> 172.21.15.22:0/740783450 conn(0x7faecc06d550 msgr2=0x7faecc071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:26.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 -- 172.21.15.22:0/740783450 shutdown_connections 2024-09-21T05:46:26.507 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.504+0000 7faed35e6640 1 -- 172.21.15.22:0/740783450 wait complete. 2024-09-21T05:46:26.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.506+0000 7faed35e6640 1 Processor -- start 2024-09-21T05:46:26.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.506+0000 7faed35e6640 1 -- start start 2024-09-21T05:46:26.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.507+0000 7faed35e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faecc1570e0 0x7faecc1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.507+0000 7faed35e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 0x7faecc155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.507+0000 7faed35e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faecc0cabd0 0x7faecc0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.507+0000 7faed35e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faecc0b7f30 con 0x7faecc15d670 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.507+0000 7faed35e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faecc0b7db0 con 0x7faecc1570e0 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.507+0000 7faed35e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faecc0b80b0 con 0x7faecc0cabd0 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 0x7faecc155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 0x7faecc155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46500/0 (socket says 172.21.15.22:46500) 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 -- 172.21.15.22:0/3941757300 learned_addr learned my addr 172.21.15.22:0/3941757300 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed1382640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faecc1570e0 0x7faecc1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed1b83640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faecc0cabd0 0x7faecc0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 -- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faecc0cabd0 msgr2=0x7faecc0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faecc0cabd0 0x7faecc0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 -- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faecc1570e0 msgr2=0x7faecc1f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faecc1570e0 0x7faecc1f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 -- 172.21.15.22:0/3941757300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc0cb520 con 0x7faecc15d670 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed0b81640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 0x7faecc155720 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7faec0009950 tx=0x7faec0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0031d60 con 0x7faecc15d670 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec00330d0 con 0x7faecc15d670 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1f5230 con 0x7faecc15d670 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.508+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec0033520 con 0x7faecc15d670 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.509+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1f4c70 con 0x7faecc15d670 2024-09-21T05:46:26.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.510+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc1f4700 con 0x7faecc15d670 2024-09-21T05:46:26.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.510+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faec00035f0 con 0x7faecc15d670 2024-09-21T05:46:26.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.511+0000 7faec9ffb640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faeb8078b90 0x7faeb807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.511+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7faec00bc340 con 0x7faecc15d670 2024-09-21T05:46:26.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.511+0000 7faed1382640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faeb8078b90 0x7faeb807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.512+0000 7faed1382640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faeb8078b90 0x7faeb807b0a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faebc009f20 tx=0x7faebc0094d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.516+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faecc0cb520 con 0x7faecc15d670 2024-09-21T05:46:26.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.739+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faecc15d130 con 0x7faecc15d670 2024-09-21T05:46:26.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.740+0000 7faec9ffb640 1 -- 172.21.15.22:0/3941757300 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5231 (secure 0 0 0) 0x7faec003f030 con 0x7faecc15d670 2024-09-21T05:46:26.742 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:26.743 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"btime":"2024-09-21T05:46:21:204567+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:46:20.588210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7482":{"gid":7482,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1062944206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1062944206},{"type":"v1","addr":"172.21.15.22:6834","nonce":1062944206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7482,"qdb_cluster":[7482]},"id":7}]} 2024-09-21T05:46:26.743 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 84 2024-09-21T05:46:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faeb8078b90 msgr2=0x7faeb807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faeb8078b90 0x7faeb807b0a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7faebc009f20 tx=0x7faebc0094d0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 msgr2=0x7faecc155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 0x7faecc155720 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7faec0009950 tx=0x7faec0033d60 comp rx=0 tx=0).stop 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 shutdown_connections 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faeb8078b90 0x7faeb807b0a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faecc0cabd0 0x7faecc0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faecc15d670 0x7faecc155720 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 --2- 172.21.15.22:0/3941757300 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faecc1570e0 0x7faecc1f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 >> 172.21.15.22:0/3941757300 conn(0x7faecc06d550 msgr2=0x7faecc0b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:26.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 shutdown_connections 2024-09-21T05:46:26.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.742+0000 7faed35e6640 1 -- 172.21.15.22:0/3941757300 wait complete. 2024-09-21T05:46:26.756 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-09-21T05:46:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.810+0000 7fe36fd00f40 1 Processor -- start 2024-09-21T05:46:26.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.810+0000 7fe36fd00f40 1 -- start start 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe36fd00f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583be27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe36fd00f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c71710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe36fd00f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583c71c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe36fd00f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557583c743f0 con 0x557583c6ffb0 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe36fd00f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557583aead60 con 0x557583c73530 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe36fd00f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557583c6c790 con 0x557583b993d0 2024-09-21T05:46:26.815 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe368fb9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c71710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe368fb9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c71710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46514/0 (socket says 172.21.15.22:46514) 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.811+0000 7fe368fb9640 1 -- 172.21.15.22:0/1477590519 learned_addr learned my addr 172.21.15.22:0/1477590519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe369fbb640 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583c71c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe3697ba640 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583be27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe368fb9640 1 -- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 msgr2=0x557583be27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe368fb9640 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583be27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe368fb9640 1 -- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 msgr2=0x557583c71c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe368fb9640 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583c71c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe368fb9640 1 -- 172.21.15.22:0/1477590519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557583c5b100 con 0x557583c6ffb0 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe368fb9640 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c71710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b470 tx=0x7fe36000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a616dc3bb550d341 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe35b7fe640 1 -- 172.21.15.22:0/1477590519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360009950 con 0x557583c6ffb0 2024-09-21T05:46:26.816 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe35b7fe640 1 -- 172.21.15.22:0/1477590519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe360009af0 con 0x557583c6ffb0 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.812+0000 7fe35b7fe640 1 -- 172.21.15.22:0/1477590519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360014440 con 0x557583c6ffb0 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 -- 172.21.15.22:0/1477590519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 msgr2=0x557583c71710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c71710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b470 tx=0x7fe36000b930 comp rx=0 tx=0).stop 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 -- 172.21.15.22:0/1477590519 shutdown_connections 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583c71c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c71710 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/1477590519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583be27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 -- 172.21.15.22:0/1477590519 >> 172.21.15.22:0/1477590519 conn(0x557583abbac0 msgr2=0x557583bc9830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 -- 172.21.15.22:0/1477590519 shutdown_connections 2024-09-21T05:46:26.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.813+0000 7fe36fd00f40 1 -- 172.21.15.22:0/1477590519 wait complete. 2024-09-21T05:46:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.814+0000 7fe36fd00f40 1 Processor -- start 2024-09-21T05:46:26.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.814+0000 7fe36fd00f40 1 -- start start 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe36fd00f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583c72160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe36fd00f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c5a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe36fd00f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583cfe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe36fd00f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557583c5b4a0 con 0x557583c6ffb0 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe36fd00f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557583bc9f40 con 0x557583c73530 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe36fd00f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557583bca1e0 con 0x557583b993d0 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c5a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c5a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46528/0 (socket says 172.21.15.22:46528) 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 -- 172.21.15.22:0/3442087424 learned_addr learned my addr 172.21.15.22:0/3442087424 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe3697ba640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583c72160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe369fbb640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583cfe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 -- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 msgr2=0x557583c72160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557583b993d0 0x557583c72160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 -- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 msgr2=0x557583cfe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583cfe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe368fb9640 1 -- 172.21.15.22:0/3442087424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557583d000d0 con 0x557583c6ffb0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.815+0000 7fe369fbb640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557583c73530 0x557583cfe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.816+0000 7fe368fb9640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c5a1b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe36000be30 tx=0x7fe3600146f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.816+0000 7fe35a7fc640 1 -- 172.21.15.22:0/3442087424 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe360022070 con 0x557583c6ffb0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.816+0000 7fe35a7fc640 1 -- 172.21.15.22:0/3442087424 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe360014b20 con 0x557583c6ffb0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.816+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557583d003b0 con 0x557583c6ffb0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.816+0000 7fe35a7fc640 1 -- 172.21.15.22:0/3442087424 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36001d480 con 0x557583c6ffb0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.816+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557583cfb7f0 con 0x557583c6ffb0 2024-09-21T05:46:26.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.817+0000 7fe35a7fc640 1 -- 172.21.15.22:0/3442087424 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe360025070 con 0x557583c6ffb0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.818+0000 7fe35a7fc640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe34c0775a0 0x7fe34c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.818+0000 7fe35a7fc640 1 -- 172.21.15.22:0/3442087424 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe36009ae30 con 0x557583c6ffb0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.818+0000 7fe3697ba640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe34c0775a0 0x7fe34c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.818+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x557583cf98e0 0x557583cf9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.818+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e97) -- 0x557583cfa230 con 0x557583cf98e0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.818+0000 7fe369fbb640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x557583cf98e0 0x557583cf9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe3697ba640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe34c0775a0 0x7fe34c079ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b550 tx=0x7fe35c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe369fbb640 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x557583cf98e0 0x557583cf9cf0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe369fbb640 1 -- 172.21.15.22:0/3442087424 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fe36400b040 con 0x557583cf98e0 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x557583cf98e0 msgr2=0x557583cf9cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x557583cf98e0 0x557583cf9cf0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe34c0775a0 msgr2=0x7fe34c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe34c0775a0 0x7fe34c079ab0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00b550 tx=0x7fe35c0046d0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 msgr2=0x557583c5a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 --2- 172.21.15.22:0/3442087424 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557583c6ffb0 0x557583c5a1b0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe36000be30 tx=0x7fe3600146f0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe369fbb640 1 -- 172.21.15.22:0/3442087424 reap_dead start 2024-09-21T05:46:26.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 shutdown_connections 2024-09-21T05:46:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 >> 172.21.15.22:0/3442087424 conn(0x557583abbac0 msgr2=0x557583b9b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 shutdown_connections 2024-09-21T05:46:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.819+0000 7fe36fd00f40 1 -- 172.21.15.22:0/3442087424 wait complete. 2024-09-21T05:46:26.826 INFO:teuthology.orchestra.run.smithi022.stdout:����� 2024-09-21T05:46:26.826 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:46:26.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f49a4f21640 1 -- 172.21.15.22:0/3914278193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49880041c0 msgr2=0x7f4988004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f49a4f21640 1 --2- 172.21.15.22:0/3914278193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49880041c0 0x7f4988004800 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f49a0070050 tx=0x7f4988005830 comp rx=0 tx=0).stop 2024-09-21T05:46:26.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f499ed9d640 1 -- 172.21.15.22:0/3914278193 reap_dead start 2024-09-21T05:46:26.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f49a4f21640 1 -- 172.21.15.22:0/3914278193 shutdown_connections 2024-09-21T05:46:26.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f49a4f21640 1 -- 172.21.15.22:0/3914278193 >> 172.21.15.22:0/3914278193 conn(0x7f49a009d7f0 msgr2=0x7f49a00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:26.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f49a4f21640 1 -- 172.21.15.22:0/3914278193 shutdown_connections 2024-09-21T05:46:26.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.941+0000 7f49a4f21640 1 -- 172.21.15.22:0/3914278193 wait complete. 2024-09-21T05:46:26.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.943+0000 7f49a4f21640 1 Processor -- start 2024-09-21T05:46:26.945 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.943+0000 7f49a4f21640 1 -- start start 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f49a4f21640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49880041c0 0x7f49a01f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f49a4f21640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a015e930 0x7f49a015b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f49a4f21640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 0x7f49a00c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f49a4f21640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f49a006f340 con 0x7f49a00c8050 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f49a4f21640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f49a006f1c0 con 0x7f49a015e930 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f49a4f21640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f49a006f4c0 con 0x7f49880041c0 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f499ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 0x7f49a00c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f499ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 0x7f49a00c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46556/0 (socket says 172.21.15.22:46556) 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.944+0000 7f499ed9d640 1 -- 172.21.15.22:0/248020605 learned_addr learned my addr 172.21.15.22:0/248020605 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:26.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499dd9b640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a015e930 0x7f49a015b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499e59c640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49880041c0 0x7f49a01f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499ed9d640 1 -- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49880041c0 msgr2=0x7f49a01f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499ed9d640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49880041c0 0x7f49a01f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499ed9d640 1 -- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a015e930 msgr2=0x7f49a015b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499ed9d640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a015e930 0x7f49a015b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499ed9d640 1 -- 172.21.15.22:0/248020605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a00cabd0 con 0x7f49a00c8050 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f499ed9d640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 0x7f49a00c8460 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f49a00cb530 tx=0x7f499800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998004b20 con 0x7f49a00c8050 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4998004cc0 con 0x7f49a00c8050 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a015de90 con 0x7f49a00c8050 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4998003bd0 con 0x7f49a00c8050 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.945+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a015e130 con 0x7f49a00c8050 2024-09-21T05:46:26.949 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.946+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a00ca660 con 0x7f49a00c8050 2024-09-21T05:46:26.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.947+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4998004030 con 0x7f49a00c8050 2024-09-21T05:46:26.950 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.947+0000 7f498effd640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4980078bc0 0x7f498007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:26.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.948+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f499809b4c0 con 0x7f49a00c8050 2024-09-21T05:46:26.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.948+0000 7f499e59c640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4980078bc0 0x7f498007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:26.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.948+0000 7f499e59c640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4980078bc0 0x7f498007b0d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4990001d70 tx=0x7f4990001ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:26.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:26.952+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4998063990 con 0x7f49a00c8050 2024-09-21T05:46:27.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.178+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f49a006fa20 con 0x7f49a00c8050 2024-09-21T05:46:27.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.178+0000 7f498effd640 1 -- 172.21.15.22:0/248020605 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f4998063290 con 0x7f49a00c8050 2024-09-21T05:46:27.181 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:46:27.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.180+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4980078bc0 msgr2=0x7f498007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:27.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.180+0000 7f49a4f21640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4980078bc0 0x7f498007b0d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4990001d70 tx=0x7f4990001ce0 comp rx=0 tx=0).stop 2024-09-21T05:46:27.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.180+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 msgr2=0x7f49a00c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:27.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.180+0000 7f49a4f21640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 0x7f49a00c8460 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f49a00cb530 tx=0x7f499800f3c0 comp rx=0 tx=0).stop 2024-09-21T05:46:27.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 shutdown_connections 2024-09-21T05:46:27.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f49a00c8050 0x7f49a00c8460 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:27.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f49a015e930 0x7f49a015b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:27.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4980078bc0 0x7f498007b0d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:27.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 --2- 172.21.15.22:0/248020605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f49880041c0 0x7f49a01f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:27.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 >> 172.21.15.22:0/248020605 conn(0x7f49a009d7f0 msgr2=0x7f49a00b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:27.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 shutdown_connections 2024-09-21T05:46:27.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27.181+0000 7f49a4f21640 1 -- 172.21.15.22:0/248020605 wait complete. 2024-09-21T05:46:27.194 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:46:27.194 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:46:27.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:46:27.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.196 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:46:27.212 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:27.213 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42357sec preferred_lft 42357sec 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.214 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:46:27.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.215 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:46:27.277 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:46:27.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.279 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:46:27.336 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:46:27.336 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:46:27.336 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:46:27.337 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:46:27.337 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:46:27.337 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:46:27.337 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:46:27.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:46:27.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:46:27.337 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:46:27.337 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:46:27.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:46:27.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:27.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.466 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:46:27.481 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:46:27.481 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:46:27.481 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:46:27.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:46:27.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:46:27.548 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:27.604 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:46:27.604 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:46:27.604 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:27.661 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:46:27.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.661 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:27.757 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:27.757 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:46:27.757 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:46:27.768 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:27.768 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:46:27.768 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:46:27.768 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:46:27.768 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:46:27.774 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:46:27.775 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:46:27.775 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:46:27.775 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:46:27.775 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:46:27.775 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:46:27.776 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:46:27.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.776 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:27.801 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:46:27.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:46:27.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:27.899 INFO:teuthology.orchestra.run.smithi022.stdout:client6500 2024-09-21T05:46:27.899 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:27.899 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:27.915 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:27.915 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:27.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:27.972 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-21T05:46:28.062 INFO:teuthology.orchestra.run.smithi022.stderr:6+0 records in 2024-09-21T05:46:28.062 INFO:teuthology.orchestra.run.smithi022.stderr:6+0 records out 2024-09-21T05:46:28.062 INFO:teuthology.orchestra.run.smithi022.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.031688 s, 199 MB/s 2024-09-21T05:46:28.063 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:28.063 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:28.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:28.079 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-21T05:46:28.376 INFO:teuthology.orchestra.run.smithi022.stderr:6+0 records in 2024-09-21T05:46:28.376 INFO:teuthology.orchestra.run.smithi022.stderr:6+0 records out 2024-09-21T05:46:28.376 INFO:teuthology.orchestra.run.smithi022.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.239494 s, 26.3 MB/s 2024-09-21T05:46:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:28.377 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-21T05:46:28.394 INFO:teuthology.orchestra.run.smithi022.stdout:./ 2024-09-21T05:46:28.394 INFO:teuthology.orchestra.run.smithi022.stdout:./file2_sixmegs 2024-09-21T05:46:28.394 INFO:teuthology.orchestra.run.smithi022.stdout:./file3_sixmegs 2024-09-21T05:46:28.395 INFO:teuthology.orchestra.run.smithi022.stdout:./file1_sixmegs 2024-09-21T05:46:28.395 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:46:28.395 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:46:28.395 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:46:28.395 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:46:28.395 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-21T05:46:28.395 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:46:28.502 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:46:28.505 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:28.511 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:46:28.511 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:46:28.512 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:46:28.512 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:46:28.512 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-09-21T05:46:28.512 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:46:28.619 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:46:28.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:28.629 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:46:28.629 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:46:28.629 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:46:28.629 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:46:28.629 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-09-21T05:46:28.629 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:46:28.735 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627778 2024-09-21T05:46:28.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:28.744 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:46:28.744 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:46:28.744 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:46:28.744 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:46:28.744 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-09-21T05:46:28.744 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:46:28.851 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:46:28.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:46:28.860 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:28.860 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:28.917 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:28.917 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:28.973 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:46:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb617a8640 1 -- 172.21.15.22:0/1127986850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 msgr2=0x7fbb48003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb617a8640 1 --2- 172.21.15.22:0/1127986850 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb48003e70 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c290 tx=0x7fbb48005830 comp rx=0 tx=0).stop 2024-09-21T05:46:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb5b7fe640 1 -- 172.21.15.22:0/1127986850 reap_dead start 2024-09-21T05:46:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb617a8640 1 -- 172.21.15.22:0/1127986850 shutdown_connections 2024-09-21T05:46:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb617a8640 1 -- 172.21.15.22:0/1127986850 >> 172.21.15.22:0/1127986850 conn(0x7fbb5c0b2e10 msgr2=0x7fbb5c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb617a8640 1 -- 172.21.15.22:0/1127986850 shutdown_connections 2024-09-21T05:46:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.129+0000 7fbb617a8640 1 -- 172.21.15.22:0/1127986850 wait complete. 2024-09-21T05:46:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.131+0000 7fbb617a8640 1 Processor -- start 2024-09-21T05:46:29.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.131+0000 7fbb617a8640 1 -- start start 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb617a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb5c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb617a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5c1f94b0 0x7fbb5c1368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb617a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb5c136e20 0x7fbb5c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb617a8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0cdfa0 con 0x7fbb48003a00 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb617a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0cde20 con 0x7fbb5c1f94b0 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb617a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0ce120 con 0x7fbb5c136e20 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.132+0000 7fbb5affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb5c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb5c0b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46582/0 (socket says 172.21.15.22:46582) 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 -- 172.21.15.22:0/962038337 learned_addr learned my addr 172.21.15.22:0/962038337 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb53fff640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5c1f94b0 0x7fbb5c1368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5b7fe640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb5c136e20 0x7fbb5c13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 -- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb5c136e20 msgr2=0x7fbb5c13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb5c136e20 0x7fbb5c13bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 -- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5c1f94b0 msgr2=0x7fbb5c1368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5c1f94b0 0x7fbb5c1368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 -- 172.21.15.22:0/962038337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c0b4660 con 0x7fbb48003a00 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb5affd640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb5c0b5700 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c02b110 tx=0x7fbb4c001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c031040 con 0x7fbb48003a00 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb4c003800 con 0x7fbb48003a00 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c0b4940 con 0x7fbb48003a00 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c006600 con 0x7fbb48003a00 2024-09-21T05:46:29.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.133+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c0b4f00 con 0x7fbb48003a00 2024-09-21T05:46:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.134+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c06a6c0 con 0x7fbb48003a00 2024-09-21T05:46:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.135+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb4c03e030 con 0x7fbb48003a00 2024-09-21T05:46:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.135+0000 7fbb537fe640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbb3c078bc0 0x7fbb3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.136+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fbb4c0baa30 con 0x7fbb48003a00 2024-09-21T05:46:29.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.136+0000 7fbb53fff640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbb3c078bc0 0x7fbb3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.136+0000 7fbb53fff640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbb3c078bc0 0x7fbb3c07b0d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0c98b0 tx=0x7fbb44009450 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:29.142 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.140+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbb5c06a6c0 con 0x7fbb48003a00 2024-09-21T05:46:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.342+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbb5c0d02b0 con 0x7fbb48003a00 2024-09-21T05:46:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.342+0000 7fbb537fe640 1 -- 172.21.15.22:0/962038337 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v97) ==== 80+0+0 (secure 0 0 0) 0x7fbb4c082890 con 0x7fbb48003a00 2024-09-21T05:46:29.344 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbb3c078bc0 msgr2=0x7fbb3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbb3c078bc0 0x7fbb3c07b0d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0c98b0 tx=0x7fbb44009450 comp rx=0 tx=0).stop 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 msgr2=0x7fbb5c0b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb5c0b5700 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c02b110 tx=0x7fbb4c001d40 comp rx=0 tx=0).stop 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 shutdown_connections 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbb3c078bc0 0x7fbb3c07b0d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbb5c136e20 0x7fbb5c13bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbb5c1f94b0 0x7fbb5c1368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 --2- 172.21.15.22:0/962038337 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbb48003a00 0x7fbb5c0b5700 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 >> 172.21.15.22:0/962038337 conn(0x7fbb5c0b2e10 msgr2=0x7fbb5c1560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:29.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 shutdown_connections 2024-09-21T05:46:29.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.345+0000 7fbb617a8640 1 -- 172.21.15.22:0/962038337 wait complete. 2024-09-21T05:46:29.357 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:46:29.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:29.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:46:29.414 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:46:29.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:29.414 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:46:29.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:46:29.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032f3ba640 1 -- 172.21.15.22:0/358468630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 msgr2=0x7f031400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.604 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032f3ba640 1 --2- 172.21.15.22:0/358468630 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f031400b9c0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f032000c610 tx=0x7f0314005830 comp rx=0 tx=0).stop 2024-09-21T05:46:29.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032d957640 1 -- 172.21.15.22:0/358468630 reap_dead start 2024-09-21T05:46:29.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032f3ba640 1 -- 172.21.15.22:0/358468630 shutdown_connections 2024-09-21T05:46:29.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032f3ba640 1 -- 172.21.15.22:0/358468630 >> 172.21.15.22:0/358468630 conn(0x7f032809d7f0 msgr2=0x7f03280d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:29.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032f3ba640 1 -- 172.21.15.22:0/358468630 shutdown_connections 2024-09-21T05:46:29.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.602+0000 7f032f3ba640 1 -- 172.21.15.22:0/358468630 wait complete. 2024-09-21T05:46:29.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.604+0000 7f032f3ba640 1 Processor -- start 2024-09-21T05:46:29.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.604+0000 7f032f3ba640 1 -- start start 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032f3ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f03280cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032f3ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f032814d0c0 0x7f03281f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032f3ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03280b8ab0 0x7f03280b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032f3ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f03280d5010 con 0x7f03140051c0 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032f3ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f03280b68c0 con 0x7f03280b8ab0 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032f3ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f03280b3710 con 0x7f032814d0c0 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032d156640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f03280cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032d156640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f03280cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46606/0 (socket says 172.21.15.22:46606) 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.605+0000 7f032d156640 1 -- 172.21.15.22:0/808211221 learned_addr learned my addr 172.21.15.22:0/808211221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032c955640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f032814d0c0 0x7f03281f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d957640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03280b8ab0 0x7f03280b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d156640 1 -- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f032814d0c0 msgr2=0x7f03281f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d156640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f032814d0c0 0x7f03281f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d156640 1 -- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03280b8ab0 msgr2=0x7f03280b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d156640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03280b8ab0 0x7f03280b8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d156640 1 -- 172.21.15.22:0/808211221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280b9400 con 0x7f03140051c0 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032c955640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f032814d0c0 0x7f03281f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032d156640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f03280cc640 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0324003ec0 tx=0x7f03240017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324015070 con 0x7f03140051c0 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0324011920 con 0x7f03140051c0 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324011d70 con 0x7f03140051c0 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03280cce90 con 0x7f03140051c0 2024-09-21T05:46:29.610 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.606+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03280cd450 con 0x7f03140051c0 2024-09-21T05:46:29.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.607+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03280b3b80 con 0x7f03140051c0 2024-09-21T05:46:29.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.608+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0324018070 con 0x7f03140051c0 2024-09-21T05:46:29.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.608+0000 7f031dffb640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f030c078b90 0x7f030c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:29.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.608+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f032409bb20 con 0x7f03140051c0 2024-09-21T05:46:29.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.609+0000 7f032c955640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f030c078b90 0x7f030c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:29.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.609+0000 7f032c955640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f030c078b90 0x7f030c07b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0328159cb0 tx=0x7f0318005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:29.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.613+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03280b9400 con 0x7f03140051c0 2024-09-21T05:46:29.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.839+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03280d4240 con 0x7f03140051c0 2024-09-21T05:46:29.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.839+0000 7f031dffb640 1 -- 172.21.15.22:0/808211221 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5231 (secure 0 0 0) 0x7f0324063f50 con 0x7f03140051c0 2024-09-21T05:46:29.842 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:29.842 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":84,"btime":"2024-09-21T05:46:21:204567+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:46:20.588210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7482":{"gid":7482,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.22:6834/1062944206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":1062944206},{"type":"v1","addr":"172.21.15.22:6834","nonce":1062944206}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7482,"qdb_cluster":[7482]},"id":7}]} 2024-09-21T05:46:29.843 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 84 2024-09-21T05:46:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f030c078b90 msgr2=0x7f030c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f030c078b90 0x7f030c07b0a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0328159cb0 tx=0x7f0318005f90 comp rx=0 tx=0).stop 2024-09-21T05:46:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 msgr2=0x7f03280cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f03280cc640 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0324003ec0 tx=0x7f03240017f0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 shutdown_connections 2024-09-21T05:46:29.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f03280b8ab0 0x7f03280b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f030c078b90 0x7f030c07b0a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f032814d0c0 0x7f03281f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 --2- 172.21.15.22:0/808211221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f03140051c0 0x7f03280cc640 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 >> 172.21.15.22:0/808211221 conn(0x7f032809d7f0 msgr2=0x7f032815e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 shutdown_connections 2024-09-21T05:46:29.846 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:29.842+0000 7f032f3ba640 1 -- 172.21.15.22:0/808211221 wait complete. 2024-09-21T05:46:29.855 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-21T05:46:29.990 INFO:teuthology.orchestra.run.smithi022.stdout:{"message":"","return_code":0} 2024-09-21T05:46:29.991 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-21T05:46:29.991 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:46:29.991 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:46:30.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:46:30.007 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:46:30.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:30.007 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:46:30.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:46:30.062 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T05:46:30.063 DEBUG:teuthology.orchestra.run.smithi022:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-21T05:46:30.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:30.105 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 -- 172.21.15.22:0/2668835774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0844157110 msgr2=0x7f0844160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 --2- 172.21.15.22:0/2668835774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0844157110 0x7f0844160e60 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0834009830 tx=0x7f083402a510 comp rx=0 tx=0).stop 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 -- 172.21.15.22:0/2668835774 shutdown_connections 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 --2- 172.21.15.22:0/2668835774 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0844157110 0x7f0844160e60 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 -- 172.21.15.22:0/2668835774 >> 172.21.15.22:0/2668835774 conn(0x7f084406d550 msgr2=0x7f0844071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 -- 172.21.15.22:0/2668835774 shutdown_connections 2024-09-21T05:46:30.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.225+0000 7f0849a3f640 1 -- 172.21.15.22:0/2668835774 wait complete. 2024-09-21T05:46:30.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.227+0000 7f0849a3f640 1 Processor -- start 2024-09-21T05:46:30.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.227+0000 7f0849a3f640 1 -- start start 2024-09-21T05:46:30.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0849a3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 0x7f084415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0849a3f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08440cece0 0x7f08441f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0849a3f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0844157110 0x7f08441f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0849a3f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f08440d0a60 con 0x7f08440cd390 2024-09-21T05:46:30.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0849a3f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08440d08e0 con 0x7f0844157110 2024-09-21T05:46:30.232 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0849a3f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f08440d0be0 con 0x7f08440cece0 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 0x7f084415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 0x7f084415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46620/0 (socket says 172.21.15.22:46620) 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 -- 172.21.15.22:0/1203253969 learned_addr learned my addr 172.21.15.22:0/1203253969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f08437fe640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0844157110 0x7f08441f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f08427fc640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08440cece0 0x7f08441f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 -- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08440cece0 msgr2=0x7f08441f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08440cece0 0x7f08441f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 -- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0844157110 msgr2=0x7f08441f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0844157110 0x7f08441f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f0842ffd640 1 -- 172.21.15.22:0/1203253969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084406fd70 con 0x7f08440cd390 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.229+0000 7f08437fe640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0844157110 0x7f08441f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.230+0000 7f0842ffd640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 0x7f084415bb80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0834009950 tx=0x7f0834033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.230+0000 7f082bfff640 1 -- 172.21.15.22:0/1203253969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834031d60 con 0x7f08440cd390 2024-09-21T05:46:30.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.230+0000 7f082bfff640 1 -- 172.21.15.22:0/1203253969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08340330d0 con 0x7f08440cd390 2024-09-21T05:46:30.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.230+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084406e100 con 0x7f08440cd390 2024-09-21T05:46:30.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.230+0000 7f082bfff640 1 -- 172.21.15.22:0/1203253969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834033520 con 0x7f08440cd390 2024-09-21T05:46:30.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.230+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084406e6c0 con 0x7f08440cd390 2024-09-21T05:46:30.234 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.232+0000 7f082bfff640 1 -- 172.21.15.22:0/1203253969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08340035f0 con 0x7f08440cd390 2024-09-21T05:46:30.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.233+0000 7f082bfff640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0830078b90 0x7f083007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.233+0000 7f082bfff640 1 -- 172.21.15.22:0/1203253969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f08340bc2b0 con 0x7f08440cd390 2024-09-21T05:46:30.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.233+0000 7f08427fc640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0830078b90 0x7f083007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.233+0000 7f0850a8b740 1 Processor -- start 2024-09-21T05:46:30.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.234+0000 7f08427fc640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0830078b90 0x7f083007b0a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0844155170 tx=0x7f0838009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.234+0000 7f0850a8b740 1 -- start start 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa56a410 0x560ffa93cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa93d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 0x560ffa95a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0850a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ffa885120 con 0x560ffa549730 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ffa874550 con 0x560ffa541550 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x560ffa878360 con 0x560ffa56a410 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f08437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 0x560ffa95a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f08437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 0x560ffa95a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46632/0 (socket says 172.21.15.22:46632) 2024-09-21T05:46:30.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f08437fe640 1 -- 172.21.15.22:0/4139604371 learned_addr learned my addr 172.21.15.22:0/4139604371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f0842ffd640 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa56a410 0x560ffa93cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.235+0000 7f08427fc640 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa93d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08437fe640 1 -- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa56a410 msgr2=0x560ffa93cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08437fe640 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa56a410 0x560ffa93cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08437fe640 1 -- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 msgr2=0x560ffa93d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08437fe640 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa93d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08437fe640 1 -- 172.21.15.22:0/4139604371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ffa566470 con 0x560ffa549730 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f0842ffd640 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa56a410 0x560ffa93cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08437fe640 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 0x560ffa95a150 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f083c009fd0 tx=0x7f083c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5386be6a1864a4ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08297fa640 1 -- 172.21.15.22:0/4139604371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c003970 con 0x560ffa549730 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08297fa640 1 -- 172.21.15.22:0/4139604371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f083c003b10 con 0x560ffa549730 2024-09-21T05:46:30.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.236+0000 7f08297fa640 1 -- 172.21.15.22:0/4139604371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c004020 con 0x560ffa549730 2024-09-21T05:46:30.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 -- 172.21.15.22:0/4139604371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 msgr2=0x560ffa95a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 0x560ffa95a150 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f083c009fd0 tx=0x7f083c00c890 comp rx=0 tx=0).stop 2024-09-21T05:46:30.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 -- 172.21.15.22:0/4139604371 shutdown_connections 2024-09-21T05:46:30.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa549730 0x560ffa95a150 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa93d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 --2- 172.21.15.22:0/4139604371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa56a410 0x560ffa93cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 -- 172.21.15.22:0/4139604371 >> 172.21.15.22:0/4139604371 conn(0x560ffa55e560 msgr2=0x560ffa912fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 -- 172.21.15.22:0/4139604371 shutdown_connections 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.237+0000 7f0850a8b740 1 -- 172.21.15.22:0/4139604371 wait complete. 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.238+0000 7f0850a8b740 1 -- start start 2024-09-21T05:46:30.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.238+0000 7f0850a8b740 1 Processor -- start 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa969b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa957db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560ffa5e47c0 con 0x560ffa541550 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ffa819890 con 0x560ffa56a410 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x560ffa93d790 con 0x560ffa549730 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.239+0000 7f0850a8b740 10 client.0 _ll_register_callbacks cb 0x560ffa86dce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa969b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa969b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46638/0 (socket says 172.21.15.22:46638) 2024-09-21T05:46:30.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 -- 172.21.15.22:0/3097012874 learned_addr learned my addr 172.21.15.22:0/3097012874 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f08427fc640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f08437fe640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa957db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 -- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 msgr2=0x560ffa957870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 -- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 msgr2=0x560ffa957db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa957db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 -- 172.21.15.22:0/3097012874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ffa64a7e0 con 0x560ffa541550 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f0842ffd640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa969b60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0834009830 tx=0x7f0834083c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992343e74ff3800b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.240+0000 7f08227fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f08227fc640 1 -- 172.21.15.22:0/3097012874 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083409b060 con 0x560ffa541550 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f08227fc640 1 -- 172.21.15.22:0/3097012874 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f08340842d0 con 0x560ffa541550 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083c003b10 con 0x560ffa541550 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f08227fc640 1 -- 172.21.15.22:0/3097012874 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f08340bc0d0 con 0x560ffa541550 2024-09-21T05:46:30.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 10 client.7692 fetch_fsmap learned FSMap version 84 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x560ffa6aaf70 con 0x560ffa541550 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f08227fc640 1 -- 172.21.15.22:0/3097012874 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f083c003b10 con 0x560ffa541550 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f08227fc640 1 -- 172.21.15.22:0/3097012874 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f0834089ce0 con 0x560ffa541550 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 10 client.7692 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 10 client.7692 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 20 client.7692 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa5291f0 0x560ffa96a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 4 client.7692 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ffa887070 con 0x560ffa5291f0 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.241+0000 7f08427fc640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa5291f0 0x560ffa96a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.242+0000 7f08227fc640 10 client.7692 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.242+0000 7f08427fc640 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa5291f0 0x560ffa96a8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=311bdfabf5846164 server_cookie=e084201c82a6aa0 in_seq=0 out_seq=0 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f08227fc640 1 -- 172.21.15.22:0/3097012874 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ffa887070 con 0x560ffa5291f0 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f08227fc640 10 client.7692 handle_command_reply: tid=0 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 client.7692 shutdown 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 msgr2=0x560ffa969b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa969b60 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f0834009830 tx=0x7f0834083c40 comp rx=0 tx=0).stop 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 shutdown_connections 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa5291f0 0x560ffa96a8b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa957db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 --2- 172.21.15.22:0/3097012874 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa969b60 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 >> 172.21.15.22:0/3097012874 conn(0x560ffa55e560 msgr2=0x560ffa90daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f08227fc640 0 client.7692 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 shutdown_connections 2024-09-21T05:46:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 1 -- 172.21.15.22:0/3097012874 wait complete. 2024-09-21T05:46:30.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.243+0000 7f0850a8b740 20 client.7692 trim_cache size 0 max 16384 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 -- 172.21.15.22:0/1827268402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc0d1530 msgr2=0x7faabc0d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 --2- 172.21.15.22:0/1827268402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc0d1530 0x7faabc0d31a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7faab0009830 tx=0x7faab002a5c0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 -- 172.21.15.22:0/1827268402 shutdown_connections 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 --2- 172.21.15.22:0/1827268402 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc0d1530 0x7faabc0d31a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 -- 172.21.15.22:0/1827268402 >> 172.21.15.22:0/1827268402 conn(0x7faabc08bfc0 msgr2=0x7faabc09d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 -- 172.21.15.22:0/1827268402 shutdown_connections 2024-09-21T05:46:30.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.247+0000 7faac224d640 1 -- 172.21.15.22:0/1827268402 wait complete. 2024-09-21T05:46:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.249+0000 7faac224d640 1 Processor -- start 2024-09-21T05:46:30.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.249+0000 7faac224d640 1 -- start start 2024-09-21T05:46:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faac224d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faabc06e270 0x7faabc157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faac224d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faabc0d2d90 0x7faabc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faac224d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc158a00 0x7faabc15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faac224d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faabc06ed90 con 0x7faabc06e270 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faac224d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0d0740 con 0x7faabc158a00 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faac224d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faabc06ef10 con 0x7faabc0d2d90 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faabc06e270 0x7faabc157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faabc06e270 0x7faabc157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46648/0 (socket says 172.21.15.22:46648) 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabaffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faabc0d2d90 0x7faabc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 -- 172.21.15.22:0/3029411501 learned_addr learned my addr 172.21.15.22:0/3029411501 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabbfff640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc158a00 0x7faabc15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 -- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faabc0d2d90 msgr2=0x7faabc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faabc0d2d90 0x7faabc15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 -- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc158a00 msgr2=0x7faabc15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faabc158a00 0x7faabc15dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabb7fe640 1 -- 172.21.15.22:0/3029411501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc1f9850 con 0x7faabc06e270 2024-09-21T05:46:30.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.251+0000 7faabaffd640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faabc0d2d90 0x7faabc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:46:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.252+0000 7faabb7fe640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faabc06e270 0x7faabc157ce0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7faaac00c3a0 tx=0x7faaac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.252+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac017070 con 0x7faabc06e270 2024-09-21T05:46:30.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.252+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaac0053c0 con 0x7faabc06e270 2024-09-21T05:46:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.252+0000 7faac224d640 1 -- 172.21.15.22:0/3029411501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc1f9af0 con 0x7faabc06e270 2024-09-21T05:46:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.252+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac003850 con 0x7faabc06e270 2024-09-21T05:46:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.252+0000 7faac224d640 1 -- 172.21.15.22:0/3029411501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc0b5cc0 con 0x7faabc06e270 2024-09-21T05:46:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.253+0000 7faac224d640 1 -- 172.21.15.22:0/3029411501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7faabc0d31a0 con 0x7faabc06e270 2024-09-21T05:46:30.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.254+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faaac01a070 con 0x7faabc06e270 2024-09-21T05:46:30.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.255+0000 7faaa3fff640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faaa8078bc0 0x7faaa807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.255+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7faaac099fb0 con 0x7faabc06e270 2024-09-21T05:46:30.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.255+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7faaac09c550 con 0x7faabc06e270 2024-09-21T05:46:30.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.255+0000 7faabaffd640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faaa8078bc0 0x7faaa807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.255+0000 7faac224d640 1 -- 172.21.15.22:0/3029411501 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7faabc0b2310 con 0x7faabc06e270 2024-09-21T05:46:30.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.256+0000 7faabaffd640 1 --2- 172.21.15.22:0/3029411501 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faaa8078bc0 0x7faaa807b0d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7faab0034000 tx=0x7faab0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.265+0000 7f0850a8b740 1 Processor -- start 2024-09-21T05:46:30.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.265+0000 7f0850a8b740 1 -- start start 2024-09-21T05:46:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa957800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa93c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0850a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0844202030 con 0x560ffa541550 2024-09-21T05:46:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f08442010d0 con 0x560ffa56a410 2024-09-21T05:46:30.269 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0844148c00 con 0x560ffa549730 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa957800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0842ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa957800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46664/0 (socket says 172.21.15.22:46664) 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0842ffd640 1 -- 172.21.15.22:0/358320560 learned_addr learned my addr 172.21.15.22:0/358320560 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f08437fe640 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa93c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f08427fc640 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.266+0000 7f0842ffd640 1 -- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 msgr2=0x560ffa957d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0842ffd640 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0842ffd640 1 -- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 msgr2=0x560ffa93c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0842ffd640 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa93c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0842ffd640 1 -- 172.21.15.22:0/358320560 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ffa5b3720 con 0x560ffa541550 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0842ffd640 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa957800 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0834000c00 tx=0x7f0834005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=898d5cd4195fa9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0828ff9640 1 -- 172.21.15.22:0/358320560 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083408da00 con 0x560ffa541550 2024-09-21T05:46:30.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0828ff9640 1 -- 172.21.15.22:0/358320560 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0834088b50 con 0x560ffa541550 2024-09-21T05:46:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.267+0000 7f0828ff9640 1 -- 172.21.15.22:0/358320560 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083409e080 con 0x560ffa541550 2024-09-21T05:46:30.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 -- 172.21.15.22:0/358320560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 msgr2=0x560ffa957800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa957800 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0834000c00 tx=0x7f0834005bd0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 -- 172.21.15.22:0/358320560 shutdown_connections 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa56a410 0x560ffa93c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa957d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 --2- 172.21.15.22:0/358320560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa541550 0x560ffa957800 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 -- 172.21.15.22:0/358320560 >> 172.21.15.22:0/358320560 conn(0x560ffa55e560 msgr2=0x560ffa913ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 -- 172.21.15.22:0/358320560 shutdown_connections 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.268+0000 7f0850a8b740 1 -- 172.21.15.22:0/358320560 wait complete. 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.269+0000 7f0850a8b740 1 -- start start 2024-09-21T05:46:30.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.269+0000 7f0850a8b740 1 Processor -- start 2024-09-21T05:46:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa970b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa9da9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 0x560ffa9cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0844200a80 con 0x560ffa56a410 2024-09-21T05:46:30.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560ffa9d7740 con 0x560ffa541550 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x560ffa97b1a0 con 0x560ffa549730 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0850a8b740 10 client.0 _ll_register_callbacks cb 0x7f0834080750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f08437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 0x560ffa9cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f08437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 0x560ffa9cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46666/0 (socket says 172.21.15.22:46666) 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f08437fe640 1 -- 172.21.15.22:0/1008423341 learned_addr learned my addr 172.21.15.22:0/1008423341 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f0842ffd640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa970b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.270+0000 7f08427fc640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa9da9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08437fe640 1 -- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 msgr2=0x560ffa9da9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08437fe640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa9da9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08437fe640 1 -- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 msgr2=0x560ffa970b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08437fe640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa970b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08437fe640 1 -- 172.21.15.22:0/1008423341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ffa90f8e0 con 0x560ffa56a410 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08437fe640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 0x560ffa9cd3a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f083c017040 tx=0x7f083c0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b80cbfcdbb49ec server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08227fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08227fc640 1 -- 172.21.15.22:0/1008423341 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c004040 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08227fc640 1 -- 172.21.15.22:0/1008423341 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f083c012040 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0834088b50 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08227fc640 1 -- 172.21.15.22:0/1008423341 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f083c0041e0 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f0850a8b740 10 client.7701 fetch_fsmap learned FSMap version 84 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x560ffa9d7480 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.271+0000 7f08227fc640 1 -- 172.21.15.22:0/1008423341 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0834088b50 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f08227fc640 1 -- 172.21.15.22:0/1008423341 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f083c004380 con 0x560ffa56a410 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0850a8b740 10 client.7701 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0850a8b740 10 client.7701 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0850a8b740 20 client.7701 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0850a8b740 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa9d6ac0 0x560ffa9d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0850a8b740 4 client.7701 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-21T05:46:30.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x560ffa956f00 con 0x560ffa9d6ac0 2024-09-21T05:46:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0842ffd640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa9d6ac0 0x560ffa9d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f08227fc640 10 client.7701 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.272+0000 7f0842ffd640 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa9d6ac0 0x560ffa9d6ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ac4d532762238c server_cookie=dbbd765af8583931 in_seq=0 out_seq=0 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f08227fc640 1 -- 172.21.15.22:0/1008423341 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ffa956f00 con 0x560ffa9d6ac0 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f08227fc640 10 client.7701 handle_command_reply: tid=0 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 client.7701 shutdown 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 msgr2=0x560ffa9cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 0x560ffa9cd3a0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f083c017040 tx=0x7f083c0094d0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 shutdown_connections 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x560ffa9d6ac0 0x560ffa9d6ed0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560ffa56a410 0x560ffa9cd3a0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560ffa549730 0x560ffa9da9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 --2- 172.21.15.22:0/1008423341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560ffa541550 0x560ffa970b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f08227fc640 0 client.7701 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 >> 172.21.15.22:0/1008423341 conn(0x560ffa55e560 msgr2=0x560ffa913ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 shutdown_connections 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 1 -- 172.21.15.22:0/1008423341 wait complete. 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.274+0000 7f0850a8b740 20 client.7701 trim_cache size 0 max 16384 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:46:30.280 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "11adb65a-14cb-49e9-ae97-eb64449eee8f", 2024-09-21T05:46:30.281 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:46:30.281 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:46:30.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0830078b90 msgr2=0x7f083007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0830078b90 0x7f083007b0a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0844155170 tx=0x7f0838009040 comp rx=0 tx=0).stop 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 msgr2=0x7f084415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 0x7f084415bb80 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0834009950 tx=0x7f0834033d60 comp rx=0 tx=0).stop 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 shutdown_connections 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0844157110 0x7f08441f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0830078b90 0x7f083007b0a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f08440cece0 0x7f08441f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 --2- 172.21.15.22:0/1203253969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f08440cd390 0x7f084415bb80 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 >> 172.21.15.22:0/1203253969 conn(0x7f084406d550 msgr2=0x7f08440b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 shutdown_connections 2024-09-21T05:46:30.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.275+0000 7f0849a3f640 1 -- 172.21.15.22:0/1203253969 wait complete. 2024-09-21T05:46:30.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:46:30.288 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T05:46:30.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 -- 172.21.15.22:0/2910725086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006e170 msgr2=0x7f5a100b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 --2- 172.21.15.22:0/2910725086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006e170 0x7f5a100b3540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009860 tx=0x7f5a0c02a4b0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 -- 172.21.15.22:0/2910725086 shutdown_connections 2024-09-21T05:46:30.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 --2- 172.21.15.22:0/2910725086 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a100ceb40 0x7f5a100cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 --2- 172.21.15.22:0/2910725086 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006e170 0x7f5a100b3540 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 -- 172.21.15.22:0/2910725086 >> 172.21.15.22:0/2910725086 conn(0x7f5a1008bfc0 msgr2=0x7f5a1009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 -- 172.21.15.22:0/2910725086 shutdown_connections 2024-09-21T05:46:30.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.405+0000 7f5a18594640 1 -- 172.21.15.22:0/2910725086 wait complete. 2024-09-21T05:46:30.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.407+0000 7f5a18594640 1 Processor -- start 2024-09-21T05:46:30.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.407+0000 7f5a18594640 1 -- start start 2024-09-21T05:46:30.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a18594640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006f4b0 0x7f5a100ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a18594640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a100ceb40 0x7f5a101e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a18594640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 0x7f5a100b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a18594640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5a100d0e60 con 0x7f5a101e9df0 2024-09-21T05:46:30.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a18594640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a100d0ce0 con 0x7f5a1006f4b0 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a18594640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5a100d0fe0 con 0x7f5a100ceb40 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 0x7f5a100b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 0x7f5a100b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46670/0 (socket says 172.21.15.22:46670) 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 -- 172.21.15.22:0/4020395576 learned_addr learned my addr 172.21.15.22:0/4020395576 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a100ceb40 0x7f5a101e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16330640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006f4b0 0x7f5a100ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 -- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a100ceb40 msgr2=0x7f5a101e98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a100ceb40 0x7f5a101e98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 -- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006f4b0 msgr2=0x7f5a100ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006f4b0 0x7f5a100ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.408+0000 7f5a16b31640 1 -- 172.21.15.22:0/4020395576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a101e8330 con 0x7f5a101e9df0 2024-09-21T05:46:30.412 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.409+0000 7f5a16b31640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 0x7f5a100b4fa0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800efc0 tx=0x7f5a0800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.409+0000 7f5a06ffd640 1 -- 172.21.15.22:0/4020395576 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08003970 con 0x7f5a101e9df0 2024-09-21T05:46:30.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.409+0000 7f5a06ffd640 1 -- 172.21.15.22:0/4020395576 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a08003c00 con 0x7f5a101e9df0 2024-09-21T05:46:30.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.409+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101f5830 con 0x7f5a101e9df0 2024-09-21T05:46:30.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.409+0000 7f5a06ffd640 1 -- 172.21.15.22:0/4020395576 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08004020 con 0x7f5a101e9df0 2024-09-21T05:46:30.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.409+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a101f5df0 con 0x7f5a101e9df0 2024-09-21T05:46:30.413 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.411+0000 7f5a06ffd640 1 -- 172.21.15.22:0/4020395576 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a080041c0 con 0x7f5a101e9df0 2024-09-21T05:46:30.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.411+0000 7f5a06ffd640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f59f4078b90 0x7f59f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.411+0000 7f5a06ffd640 1 -- 172.21.15.22:0/4020395576 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0809b5d0 con 0x7f5a101e9df0 2024-09-21T05:46:30.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.412+0000 7f5a16330640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f59f4078b90 0x7f59f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.412+0000 7f5a16330640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f59f4078b90 0x7f59f407b0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c034000 tx=0x7f5a0c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.412+0000 7f5a1f5d9740 1 Processor -- start 2024-09-21T05:46:30.415 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.413+0000 7f5a1f5d9740 1 -- start start 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b86390 0x564748f72a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 0x564748f47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b656b0 0x564748f47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564748ea0a60 con 0x564748b5d4d0 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564748e8fe90 con 0x564748b656b0 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564748e93ca0 con 0x564748b86390 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 0x564748f47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 0x564748f47860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46678/0 (socket says 172.21.15.22:46678) 2024-09-21T05:46:30.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1096449675 learned_addr learned my addr 172.21.15.22:0/1096449675 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a16330640 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b86390 0x564748f72a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a16b31640 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b656b0 0x564748f47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b86390 msgr2=0x564748f72a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b86390 0x564748f72a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b656b0 msgr2=0x564748f47da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b656b0 0x564748f47da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.414+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1096449675 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564748c687d0 con 0x564748b5d4d0 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.415+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 0x564748f47860 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5a00001780 tx=0x7f5a00007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7b89abbd292f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.415+0000 7f59f3fff640 1 -- 172.21.15.22:0/1096449675 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a000052e0 con 0x564748b5d4d0 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.415+0000 7f59f3fff640 1 -- 172.21.15.22:0/1096449675 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a00018070 con 0x564748b5d4d0 2024-09-21T05:46:30.418 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.415+0000 7f59f3fff640 1 -- 172.21.15.22:0/1096449675 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000f9b0 con 0x564748b5d4d0 2024-09-21T05:46:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1096449675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 msgr2=0x564748f47860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 0x564748f47860 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5a00001780 tx=0x7f5a00007e30 comp rx=0 tx=0).stop 2024-09-21T05:46:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1096449675 shutdown_connections 2024-09-21T05:46:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b656b0 0x564748f47da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b5d4d0 0x564748f47860 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.419 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1096449675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b86390 0x564748f72a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1096449675 >> 172.21.15.22:0/1096449675 conn(0x564748b7a4e0 msgr2=0x564748f2dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1096449675 shutdown_connections 2024-09-21T05:46:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.416+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1096449675 wait complete. 2024-09-21T05:46:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.417+0000 7f5a1f5d9740 1 -- start start 2024-09-21T05:46:30.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.417+0000 7f5a1f5d9740 1 Processor -- start 2024-09-21T05:46:30.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748f55600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f6f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564748c00740 con 0x564748b86390 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x564748e35970 con 0x564748b5d4d0 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564748f471a0 con 0x564748b656b0 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a1f5d9740 10 client.0 _ll_register_callbacks cb 0x564748e44780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a16b31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f6f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a16b31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f6f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46686/0 (socket says 172.21.15.22:46686) 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.418+0000 7f5a16b31640 1 -- 172.21.15.22:0/2943212158 learned_addr learned my addr 172.21.15.22:0/2943212158 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16330640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748f55600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16b31640 1 -- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 msgr2=0x564748f44250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16b31640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f44250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16b31640 1 -- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 msgr2=0x564748f55600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16b31640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748f55600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16b31640 1 -- 172.21.15.22:0/2943212158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564748bf1970 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a16b31640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f6f040 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800cd00 tx=0x7f5a080099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ce5be286ae2e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f59f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f59f17fa640 1 -- 172.21.15.22:0/2943212158 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0806c040 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f59f17fa640 1 -- 172.21.15.22:0/2943212158 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0804e800 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a00018070 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f59f17fa640 1 -- 172.21.15.22:0/2943212158 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f5a08067df0 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a1f5d9740 10 client.7710 fetch_fsmap learned FSMap version 84 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x564748bcf6a0 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.419+0000 7f59f17fa640 1 -- 172.21.15.22:0/2943212158 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a00018070 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f59f17fa640 1 -- 172.21.15.22:0/2943212158 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f5a0800b040 con 0x564748b86390 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a1f5d9740 10 client.7710 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a1f5d9740 10 client.7710 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:46:30.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a1f5d9740 20 client.7710 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748b44f60 0x564748f850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a1f5d9740 4 client.7710 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564748ea29b0 con 0x564748b44f60 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a16330640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748b44f60 0x564748f850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f59f17fa640 10 client.7710 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.420+0000 7f5a16330640 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748b44f60 0x564748f850d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edfb54c86724796 server_cookie=93cb41bebb68215b in_seq=0 out_seq=0 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f59f17fa640 1 -- 172.21.15.22:0/2943212158 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564748ea29b0 con 0x564748b44f60 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f59f17fa640 10 client.7710 handle_command_reply: tid=0 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 client.7710 shutdown 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 msgr2=0x564748f6f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f6f040 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800cd00 tx=0x7f5a080099b0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 shutdown_connections 2024-09-21T05:46:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748b44f60 0x564748f850d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f6f040 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f59f17fa640 0 client.7710 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f44250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/2943212158 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748f55600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 >> 172.21.15.22:0/2943212158 conn(0x564748b7a4e0 msgr2=0x564748f54e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 shutdown_connections 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/2943212158 wait complete. 2024-09-21T05:46:30.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.421+0000 7f5a1f5d9740 20 client.7710 trim_cache size 0 max 16384 2024-09-21T05:46:30.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.439+0000 7f5a1f5d9740 1 Processor -- start 2024-09-21T05:46:30.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.439+0000 7f5a1f5d9740 1 -- start start 2024-09-21T05:46:30.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.440+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b5d4d0 0x564748f43bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.440+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 0x564748f28890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.440+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b86390 0x564748f28dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.440+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5a102021d0 con 0x564748b656b0 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.440+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a102011f0 con 0x564748b86390 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.440+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5a101f9540 con 0x564748b5d4d0 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 0x564748f28890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 0x564748f28890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46692/0 (socket says 172.21.15.22:46692) 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1336315731 learned_addr learned my addr 172.21.15.22:0/1336315731 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1336315731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b5d4d0 msgr2=0x564748f43bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b5d4d0 0x564748f43bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1336315731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b86390 msgr2=0x564748f28dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b86390 0x564748f28dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 -- 172.21.15.22:0/1336315731 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564748f2a860 con 0x564748b656b0 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 0x564748f28890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000f5a0 tx=0x7f5a00001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a400e0fddb69d34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f59f37fe640 1 -- 172.21.15.22:0/1336315731 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000f7b0 con 0x564748b656b0 2024-09-21T05:46:30.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f59f37fe640 1 -- 172.21.15.22:0/1336315731 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a00021030 con 0x564748b656b0 2024-09-21T05:46:30.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.441+0000 7f59f37fe640 1 -- 172.21.15.22:0/1336315731 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000fce0 con 0x564748b656b0 2024-09-21T05:46:30.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1336315731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 msgr2=0x564748f28890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 0x564748f28890 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000f5a0 tx=0x7f5a00001080 comp rx=0 tx=0).stop 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1336315731 shutdown_connections 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b86390 0x564748f28dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b656b0 0x564748f28890 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/1336315731 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b5d4d0 0x564748f43bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1336315731 >> 172.21.15.22:0/1336315731 conn(0x564748b7a4e0 msgr2=0x564748f6f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1336315731 shutdown_connections 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.442+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/1336315731 wait complete. 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.443+0000 7f5a1f5d9740 1 -- start start 2024-09-21T05:46:30.446 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.443+0000 7f5a1f5d9740 1 Processor -- start 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.444+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748ff21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.444+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f82b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.444+0000 7f5a1f5d9740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f830d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.444+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x564748f2e230 con 0x564748b86390 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.444+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10200dc0 con 0x564748b5d4d0 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.444+0000 7f5a1f5d9740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x564748f2df50 con 0x564748b656b0 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a1f5d9740 10 client.0 _ll_register_callbacks cb 0x7f5a0805f350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f830d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f830d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46708/0 (socket says 172.21.15.22:46708) 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 -- 172.21.15.22:0/3986915368 learned_addr learned my addr 172.21.15.22:0/3986915368 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:46:30.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a15b2f640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f82b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16330640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748ff21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 -- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 msgr2=0x564748f82b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f82b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 -- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 msgr2=0x564748ff21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748ff21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 -- 172.21.15.22:0/3986915368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564748f55b60 con 0x564748b86390 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a16b31640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f830d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f5a0805fb80 tx=0x7f5a0800e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852f1a3d1a0d20ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f59f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f59f17fa640 1 -- 172.21.15.22:0/3986915368 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a08062b30 con 0x564748b86390 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f59f17fa640 1 -- 172.21.15.22:0/3986915368 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0809b0b0 con 0x564748b86390 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a00021030 con 0x564748b86390 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f59f17fa640 1 -- 172.21.15.22:0/3986915368 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f5a08099a40 con 0x564748b86390 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a1f5d9740 10 client.7716 fetch_fsmap learned FSMap version 84 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.445+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x564748f442e0 con 0x564748b86390 2024-09-21T05:46:30.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f59f17fa640 1 -- 172.21.15.22:0/3986915368 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a00021030 con 0x564748b86390 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f59f17fa640 1 -- 172.21.15.22:0/3986915368 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7f5a0809b2b0 con 0x564748b86390 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 10 client.7716 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 10 client.7716 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 20 client.7716 populate_metadata read hostname 'smithi022' 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748f575c0 0x564748ff6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 4 client.7716 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564748e9e460 con 0x564748f575c0 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a16330640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748f575c0 0x564748ff6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f59f17fa640 10 client.7716 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a16330640 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748f575c0 0x564748ff6010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4bcb5b2cfb05ca server_cookie=b92f71574eb954f0 in_seq=0 out_seq=0 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f59f17fa640 1 -- 172.21.15.22:0/3986915368 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x564748e9e460 con 0x564748f575c0 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f59f17fa640 10 client.7716 handle_command_reply: tid=0 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.446+0000 7f5a1f5d9740 1 client.7716 shutdown 2024-09-21T05:46:30.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 msgr2=0x564748f830d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f830d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f5a0805fb80 tx=0x7f5a0800e300 comp rx=0 tx=0).stop 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 shutdown_connections 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x564748f575c0 0x564748ff6010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x564748b86390 0x564748f830d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x564748b656b0 0x564748f82b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 --2- 172.21.15.22:0/3986915368 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x564748b5d4d0 0x564748ff21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f59f17fa640 0 client.7716 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 >> 172.21.15.22:0/3986915368 conn(0x564748b7a4e0 msgr2=0x564748f43bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 shutdown_connections 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 1 -- 172.21.15.22:0/3986915368 wait complete. 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a1f5d9740 20 client.7716 trim_cache size 0 max 16384 2024-09-21T05:46:30.453 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "11adb65a-14cb-49e9-ae97-eb64449eee8f": { 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "11adb65a-14cb-49e9-ae97-eb64449eee8f", 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,repair" 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "11adb65a-14cb-49e9-ae97-eb64449eee8f": { 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:46:30.272258+0000", 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:46:30.454 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:46:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:46:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:46:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:46:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:30.455 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:46:30.455 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f59f4078b90 msgr2=0x7f59f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f59f4078b90 0x7f59f407b0a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c034000 tx=0x7f5a0c004730 comp rx=0 tx=0).stop 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 msgr2=0x7f5a100b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 0x7f5a100b4fa0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800efc0 tx=0x7f5a0800c800 comp rx=0 tx=0).stop 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 shutdown_connections 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a101e9df0 0x7f5a100b4fa0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f59f4078b90 0x7f59f407b0a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a100ceb40 0x7f5a101e98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 --2- 172.21.15.22:0/4020395576 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5a1006f4b0 0x7f5a100ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.447+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 >> 172.21.15.22:0/4020395576 conn(0x7f5a1008bfc0 msgr2=0x7f5a100b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.448+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 shutdown_connections 2024-09-21T05:46:30.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:30.448+0000 7f5a18594640 1 -- 172.21.15.22:0/4020395576 wait complete. 2024-09-21T05:46:30.461 INFO:tasks.cephfs.filesystem:scrub status for tag:11adb65a-14cb-49e9-ae97-eb64449eee8f - {'path': '/', 'tag': '11adb65a-14cb-49e9-ae97-eb64449eee8f', 'options': 'recursive,repair'} 2024-09-21T05:46:31.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:31.248+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 7 ==== log(4 entries from seq 1 at 2024-09-21T05:46:30.273923+0000) ==== 695+0+0 (secure 0 0 0) 0x7faaac098550 con 0x7faabc06e270 2024-09-21T05:46:31.250 INFO:teuthology.orchestra.run.smithi022.stdout: cluster: 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: id: cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: health: HEALTH_OK 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: services: 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: mon: 3 daemons, quorum a,b,c (age 7m) 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: mgr: x(active, since 7m), standbys: y 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: mds: 1/1 daemons up, 3 standby 2024-09-21T05:46:31.251 INFO:teuthology.orchestra.run.smithi022.stdout: osd: 8 osds: 8 up (since 7m), 8 in (since 7m) 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: data: 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: volumes: 1/1 healthy 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: pools: 3 pools, 97 pgs 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: objects: 26 objects, 6.6 MiB 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: usage: 280 MiB used, 720 GiB / 720 GiB avail 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: pgs: 97 active+clean 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: io: 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: client: 579 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:46:31.252 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:46:30.273923+0000 mds.c [INF] scrub queued for path: / 2024-09-21T05:46:31.253 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:46:30.273926+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-21T05:46:31.253 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:46:30.273933+0000 mds.c [INF] scrub summary: active paths [/] 2024-09-21T05:46:31.253 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:46:30.275849+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-21T05:46:35.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:46:35.254+0000 7faaa3fff640 1 -- 172.21.15.22:0/3029411501 <== mon.0 v2:172.21.15.22:3300/0 8 ==== log(1 entries from seq 5 at 2024-09-21T05:46:33.554431+0000) ==== 189+0+0 (secure 0 0 0) 0x7faaac062080 con 0x7faabc06e270 2024-09-21T05:46:35.257 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:46:33.554431+0000 mds.c [INF] scrub summary: idle 2024-09-21T05:46:52.590 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:46:52.593 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:47:00.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:47:00.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T05:47:00.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 -- 172.21.15.22:0/1071941792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 msgr2=0x7fe1a00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 --2- 172.21.15.22:0/1071941792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a00ccbc0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0074af0 tx=0x7fe19002a500 comp rx=0 tx=0).stop 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 -- 172.21.15.22:0/1071941792 shutdown_connections 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 --2- 172.21.15.22:0/1071941792 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1a00cf320 0x7fe1a00d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 --2- 172.21.15.22:0/1071941792 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a00ccbc0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 -- 172.21.15.22:0/1071941792 >> 172.21.15.22:0/1071941792 conn(0x7fe1a006d4f0 msgr2=0x7fe1a0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 -- 172.21.15.22:0/1071941792 shutdown_connections 2024-09-21T05:47:00.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.593+0000 7fe1a697b640 1 -- 172.21.15.22:0/1071941792 wait complete. 2024-09-21T05:47:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.595+0000 7fe1a697b640 1 Processor -- start 2024-09-21T05:47:00.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.595+0000 7fe1a697b640 1 -- start start 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a00b7cc0 0x7fe1a014b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1a00cf320 0x7fe1a01f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a01f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00d1410 con 0x7fe1a01600b0 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00d1290 con 0x7fe1a00b7cc0 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a00d1590 con 0x7fe1a00cf320 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a01f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a01f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46754/0 (socket says 172.21.15.22:46754) 2024-09-21T05:47:00.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 -- 172.21.15.22:0/2830455169 learned_addr learned my addr 172.21.15.22:0/2830455169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe19ffff640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a00b7cc0 0x7fe1a014b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1a00cf320 0x7fe1a01f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 -- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1a00cf320 msgr2=0x7fe1a01f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1a00cf320 0x7fe1a01f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 -- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a00b7cc0 msgr2=0x7fe1a014b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a00b7cc0 0x7fe1a014b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 -- 172.21.15.22:0/2830455169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a00ca3d0 con 0x7fe1a01600b0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a01f7350 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe19800caa0 tx=0x7fe19800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe19cff9640 1 -- 172.21.15.22:0/2830455169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19800ad90 con 0x7fe1a01600b0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe19cff9640 1 -- 172.21.15.22:0/2830455169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19800f540 con 0x7fe1a01600b0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.597+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a015b480 con 0x7fe1a01600b0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.598+0000 7fe19cff9640 1 -- 172.21.15.22:0/2830455169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19800f990 con 0x7fe1a01600b0 2024-09-21T05:47:00.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.598+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a014e4a0 con 0x7fe1a01600b0 2024-09-21T05:47:00.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.599+0000 7fe19cff9640 1 -- 172.21.15.22:0/2830455169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe19800fb30 con 0x7fe1a01600b0 2024-09-21T05:47:00.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.600+0000 7fe19cff9640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe18c078b90 0x7fe18c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.600+0000 7fe19cff9640 1 -- 172.21.15.22:0/2830455169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe19809b750 con 0x7fe1a01600b0 2024-09-21T05:47:00.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.600+0000 7fe19ffff640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe18c078b90 0x7fe18c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.601+0000 7fe19ffff640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe18c078b90 0x7fe18c07b0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe190033000 tx=0x7fe190004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.601+0000 7fe1ad9c5740 1 Processor -- start 2024-09-21T05:47:00.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.601+0000 7fe1ad9c5740 1 -- start start 2024-09-21T05:47:00.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c25860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd38436b0 0x55bcd3c25da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3b7ea60 con 0x55bcd383b4d0 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3b6de90 con 0x55bcd3864390 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bcd3b71ca0 con 0x55bcd38436b0 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe19f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c25860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe19f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c25860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46762/0 (socket says 172.21.15.22:46762) 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe19f7fe640 1 -- 172.21.15.22:0/3592875513 learned_addr learned my addr 172.21.15.22:0/3592875513 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.602+0000 7fe19ffff640 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd38436b0 0x55bcd3c25da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe19f7fe640 1 -- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd38436b0 msgr2=0x55bcd3c25da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd38436b0 0x55bcd3c25da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe19f7fe640 1 -- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 msgr2=0x55bcd3c50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe19f7fe640 1 -- 172.21.15.22:0/3592875513 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd39467d0 con 0x55bcd383b4d0 2024-09-21T05:47:00.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c25860 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe1940099d0 tx=0x7fe194009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc978e6ab1be96d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe185ffb640 1 -- 172.21.15.22:0/3592875513 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194017070 con 0x55bcd383b4d0 2024-09-21T05:47:00.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe185ffb640 1 -- 172.21.15.22:0/3592875513 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19400eae0 con 0x55bcd383b4d0 2024-09-21T05:47:00.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.603+0000 7fe185ffb640 1 -- 172.21.15.22:0/3592875513 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194013470 con 0x55bcd383b4d0 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/3592875513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 msgr2=0x55bcd3c25860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c25860 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fe1940099d0 tx=0x7fe194009e90 comp rx=0 tx=0).stop 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/3592875513 shutdown_connections 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd38436b0 0x55bcd3c25da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c25860 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/3592875513 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/3592875513 >> 172.21.15.22:0/3592875513 conn(0x55bcd38584e0 msgr2=0x55bcd3c0bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/3592875513 shutdown_connections 2024-09-21T05:47:00.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.604+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/3592875513 wait complete. 2024-09-21T05:47:00.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.605+0000 7fe1ad9c5740 1 -- start start 2024-09-21T05:47:00.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.605+0000 7fe1ad9c5740 1 Processor -- start 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.606+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c46950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.606+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd38436b0 0x55bcd3c47990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.606+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.606+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55bcd38de740 con 0x55bcd383b4d0 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.606+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3b13970 con 0x55bcd38436b0 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.606+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bcd3c241e0 con 0x55bcd3864390 2024-09-21T05:47:00.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe1ad9c5740 10 client.0 _ll_register_callbacks cb 0x55bcd3b22780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c46950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c46950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46778/0 (socket says 172.21.15.22:46778) 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 -- 172.21.15.22:0/730561764 learned_addr learned my addr 172.21.15.22:0/730561764 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd38436b0 0x55bcd3c47990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 -- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 msgr2=0x55bcd3c4d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c4d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 -- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd38436b0 msgr2=0x55bcd3c47990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd38436b0 0x55bcd3c47990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 -- 172.21.15.22:0/730561764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd38cf970 con 0x55bcd383b4d0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe19ffff640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c46950 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe19002aaf0 tx=0x7fe190009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7001e8aada9e7d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe17effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe17effd640 1 -- 172.21.15.22:0/730561764 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19004e070 con 0x55bcd383b4d0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe17effd640 1 -- 172.21.15.22:0/730561764 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe190003c50 con 0x55bcd383b4d0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.607+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe19400eae0 con 0x55bcd383b4d0 2024-09-21T05:47:00.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe17effd640 1 -- 172.21.15.22:0/730561764 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe190059050 con 0x55bcd383b4d0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 10 client.7728 fetch_fsmap learned FSMap version 84 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55bcd38ad6a0 con 0x55bcd383b4d0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe17effd640 1 -- 172.21.15.22:0/730561764 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19400eae0 con 0x55bcd383b4d0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe17effd640 1 -- 172.21.15.22:0/730561764 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7fe19005c020 con 0x55bcd383b4d0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 10 client.7728 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 10 client.7728 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 20 client.7728 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3822f60 0x55bcd3c22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 4 client.7728 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd3b809b0 con 0x55bcd3822f60 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3822f60 0x55bcd3c22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe17effd640 10 client.7728 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.608+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3822f60 0x55bcd3c22fb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f24097ef172b59 server_cookie=fdb661732a9a1c45 in_seq=0 out_seq=0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe17effd640 1 -- 172.21.15.22:0/730561764 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bcd3b809b0 con 0x55bcd3822f60 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe17effd640 10 client.7728 handle_command_reply: tid=0 2024-09-21T05:47:00.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 client.7728 shutdown 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 msgr2=0x55bcd3c46950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c46950 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fe19002aaf0 tx=0x7fe190009260 comp rx=0 tx=0).stop 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 shutdown_connections 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3822f60 0x55bcd3c22fb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c4d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd38436b0 0x55bcd3c47990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe17effd640 0 client.7728 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/730561764 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd383b4d0 0x55bcd3c46950 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 >> 172.21.15.22:0/730561764 conn(0x55bcd38584e0 msgr2=0x55bcd3c06a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.609+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 shutdown_connections 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.610+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/730561764 wait complete. 2024-09-21T05:47:00.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.610+0000 7fe1ad9c5740 20 client.7728 trim_cache size 0 max 16384 2024-09-21T05:47:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.628+0000 7fe1ad9c5740 1 Processor -- start 2024-09-21T05:47:00.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.628+0000 7fe1ad9c5740 1 -- start start 2024-09-21T05:47:00.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.629+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd383b4d0 0x55bcd3c73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.629+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c47190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.629+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.629+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a01fff70 con 0x55bcd38436b0 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.629+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00cbb30 con 0x55bcd383b4d0 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.629+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1a01f4620 con 0x55bcd3864390 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c47190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c47190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46780/0 (socket says 172.21.15.22:46780) 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 -- 172.21.15.22:0/2679463967 learned_addr learned my addr 172.21.15.22:0/2679463967 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19ffff640 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd383b4d0 0x55bcd3c73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 -- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 msgr2=0x55bcd3c476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 -- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd383b4d0 msgr2=0x55bcd3c73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd383b4d0 0x55bcd3c73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 -- 172.21.15.22:0/2679463967 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd3c06120 con 0x55bcd38436b0 2024-09-21T05:47:00.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c47190 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe1940091f0 tx=0x7fe194009060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed54a8aeece04a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe1857fa640 1 -- 172.21.15.22:0/2679463967 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194029020 con 0x55bcd38436b0 2024-09-21T05:47:00.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe1857fa640 1 -- 172.21.15.22:0/2679463967 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1940138f0 con 0x55bcd38436b0 2024-09-21T05:47:00.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.630+0000 7fe1857fa640 1 -- 172.21.15.22:0/2679463967 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194017040 con 0x55bcd38436b0 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/2679463967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 msgr2=0x55bcd3c47190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c47190 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fe1940091f0 tx=0x7fe194009060 comp rx=0 tx=0).stop 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/2679463967 shutdown_connections 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd3864390 0x55bcd3c476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c47190 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/2679463967 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd383b4d0 0x55bcd3c73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/2679463967 >> 172.21.15.22:0/2679463967 conn(0x55bcd38584e0 msgr2=0x55bcd3c069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/2679463967 shutdown_connections 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.631+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/2679463967 wait complete. 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.632+0000 7fe1ad9c5740 1 -- start start 2024-09-21T05:47:00.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.632+0000 7fe1ad9c5740 1 Processor -- start 2024-09-21T05:47:00.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd383b4d0 0x55bcd3c57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c32090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c47190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a00cb480 con 0x55bcd38436b0 2024-09-21T05:47:00.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55bcd3cd1010 con 0x55bcd3864390 2024-09-21T05:47:00.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55bcd3c4a840 con 0x55bcd383b4d0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1ad9c5740 10 client.0 _ll_register_callbacks cb 0x7fe190004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c32090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c32090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46796/0 (socket says 172.21.15.22:46796) 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 -- 172.21.15.22:0/531369780 learned_addr learned my addr 172.21.15.22:0/531369780 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19ffff640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd383b4d0 0x55bcd3c57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe1a4f18640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c47190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 -- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd383b4d0 msgr2=0x55bcd3c57eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd383b4d0 0x55bcd3c57eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 -- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 msgr2=0x55bcd3c47190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c47190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 -- 172.21.15.22:0/531369780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd3c4aa90 con 0x55bcd38436b0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19ffff640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd383b4d0 0x55bcd3c57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe19f7fe640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c32090 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe194015040 tx=0x7fe1940136f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7948252df9e30df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.634+0000 7fe17effd640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:00.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe17effd640 1 -- 172.21.15.22:0/531369780 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194017070 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe17effd640 1 -- 172.21.15.22:0/531369780 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7fe194004030 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1940138f0 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe17effd640 1 -- 172.21.15.22:0/531369780 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe194029040 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 10 client.7734 fetch_fsmap learned FSMap version 84 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x55bcd3cd0a20 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe17effd640 1 -- 172.21.15.22:0/531369780 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1940138f0 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe17effd640 1 -- 172.21.15.22:0/531369780 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7fe194004230 con 0x55bcd38436b0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 10 client.7734 fetch_fsmap finished waiting for FSMap version 84 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 10 client.7734 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 20 client.7734 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3c476d0 0x55bcd3cd4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 4 client.7734 mds_command: new command op to 7482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 --> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bcd3b7c460 con 0x55bcd3c476d0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.635+0000 7fe19ffff640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3c476d0 0x55bcd3cd4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe17effd640 10 client.7734 ms_handle_connect on v2:172.21.15.22:6832/1062944206 2024-09-21T05:47:00.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe19ffff640 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3c476d0 0x55bcd3cd4380 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6838f0f843c642d6 server_cookie=6649c604d1a93bf7 in_seq=0 out_seq=0 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe17effd640 1 -- 172.21.15.22:0/531369780 <== mds.0 v2:172.21.15.22:6832/1062944206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55bcd3b7c460 con 0x55bcd3c476d0 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe17effd640 10 client.7734 handle_command_reply: tid=0 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 client.7734 shutdown 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 msgr2=0x55bcd3c32090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c32090 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe194015040 tx=0x7fe1940136f0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 shutdown_connections 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:6832/1062944206,v1:172.21.15.22:6834/1062944206] conn(0x55bcd3c476d0 0x55bcd3cd4380 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55bcd3864390 0x55bcd3c47190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55bcd38436b0 0x55bcd3c32090 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 --2- 172.21.15.22:0/531369780 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55bcd383b4d0 0x55bcd3c57eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe17effd640 0 client.7734 ms_handle_reset on v2:172.21.15.22:6832/1062944206 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 >> 172.21.15.22:0/531369780 conn(0x55bcd38584e0 msgr2=0x55bcd3c4c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 shutdown_connections 2024-09-21T05:47:00.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 1 -- 172.21.15.22:0/531369780 wait complete. 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.636+0000 7fe1ad9c5740 20 client.7734 trim_cache size 0 max 16384 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "11adb65a-14cb-49e9-ae97-eb64449eee8f": { 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:46:30.272258+0000", 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:47:00.643 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:47:00.644 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:47:00.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:00.644 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:00.644 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe18c078b90 msgr2=0x7fe18c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe18c078b90 0x7fe18c07b0a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe190033000 tx=0x7fe190004730 comp rx=0 tx=0).stop 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 msgr2=0x7fe1a01f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a01f7350 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe19800caa0 tx=0x7fe19800cf60 comp rx=0 tx=0).stop 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 shutdown_connections 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a01600b0 0x7fe1a01f7350 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe18c078b90 0x7fe18c07b0a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1a00cf320 0x7fe1a01f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 --2- 172.21.15.22:0/2830455169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a00b7cc0 0x7fe1a014b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 >> 172.21.15.22:0/2830455169 conn(0x7fe1a006d4f0 msgr2=0x7fe1a015f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 shutdown_connections 2024-09-21T05:47:00.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.637+0000 7fe1a697b640 1 -- 172.21.15.22:0/2830455169 wait complete. 2024-09-21T05:47:00.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:47:00.751 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.866+0000 7efc34e94640 1 -- 172.21.15.22:0/2579279431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc3006e0e0 msgr2=0x7efc3006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.866+0000 7efc34e94640 1 --2- 172.21.15.22:0/2579279431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc3006e0e0 0x7efc3006e4f0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7efc300d3de0 tx=0x7efc2802aac0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.867+0000 7efc34e94640 1 -- 172.21.15.22:0/2579279431 shutdown_connections 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.867+0000 7efc34e94640 1 --2- 172.21.15.22:0/2579279431 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc3006e0e0 0x7efc3006e4f0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.867+0000 7efc34e94640 1 --2- 172.21.15.22:0/2579279431 >> v2:172.21.15.134:3300/0 conn(0x7efc300b66b0 0x7efc300b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.867+0000 7efc34e94640 1 -- 172.21.15.22:0/2579279431 >> 172.21.15.22:0/2579279431 conn(0x7efc3009d7f0 msgr2=0x7efc300d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:00.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.867+0000 7efc34e94640 1 -- 172.21.15.22:0/2579279431 shutdown_connections 2024-09-21T05:47:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.867+0000 7efc34e94640 1 -- 172.21.15.22:0/2579279431 wait complete. 2024-09-21T05:47:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.869+0000 7efc34e94640 1 Processor -- start 2024-09-21T05:47:00.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.869+0000 7efc34e94640 1 -- start start 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc34e94640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc3006e0e0 0x7efc300ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc34e94640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc300b9970 0x7efc3015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc34e94640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 0x7efc300cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc34e94640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7efc3006ea30 con 0x7efc300cc180 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc34e94640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7efc300d5010 con 0x7efc3006e0e0 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc34e94640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7efc3006ebb0 con 0x7efc300b9970 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 0x7efc300cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 0x7efc300cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46808/0 (socket says 172.21.15.22:46808) 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 -- 172.21.15.22:0/2534285043 learned_addr learned my addr 172.21.15.22:0/2534285043 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2e59c640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc3006e0e0 0x7efc300ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2dd9b640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc300b9970 0x7efc3015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 -- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc300b9970 msgr2=0x7efc3015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc300b9970 0x7efc3015d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 -- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc3006e0e0 msgr2=0x7efc300ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc3006e0e0 0x7efc300ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.870+0000 7efc2ed9d640 1 -- 172.21.15.22:0/2534285043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc300ce8f0 con 0x7efc300cc180 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.871+0000 7efc2ed9d640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 0x7efc300cc590 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7efc28009d60 tx=0x7efc28003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.871+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28044070 con 0x7efc300cc180 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.871+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc28004250 con 0x7efc300cc180 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.871+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc300ccad0 con 0x7efc300cc180 2024-09-21T05:47:00.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.871+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28004680 con 0x7efc300cc180 2024-09-21T05:47:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.871+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc300c89c0 con 0x7efc300cc180 2024-09-21T05:47:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.872+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc300c8450 con 0x7efc300cc180 2024-09-21T05:47:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.873+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc28036400 con 0x7efc300cc180 2024-09-21T05:47:00.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.873+0000 7efc1effd640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc10078bc0 0x7efc1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:00.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.873+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5709+0+0 (secure 0 0 0) 0x7efc280bb3d0 con 0x7efc300cc180 2024-09-21T05:47:00.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.873+0000 7efc2e59c640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc10078bc0 0x7efc1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:00.876 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.874+0000 7efc2e59c640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc10078bc0 0x7efc1007b0d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efc20000fd0 tx=0x7efc20009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:00.880 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:00.878+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc300ccad0 con 0x7efc300cc180 2024-09-21T05:47:01.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.103+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efc3006e4f0 con 0x7efc300cc180 2024-09-21T05:47:01.106 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:47:01.104+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:47:01.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.105+0000 7efc1effd640 1 -- 172.21.15.22:0/2534285043 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v85) ==== 161+0+0 (secure 0 0 0) 0x7efc300ce8f0 con 0x7efc300cc180 2024-09-21T05:47:01.107 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc10078bc0 msgr2=0x7efc1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc10078bc0 0x7efc1007b0d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efc20000fd0 tx=0x7efc20009040 comp rx=0 tx=0).stop 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 msgr2=0x7efc300cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 0x7efc300cc590 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7efc28009d60 tx=0x7efc28003f10 comp rx=0 tx=0).stop 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 shutdown_connections 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7efc300cc180 0x7efc300cc590 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7efc10078bc0 0x7efc1007b0d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7efc300b9970 0x7efc3015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 --2- 172.21.15.22:0/2534285043 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7efc3006e0e0 0x7efc300ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 >> 172.21.15.22:0/2534285043 conn(0x7efc3009d7f0 msgr2=0x7efc300b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 shutdown_connections 2024-09-21T05:47:01.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.109+0000 7efc34e94640 1 -- 172.21.15.22:0/2534285043 wait complete. 2024-09-21T05:47:01.116 INFO:tasks.ceph.mds.c.smithi022.stderr: -7847> 2024-09-21T05:46:24.159+0000 7f81eb182640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:47:01.126 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-09-21T05:47:01.180 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:47:01.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.182+0000 7fd550b99280 1 -- start start 2024-09-21T05:47:01.184 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.182+0000 7fd550b99280 1 Processor -- start 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd550b99280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac7680 0x5629a67e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd550b99280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 0x5629a67e0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd550b99280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac8400 0x5629a67e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd550b99280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629a68ee180 con 0x5629a5ac7b00 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd550b99280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5629a68ee300 con 0x5629a5ac8400 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd550b99280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5629a68ee480 con 0x5629a5ac7680 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd54eb3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 0x5629a67e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd54eb3f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 0x5629a67e0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46848/0 (socket says 172.21.15.22:46848) 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd54eb3f640 1 -- 172.21.15.22:0/1477642073 learned_addr learned my addr 172.21.15.22:0/1477642073 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd54f340640 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac7680 0x5629a67e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.183+0000 7fd54fb41640 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac8400 0x5629a67e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54eb3f640 1 -- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac7680 msgr2=0x5629a67e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac7680 0x5629a67e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54eb3f640 1 -- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac8400 msgr2=0x5629a67e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac8400 0x5629a67e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54eb3f640 1 -- 172.21.15.22:0/1477642073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a5a50780 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 0x5629a67e0000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x5629a6837c20 tx=0x5629a5b17320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67691955ac59c588 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54cb3b640 1 -- 172.21.15.22:0/1477642073 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5629a1a08ea0 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54cb3b640 1 -- 172.21.15.22:0/1477642073 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5629a1a0e380 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54cb3b640 1 -- 172.21.15.22:0/1477642073 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5629a1a09380 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a5a503c0 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5629a5a51680 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd550b99280 4 waiting for MDS map... 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54cb3b640 1 -- 172.21.15.22:0/1477642073 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x5629a6904600 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd54cb3b640 1 -- 172.21.15.22:0/1477642073 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 85) ==== 2893+0+0 (secure 0 0 0) 0x5629a5a6d180 con 0x5629a5ac7b00 2024-09-21T05:47:01.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd550b99280 4 Got MDS map 85 2024-09-21T05:47:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.184+0000 7fd550b99280 10 main: JournalTool::main 2024-09-21T05:47:01.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.185+0000 7fd550b99280 4 main: JournalTool: connecting to RADOS... 2024-09-21T05:47:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.185+0000 7fd550b99280 1 Processor -- start 2024-09-21T05:47:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.185+0000 7fd550b99280 1 -- start start 2024-09-21T05:47:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd550b99280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd550b99280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd550b99280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd550b99280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629a68ee600 con 0x5629a5ac6900 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd550b99280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5629a68ee780 con 0x5629a5ac6d80 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd550b99280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5629a68ee900 con 0x5629a5ac6480 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46856/0 (socket says 172.21.15.22:46856) 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 -- 172.21.15.22:0/695831906 learned_addr learned my addr 172.21.15.22:0/695831906 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54fb41640 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 -- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 msgr2=0x5629a67e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 -- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 msgr2=0x5629a67e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 -- 172.21.15.22:0/695831906 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a5a514a0 con 0x5629a5ac6900 2024-09-21T05:47:01.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54fb41640 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54f340640 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x5629a6913050 tx=0x5629a6970000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a44014f4455209c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54ab37640 1 -- 172.21.15.22:0/695831906 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5629a1a08ea0 con 0x5629a5ac6900 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54ab37640 1 -- 172.21.15.22:0/695831906 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5629a1a0f180 con 0x5629a5ac6900 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.187+0000 7fd54ab37640 1 -- 172.21.15.22:0/695831906 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5629a1a09380 con 0x5629a5ac6900 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 -- 172.21.15.22:0/695831906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 msgr2=0x5629a67e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x5629a6913050 tx=0x5629a6970000 comp rx=0 tx=0).stop 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 -- 172.21.15.22:0/695831906 shutdown_connections 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 --2- 172.21.15.22:0/695831906 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 -- 172.21.15.22:0/695831906 >> 172.21.15.22:0/695831906 conn(0x5629a5ac7200 msgr2=0x5629a5b3b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 -- 172.21.15.22:0/695831906 shutdown_connections 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.188+0000 7fd550b99280 1 -- 172.21.15.22:0/695831906 wait complete. 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.189+0000 7fd550b99280 1 Processor -- start 2024-09-21T05:47:01.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.189+0000 7fd550b99280 1 -- start start 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd550b99280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd550b99280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd550b99280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd550b99280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5629a68efb00 con 0x5629a5ac6900 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd550b99280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5629a68ef980 con 0x5629a5ac6d80 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd550b99280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5629a68ef800 con 0x5629a5ac6480 2024-09-21T05:47:01.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46860/0 (socket says 172.21.15.22:46860) 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 -- 172.21.15.22:0/1511495389 learned_addr learned my addr 172.21.15.22:0/1511495389 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 msgr2=0x5629a67e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54fb41640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac6480 0x5629a67e1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 msgr2=0x5629a67e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.190+0000 7fd54f340640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac6d80 0x5629a67e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd54f340640 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629a5a50780 con 0x5629a5ac6900 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd54f340640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x5629a6913d10 tx=0x5629a69701e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd549334640 1 -- 172.21.15.22:0/1511495389 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5629a1a09040 con 0x5629a5ac6900 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd549334640 1 -- 172.21.15.22:0/1511495389 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5629a1a0efc0 con 0x5629a5ac6900 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5629a5a503c0 con 0x5629a5ac6900 2024-09-21T05:47:01.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd549334640 1 -- 172.21.15.22:0/1511495389 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5629a1a08ea0 con 0x5629a5ac6900 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629a5a51680 con 0x5629a5ac6900 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.191+0000 7fd550b99280 4 main: JournalTool: resolving pool 14 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd549334640 1 -- 172.21.15.22:0/1511495389 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x5629a6818c00 con 0x5629a5ac6900 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd549334640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x5629a5ac9180 0x5629a67e2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd549334640 1 -- 172.21.15.22:0/1511495389 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x5629a6816a00 con 0x5629a5ac6900 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd550b99280 4 main: JournalTool: creating IoCtx.. 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd550b99280 4 main: Executing for rank 0 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd550b99280 4 Filtering by inode '1099511627777' 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x5629a5ac6000 0x5629a67e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5629a6972800 con 0x5629a5ac6000 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd54fb41640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x5629a5ac9180 0x5629a67e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.193+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x5629a5ac6000 0x5629a67e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.194+0000 7fd54fb41640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x5629a5ac9180 0x5629a67e2100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5629a5ba9b60 tx=0x5629a69702a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.194+0000 7fd54eb3f640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x5629a5ac6000 0x5629a67e2680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.196+0000 7fd54eb3f640 1 -- 172.21.15.22:0/1511495389 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5629a5b56000 con 0x5629a5ac6000 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.196+0000 7fd550b99280 4 Pointer 400.00000000 is readable 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.196+0000 7fd550b99280 4 JournalScanner::scan: reading header object '200.00000000' 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.196+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x5629a5ac8880 0x5629a67e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.196+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5629a6972c00 con 0x5629a5ac8880 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.197+0000 7fd54fb41640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x5629a5ac8880 0x5629a67e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.197+0000 7fd54fb41640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x5629a5ac8880 0x5629a67e2c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd54fb41640 1 -- 172.21.15.22:0/1511495389 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5629a5b56500 con 0x5629a5ac8880 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 10 Header 0x400000 0x407d21 0x40809d 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 10 Starting journal scan from offset 0x407d21 2024-09-21T05:47:01.200 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5629a6973000 con 0x5629a5ac8880 2024-09-21T05:47:01.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd54fb41640 1 -- 172.21.15.22:0/1511495389 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv27 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5629a5b56500 con 0x5629a5ac8880 2024-09-21T05:47:01.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-09-21T05:47:01.201 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 10 Parsing data, 0x37c bytes available 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 10 Attempting decode at 0x407d21 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 10 Consumed 0x37c bytes 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 10 Valid entry at 0x407d21 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x5629a5ac8d00 0x5629a67e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.198+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x5629a6973400 con 0x5629a5ac8d00 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd54f340640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x5629a5ac8d00 0x5629a67e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd54f340640 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x5629a5ac8d00 0x5629a67e3180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd54f340640 1 -- 172.21.15.22:0/1511495389 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5629a5b56780 con 0x5629a5ac8d00 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stdout:Events by type: 2024-09-21T05:47:01.202 INFO:teuthology.orchestra.run.smithi022.stdout:Errors: 0 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd550b99280 4 Reached end of journal objects 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd550b99280 4 Scanned objects, 0 missing, 1 valid 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd550b99280 4 Events scanned, 0 gaps 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd550b99280 4 Found 1 valid events 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd550b99280 4 Selected 0 events events for processing 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.199+0000 7fd550b99280 4 0 events 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x5629a5ac8d00 msgr2=0x5629a67e3180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x5629a5ac8d00 0x5629a67e3180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x5629a5ac8880 msgr2=0x5629a67e2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x5629a5ac8880 0x5629a67e2c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x5629a5ac6000 msgr2=0x5629a67e2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x5629a5ac6000 0x5629a67e2680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x5629a5ac9180 msgr2=0x5629a67e2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd54f340640 1 -- 172.21.15.22:0/1511495389 reap_dead start 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x5629a5ac9180 0x5629a67e2100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5629a5ba9b60 tx=0x5629a69702a0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 msgr2=0x5629a67e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x5629a6913d10 tx=0x5629a69701e0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 shutdown_connections 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac6900 0x5629a67e1600 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1511495389 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x5629a5ac9180 0x5629a67e2100 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 >> 172.21.15.22:0/1511495389 conn(0x5629a5ac7200 msgr2=0x5629a5b3b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 shutdown_connections 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1511495389 wait complete. 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 msgr2=0x5629a67e0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 0x5629a67e0000 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x5629a6837c20 tx=0x5629a5b17320 comp rx=0 tx=0).stop 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 shutdown_connections 2024-09-21T05:47:01.205 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5629a5ac8400 0x5629a67e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5629a5ac7b00 0x5629a67e0000 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 --2- 172.21.15.22:0/1477642073 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5629a5ac7680 0x5629a67e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 >> 172.21.15.22:0/1477642073 conn(0x5629a5ac7f80 msgr2=0x5629a5b3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 shutdown_connections 2024-09-21T05:47:01.206 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.200+0000 7fd550b99280 1 -- 172.21.15.22:0/1477642073 wait complete. 2024-09-21T05:47:01.207 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.080321, result: Events by type: Errors: 0 2024-09-21T05:47:01.207 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-09-21T05:47:01.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.261+0000 7fec6681f280 1 -- start start 2024-09-21T05:47:01.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.261+0000 7fec6681f280 1 Processor -- start 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.262+0000 7fec6681f280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 0x555680bfcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedb200 0x555680bfd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedad80 0x555680bfc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555680c0e000 con 0x55567fedd600 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555680c0e180 con 0x55567fedb200 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555680c0e300 con 0x55567fedad80 2024-09-21T05:47:01.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 0x555680bfcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 0x555680bfcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46870/0 (socket says 172.21.15.22:46870) 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 -- 172.21.15.22:0/2664469785 learned_addr learned my addr 172.21.15.22:0/2664469785 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec63b53640 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedb200 0x555680bfd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64b55640 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedad80 0x555680bfc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 -- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedad80 msgr2=0x555680bfc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedad80 0x555680bfc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 -- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedb200 msgr2=0x555680bfd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedb200 0x555680bfd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 -- 172.21.15.22:0/2664469785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567fe64780 con 0x55567fedd600 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec64354640 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 0x555680bfcb00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x555680c59a10 tx=0x55567ff2b1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51583af8a9af8f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec61b4f640 1 -- 172.21.15.22:0/2664469785 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55567be1cb60 con 0x55567fedd600 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec61b4f640 1 -- 172.21.15.22:0/2664469785 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55567be22380 con 0x55567fedd600 2024-09-21T05:47:01.267 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec61b4f640 1 -- 172.21.15.22:0/2664469785 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55567be1cea0 con 0x55567fedd600 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567fe64000 con 0x55567fedd600 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55567fe65a40 con 0x55567fedd600 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.263+0000 7fec6681f280 4 waiting for MDS map... 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.264+0000 7fec61b4f640 1 -- 172.21.15.22:0/2664469785 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x555680d10600 con 0x55567fedd600 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.264+0000 7fec61b4f640 1 -- 172.21.15.22:0/2664469785 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 85) ==== 2893+0+0 (secure 0 0 0) 0x55567fe81180 con 0x55567fedd600 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.264+0000 7fec6681f280 4 Got MDS map 85 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.264+0000 7fec6681f280 10 main: JournalTool::main 2024-09-21T05:47:01.268 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.264+0000 7fec6681f280 4 main: JournalTool: connecting to RADOS... 2024-09-21T05:47:01.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.265+0000 7fec6681f280 1 Processor -- start 2024-09-21T05:47:01.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.265+0000 7fec6681f280 1 -- start start 2024-09-21T05:47:01.270 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec6681f280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec6681f280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec6681f280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec6681f280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555680c0e480 con 0x55567feda000 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec6681f280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555680c0e600 con 0x55567feda480 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec6681f280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555680c0e780 con 0x55567fedc880 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46876/0 (socket says 172.21.15.22:46876) 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 -- 172.21.15.22:0/4000000257 learned_addr learned my addr 172.21.15.22:0/4000000257 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec63b53640 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64b55640 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 -- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedc880 msgr2=0x555680bfe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.271 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 -- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567feda480 msgr2=0x555680bfd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 -- 172.21.15.22:0/4000000257 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567fe65860 con 0x55567feda000 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64b55640 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec64354640 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 0x555680bfdb80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x555680d26de0 tx=0x55567ff2a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d60cc296698fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.266+0000 7fec5fb4b640 1 -- 172.21.15.22:0/4000000257 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55567be1d1e0 con 0x55567feda000 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec5fb4b640 1 -- 172.21.15.22:0/4000000257 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55567be22fc0 con 0x55567feda000 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec5fb4b640 1 -- 172.21.15.22:0/4000000257 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55567be1cb60 con 0x55567feda000 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 -- 172.21.15.22:0/4000000257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 msgr2=0x555680bfdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 0x555680bfdb80 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x555680d26de0 tx=0x55567ff2a960 comp rx=0 tx=0).stop 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 -- 172.21.15.22:0/4000000257 shutdown_connections 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 --2- 172.21.15.22:0/4000000257 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 -- 172.21.15.22:0/4000000257 >> 172.21.15.22:0/4000000257 conn(0x55567feda900 msgr2=0x55567ff4f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 -- 172.21.15.22:0/4000000257 shutdown_connections 2024-09-21T05:47:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.267+0000 7fec6681f280 1 -- 172.21.15.22:0/4000000257 wait complete. 2024-09-21T05:47:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.268+0000 7fec6681f280 1 Processor -- start 2024-09-21T05:47:01.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.268+0000 7fec6681f280 1 -- start start 2024-09-21T05:47:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec6681f280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec6681f280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec6681f280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec6681f280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x555680c0fe00 con 0x55567feda480 2024-09-21T05:47:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec6681f280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x555680c0fc80 con 0x55567fedc880 2024-09-21T05:47:01.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec6681f280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x555680c0fb00 con 0x55567feda000 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec64354640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec64354640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46888/0 (socket says 172.21.15.22:46888) 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.269+0000 7fec64354640 1 -- 172.21.15.22:0/1463431634 learned_addr learned my addr 172.21.15.22:0/1463431634 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec63b53640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64b55640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64354640 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567feda000 msgr2=0x555680bfdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64354640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567feda000 0x555680bfdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64354640 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedc880 msgr2=0x555680bfe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64354640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64354640 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55567fe64780 con 0x55567feda480 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec63b53640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedc880 0x555680bfe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec64354640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 0x555680bfd600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x555680d84060 tx=0x555680d820c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec5e348640 1 -- 172.21.15.22:0/1463431634 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55567be1cea0 con 0x55567feda480 2024-09-21T05:47:01.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec5e348640 1 -- 172.21.15.22:0/1463431634 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55567be22e00 con 0x55567feda480 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec5e348640 1 -- 172.21.15.22:0/1463431634 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55567be1d040 con 0x55567feda480 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55567fe64000 con 0x55567feda480 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567fe65680 con 0x55567feda480 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.270+0000 7fec6681f280 4 main: JournalTool: resolving pool 14 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec5e348640 1 -- 172.21.15.22:0/1463431634 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x555680d7cc00 con 0x55567feda480 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec5e348640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55567fedc400 0x555680bfe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec5e348640 1 -- 172.21.15.22:0/1463431634 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x555680d86600 con 0x55567feda480 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec6681f280 4 main: JournalTool: creating IoCtx.. 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec6681f280 4 main: Executing for rank 0 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec6681f280 4 Filtering by inode '1099511627778' 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55567fedcd00 0x555680bfec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x555680238000 con 0x55567fedcd00 2024-09-21T05:47:01.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec64b55640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55567fedc400 0x555680bfe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.272+0000 7fec63b53640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55567fedcd00 0x555680bfec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec64b55640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55567fedc400 0x555680bfe680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55567ff2c810 tx=0x555680d82300 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec63b53640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55567fedcd00 0x555680bfec00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec63b53640 1 -- 172.21.15.22:0/1463431634 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55567ff6a000 con 0x55567fedcd00 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec6681f280 4 Pointer 400.00000000 is readable 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec6681f280 4 JournalScanner::scan: reading header object '200.00000000' 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55567fedbb00 0x555680bff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x555680238400 con 0x55567fedbb00 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.273+0000 7fec64b55640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55567fedbb00 0x555680bff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.274+0000 7fec64b55640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55567fedbb00 0x555680bff180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.274+0000 7fec64b55640 1 -- 172.21.15.22:0/1463431634 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55567ff6a500 con 0x55567fedbb00 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.274+0000 7fec6681f280 10 Header 0x400000 0x407d21 0x40809d 2024-09-21T05:47:01.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.274+0000 7fec6681f280 10 Starting journal scan from offset 0x407d21 2024-09-21T05:47:01.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.274+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32033~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x555680239400 con 0x55567fedbb00 2024-09-21T05:47:01.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec64b55640 1 -- 172.21.15.22:0/1463431634 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(3 200.00000001 [read 32033~892 out=892b] v0'0 uv27 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x55567ff6a500 con 0x55567fedbb00 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 10 Parsing data, 0x37c bytes available 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 10 Attempting decode at 0x407d21 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 10 Consumed 0x37c bytes 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 10 Valid entry at 0x407d21 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55567fedb680 0x555680bff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) -- 0x555680239800 con 0x55567fedb680 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec64354640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55567fedb680 0x555680bff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec64354640 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55567fedb680 0x555680bff700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec64354640 1 -- 172.21.15.22:0/1463431634 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55567ff6a780 con 0x55567fedb680 2024-09-21T05:47:01.282 INFO:teuthology.orchestra.run.smithi022.stdout:Events by type: 2024-09-21T05:47:01.282 INFO:teuthology.orchestra.run.smithi022.stdout:Errors: 0 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 Reached end of journal objects 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 Scanned objects, 0 missing, 1 valid 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 Events scanned, 0 gaps 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 Found 1 valid events 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 Selected 0 events events for processing 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 4 0 events 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55567fedb680 msgr2=0x555680bff700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x55567fedb680 0x555680bff700 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55567fedbb00 msgr2=0x555680bff180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x55567fedbb00 0x555680bff180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55567fedcd00 msgr2=0x555680bfec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x55567fedcd00 0x555680bfec00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec63b53640 1 -- 172.21.15.22:0/1463431634 reap_dead start 2024-09-21T05:47:01.284 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55567fedc400 msgr2=0x555680bfe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55567fedc400 0x555680bfe680 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55567ff2c810 tx=0x555680d82300 comp rx=0 tx=0).stop 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 msgr2=0x555680bfd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.275+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 0x555680bfd600 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x555680d84060 tx=0x555680d820c0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 shutdown_connections 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567feda480 0x555680bfd600 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 --2- 172.21.15.22:0/1463431634 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55567fedc400 0x555680bfe680 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 >> 172.21.15.22:0/1463431634 conn(0x55567feda900 msgr2=0x55567ff4f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 shutdown_connections 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/1463431634 wait complete. 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 msgr2=0x555680bfcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 0x555680bfcb00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x555680c59a10 tx=0x55567ff2b1a0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 shutdown_connections 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55567fedad80 0x555680bfc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55567fedb200 0x555680bfd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 --2- 172.21.15.22:0/2664469785 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55567fedd600 0x555680bfcb00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 >> 172.21.15.22:0/2664469785 conn(0x55567fedbf80 msgr2=0x55567ff4f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 shutdown_connections 2024-09-21T05:47:01.286 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.276+0000 7fec6681f280 1 -- 172.21.15.22:0/2664469785 wait complete. 2024-09-21T05:47:01.286 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.079160, result: Events by type: Errors: 0 2024-09-21T05:47:01.286 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-21T05:47:01.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.340+0000 7f980bbdcf40 1 Processor -- start 2024-09-21T05:47:01.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.340+0000 7f980bbdcf40 1 -- start start 2024-09-21T05:47:01.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f980bbdcf40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab626c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f980bbdcf40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 0x5591ab623f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f980bbdcf40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab621790 0x5591ab625960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f980bbdcf40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591ab4a3d60 con 0x5591ab59b220 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f980bbdcf40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591ab502430 con 0x5591ab621790 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f980bbdcf40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591ab59b630 con 0x5591ab551d50 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f9804e95640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 0x5591ab623f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f9804e95640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 0x5591ab623f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46908/0 (socket says 172.21.15.22:46908) 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.341+0000 7f9804e95640 1 -- 172.21.15.22:0/1891518166 learned_addr learned my addr 172.21.15.22:0/1891518166 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9805696640 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab626c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9804e95640 1 -- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 msgr2=0x5591ab626c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9805e97640 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab621790 0x5591ab625960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9804e95640 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab626c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9804e95640 1 -- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab621790 msgr2=0x5591ab625960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9804e95640 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab621790 0x5591ab625960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9804e95640 1 -- 172.21.15.22:0/1891518166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ab6271e0 con 0x5591ab59b220 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f9804e95640 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 0x5591ab623f00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c580 tx=0x7f97fc00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55125577790a38e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f97f77fe640 1 -- 172.21.15.22:0/1891518166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00a9e0 con 0x5591ab59b220 2024-09-21T05:47:01.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f97f77fe640 1 -- 172.21.15.22:0/1891518166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97fc00ab80 con 0x5591ab59b220 2024-09-21T05:47:01.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.342+0000 7f97f77fe640 1 -- 172.21.15.22:0/1891518166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc008420 con 0x5591ab59b220 2024-09-21T05:47:01.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1891518166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 msgr2=0x5591ab623f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 0x5591ab623f00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00c580 tx=0x7f97fc00c980 comp rx=0 tx=0).stop 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1891518166 shutdown_connections 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab621790 0x5591ab625960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab59b220 0x5591ab623f00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1891518166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab626c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1891518166 >> 172.21.15.22:0/1891518166 conn(0x5591ab474ac0 msgr2=0x5591ab59bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1891518166 shutdown_connections 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.343+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1891518166 wait complete. 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.344+0000 7f980bbdcf40 1 Processor -- start 2024-09-21T05:47:01.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.344+0000 7f980bbdcf40 1 -- start start 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f980bbdcf40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab614890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f980bbdcf40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab59b220 0x5591ab612a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f980bbdcf40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab621790 0x5591ab617b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f980bbdcf40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591ab627480 con 0x5591ab621790 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f980bbdcf40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591ab614350 con 0x5591ab59b220 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f980bbdcf40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591ab627860 con 0x5591ab551d50 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f9805e97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab621790 0x5591ab617b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f9805e97640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab621790 0x5591ab617b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46912/0 (socket says 172.21.15.22:46912) 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f9805e97640 1 -- 172.21.15.22:0/1984507930 learned_addr learned my addr 172.21.15.22:0/1984507930 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f9804e95640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab59b220 0x5591ab612a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.345+0000 7f9805696640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab614890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f9805e97640 1 -- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 msgr2=0x5591ab614890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f9805e97640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591ab551d50 0x5591ab614890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f9805e97640 1 -- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab59b220 msgr2=0x5591ab612a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f9805e97640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591ab59b220 0x5591ab612a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f9805e97640 1 -- 172.21.15.22:0/1984507930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ab555e30 con 0x5591ab621790 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f9805e97640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab621790 0x5591ab617b80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f98000463f0 tx=0x7f980000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f97f67fc640 1 -- 172.21.15.22:0/1984507930 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800065070 con 0x5591ab621790 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f97f67fc640 1 -- 172.21.15.22:0/1984507930 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f980005f430 con 0x5591ab621790 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f97f67fc640 1 -- 172.21.15.22:0/1984507930 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980005f880 con 0x5591ab621790 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5591ab5533e0 con 0x5591ab621790 2024-09-21T05:47:01.350 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.346+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ab612f70 con 0x5591ab621790 2024-09-21T05:47:01.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.348+0000 7f97f67fc640 1 -- 172.21.15.22:0/1984507930 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9800068070 con 0x5591ab621790 2024-09-21T05:47:01.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.348+0000 7f97f67fc640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f97e80775a0 0x7f97e8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.348+0000 7f97f67fc640 1 -- 172.21.15.22:0/1984507930 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f98000ddbb0 con 0x5591ab621790 2024-09-21T05:47:01.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.349+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x5591ab61d710 0x5591ab6200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.349+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) -- 0x5591ab6180c0 con 0x5591ab61d710 2024-09-21T05:47:01.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.349+0000 7f9805696640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f97e80775a0 0x7f97e8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.349+0000 7f9804e95640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x5591ab61d710 0x5591ab6200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.349+0000 7f9804e95640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x5591ab61d710 0x5591ab6200d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.349+0000 7f9805696640 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f97e80775a0 0x7f97e8079ab0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f97f8000c00 tx=0x7f97f8007810 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f9804e95640 1 -- 172.21.15.22:0/1984507930 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v98'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f97fc00aa60 con 0x5591ab61d710 2024-09-21T05:47:01.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x5591ab61d710 msgr2=0x5591ab6200d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x5591ab61d710 0x5591ab6200d0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f97e80775a0 msgr2=0x7f97e8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f97e80775a0 0x7f97e8079ab0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f97f8000c00 tx=0x7f97f8007810 comp rx=0 tx=0).stop 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab621790 msgr2=0x5591ab617b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 --2- 172.21.15.22:0/1984507930 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591ab621790 0x5591ab617b80 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f98000463f0 tx=0x7f980000a5c0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f9805e97640 1 -- 172.21.15.22:0/1984507930 reap_dead start 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 shutdown_connections 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 >> 172.21.15.22:0/1984507930 conn(0x5591ab474ac0 msgr2=0x5591ab621ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 shutdown_connections 2024-09-21T05:47:01.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.353+0000 7f980bbdcf40 1 -- 172.21.15.22:0/1984507930 wait complete. 2024-09-21T05:47:01.358 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.515+0000 7f79611a8640 1 -- 172.21.15.22:0/1384673871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0b4a50 msgr2=0x7f795c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.515+0000 7f79611a8640 1 --2- 172.21.15.22:0/1384673871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0b4a50 0x7f795c0b4e60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f795c08d4d0 tx=0x7f795402aac0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.516+0000 7f79611a8640 1 -- 172.21.15.22:0/1384673871 shutdown_connections 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.516+0000 7f79611a8640 1 --2- 172.21.15.22:0/1384673871 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0b4a50 0x7f795c0b4e60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.516+0000 7f79611a8640 1 --2- 172.21.15.22:0/1384673871 >> v2:172.21.15.134:3300/0 conn(0x7f795c0d2430 0x7f795c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.516+0000 7f79611a8640 1 -- 172.21.15.22:0/1384673871 >> 172.21.15.22:0/1384673871 conn(0x7f795c09d880 msgr2=0x7f795c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:01.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.516+0000 7f79611a8640 1 -- 172.21.15.22:0/1384673871 shutdown_connections 2024-09-21T05:47:01.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.516+0000 7f79611a8640 1 -- 172.21.15.22:0/1384673871 wait complete. 2024-09-21T05:47:01.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.518+0000 7f79611a8640 1 Processor -- start 2024-09-21T05:47:01.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.518+0000 7f79611a8640 1 -- start start 2024-09-21T05:47:01.522 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f79611a8640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f795c0b4a50 0x7f795c0ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f79611a8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 0x7f795c1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f79611a8640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f795c0d2430 0x7f795c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f79611a8640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f795c0b53a0 con 0x7f795c0d0c70 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f79611a8640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f795c1624a0 con 0x7f795c0d2430 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f79611a8640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f795c0b5520 con 0x7f795c0b4a50 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f795a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 0x7f795c1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f795a59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 0x7f795c1509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46918/0 (socket says 172.21.15.22:46918) 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f795a59c640 1 -- 172.21.15.22:0/1929228333 learned_addr learned my addr 172.21.15.22:0/1929228333 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f795b59e640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f795c0d2430 0x7f795c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.519+0000 7f795ad9d640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f795c0b4a50 0x7f795c0ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795a59c640 1 -- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f795c0b4a50 msgr2=0x7f795c0ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795a59c640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f795c0b4a50 0x7f795c0ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795a59c640 1 -- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f795c0d2430 msgr2=0x7f795c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795a59c640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f795c0d2430 0x7f795c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795a59c640 1 -- 172.21.15.22:0/1929228333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c1f9010 con 0x7f795c0d0c70 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795b59e640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f795c0d2430 0x7f795c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f795a59c640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 0x7f795c1509d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f795000c3a0 tx=0x7f795000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950017070 con 0x7f795c0d0c70 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79500053c0 con 0x7f795c0d0c70 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c1f92f0 con 0x7f795c0d0c70 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950003850 con 0x7f795c0d0c70 2024-09-21T05:47:01.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.520+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c1f98b0 con 0x7f795c0d0c70 2024-09-21T05:47:01.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.521+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c0b59d0 con 0x7f795c0d0c70 2024-09-21T05:47:01.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.522+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f795001a070 con 0x7f795c0d0c70 2024-09-21T05:47:01.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.522+0000 7f79437fe640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7948078b90 0x7f794807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:01.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.523+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f795009a070 con 0x7f795c0d0c70 2024-09-21T05:47:01.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.523+0000 7f795ad9d640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7948078b90 0x7f794807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:01.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.523+0000 7f795ad9d640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7948078b90 0x7f794807b0a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f794c009b30 tx=0x7f794c005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:01.529 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.527+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7950061e10 con 0x7f795c0d0c70 2024-09-21T05:47:01.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:01.753+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f795c15e8b0 con 0x7f795c0d0c70 2024-09-21T05:47:02.109 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.107+0000 7f79437fe640 1 -- 172.21.15.22:0/1929228333 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v86) ==== 162+0+0 (secure 0 0 0) 0x7f7950066840 con 0x7f795c0d0c70 2024-09-21T05:47:02.109 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T05:47:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7948078b90 msgr2=0x7f794807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7948078b90 0x7f794807b0a0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f794c009b30 tx=0x7f794c005f90 comp rx=0 tx=0).stop 2024-09-21T05:47:02.111 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 msgr2=0x7f795c1509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 0x7f795c1509d0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f795000c3a0 tx=0x7f795000c860 comp rx=0 tx=0).stop 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 shutdown_connections 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f795c0d2430 0x7f795c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f795c0d0c70 0x7f795c1509d0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7948078b90 0x7f794807b0a0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 --2- 172.21.15.22:0/1929228333 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f795c0b4a50 0x7f795c0ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 >> 172.21.15.22:0/1929228333 conn(0x7f795c09d880 msgr2=0x7f795c0b9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 shutdown_connections 2024-09-21T05:47:02.112 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.109+0000 7f79611a8640 1 -- 172.21.15.22:0/1929228333 wait complete. 2024-09-21T05:47:02.127 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:02.246 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f25c6640 1 -- 172.21.15.22:0/2389132569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d4003a00 msgr2=0x7f54d4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f25c6640 1 --2- 172.21.15.22:0/2389132569 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54d4003a00 0x7f54d4003e70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009b50 tx=0x7f54d4005830 comp rx=0 tx=0).stop 2024-09-21T05:47:02.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f0b63640 1 -- 172.21.15.22:0/2389132569 reap_dead start 2024-09-21T05:47:02.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f25c6640 1 -- 172.21.15.22:0/2389132569 shutdown_connections 2024-09-21T05:47:02.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f25c6640 1 -- 172.21.15.22:0/2389132569 >> 172.21.15.22:0/2389132569 conn(0x7f54ec0710b0 msgr2=0x7f54ec0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:02.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f25c6640 1 -- 172.21.15.22:0/2389132569 shutdown_connections 2024-09-21T05:47:02.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.244+0000 7f54f25c6640 1 -- 172.21.15.22:0/2389132569 wait complete. 2024-09-21T05:47:02.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.246+0000 7f54f25c6640 1 Processor -- start 2024-09-21T05:47:02.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.246+0000 7f54f25c6640 1 -- start start 2024-09-21T05:47:02.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.247+0000 7f54f25c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d4003a00 0x7f54ec15e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.247+0000 7f54f25c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 0x7f54ec157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.247+0000 7f54f25c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ec157f90 0x7f54ec1f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.247+0000 7f54f25c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec0d06d0 con 0x7f54ec150bd0 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.247+0000 7f54f25c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54ec15c990 con 0x7f54d4003a00 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.247+0000 7f54f25c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54ec0d0850 con 0x7f54ec157f90 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 0x7f54ec157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 0x7f54ec157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46936/0 (socket says 172.21.15.22:46936) 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 -- 172.21.15.22:0/594112514 learned_addr learned my addr 172.21.15.22:0/594112514 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54f0b63640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ec157f90 0x7f54ec1f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54ebfff640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d4003a00 0x7f54ec15e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 -- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ec157f90 msgr2=0x7f54ec1f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ec157f90 0x7f54ec1f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 -- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d4003a00 msgr2=0x7f54ec15e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d4003a00 0x7f54ec15e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 -- 172.21.15.22:0/594112514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ec1583a0 con 0x7f54ec150bd0 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54f0b63640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ec157f90 0x7f54ec1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54ebfff640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d4003a00 0x7f54ec15e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54eb7fe640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 0x7f54ec157a50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f54e000e640 tx=0x7f54e000eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0015070 con 0x7f54ec150bd0 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e000ca90 con 0x7f54ec150bd0 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ec15f1b0 con 0x7f54ec150bd0 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0011460 con 0x7f54ec150bd0 2024-09-21T05:47:02.252 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.248+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ec15f770 con 0x7f54ec150bd0 2024-09-21T05:47:02.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.249+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ec15cb10 con 0x7f54ec150bd0 2024-09-21T05:47:02.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.250+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54e0018070 con 0x7f54ec150bd0 2024-09-21T05:47:02.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.251+0000 7f54e8ff9640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54d8078b90 0x7f54d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:02.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.251+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f54e009b310 con 0x7f54ec150bd0 2024-09-21T05:47:02.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.251+0000 7f54ebfff640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54d8078b90 0x7f54d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:02.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.252+0000 7f54ebfff640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54d8078b90 0x7f54d807b0a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54dc035000 tx=0x7f54dc02d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:02.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.255+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54ec1583a0 con 0x7f54ec150bd0 2024-09-21T05:47:02.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.485+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54ec15ef40 con 0x7f54ec150bd0 2024-09-21T05:47:02.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.486+0000 7f54e8ff9640 1 -- 172.21.15.22:0/594112514 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5225 (secure 0 0 0) 0x7f54e00636e0 con 0x7f54ec150bd0 2024-09-21T05:47:02.488 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:02.488 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":87,"btime":"2024-09-21T05:47:02:110884+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:02.110866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7755":{"gid":7755,"name":"c","rank":0,"incarnation":87,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6834/3179586532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3179586532},{"type":"v1","addr":"172.21.15.22:6834","nonce":3179586532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T05:47:02.488 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 87 2024-09-21T05:47:02.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54d8078b90 msgr2=0x7f54d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54d8078b90 0x7f54d807b0a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54dc035000 tx=0x7f54dc02d040 comp rx=0 tx=0).stop 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 msgr2=0x7f54ec157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 0x7f54ec157a50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f54e000e640 tx=0x7f54e000eb00 comp rx=0 tx=0).stop 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 shutdown_connections 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54d8078b90 0x7f54d807b0a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54ec157f90 0x7f54ec1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54ec150bd0 0x7f54ec157a50 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 --2- 172.21.15.22:0/594112514 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54d4003a00 0x7f54ec15e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 >> 172.21.15.22:0/594112514 conn(0x7f54ec0710b0 msgr2=0x7f54ec1514a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 shutdown_connections 2024-09-21T05:47:02.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:02.488+0000 7f54f25c6640 1 -- 172.21.15.22:0/594112514 wait complete. 2024-09-21T05:47:02.905 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:46:17.567401+0000', 'modified': '2024-09-21T05:47:02.110866+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7755}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7755': {'gid': 7755, 'name': 'c', 'rank': 0, 'incarnation': 87, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.22:6834/3179586532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3179586532}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3179586532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:47:02.905 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7755:up:replay 2024-09-21T05:47:03.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.035+0000 7f305ca69640 1 -- 172.21.15.22:0/256595352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 msgr2=0x7f30580d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.035+0000 7f305ca69640 1 --2- 172.21.15.22:0/256595352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580d1a80 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f305808d4e0 tx=0x7f304802a920 comp rx=0 tx=0).stop 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.036+0000 7f305ca69640 1 -- 172.21.15.22:0/256595352 shutdown_connections 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.036+0000 7f305ca69640 1 --2- 172.21.15.22:0/256595352 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30580cf7a0 0x7f30581626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.036+0000 7f305ca69640 1 --2- 172.21.15.22:0/256595352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580d1a80 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.036+0000 7f305ca69640 1 -- 172.21.15.22:0/256595352 >> 172.21.15.22:0/256595352 conn(0x7f305808bfc0 msgr2=0x7f305809d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.036+0000 7f305ca69640 1 -- 172.21.15.22:0/256595352 shutdown_connections 2024-09-21T05:47:04.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.036+0000 7f305ca69640 1 -- 172.21.15.22:0/256595352 wait complete. 2024-09-21T05:47:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.038+0000 7f305ca69640 1 Processor -- start 2024-09-21T05:47:04.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.038+0000 7f305ca69640 1 -- start start 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305ca69640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30580b9b70 0x7f30580cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305ca69640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30580cf7a0 0x7f30580caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305ca69640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305ca69640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3058162da0 con 0x7f30580d2ed0 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305ca69640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3058162c20 con 0x7f30580b9b70 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305ca69640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3058162f20 con 0x7f30580cf7a0 2024-09-21T05:47:04.042 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580b8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46956/0 (socket says 172.21.15.22:46956) 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 -- 172.21.15.22:0/2918648910 learned_addr learned my addr 172.21.15.22:0/2918648910 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f305659c640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30580b9b70 0x7f30580cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3055d9b640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30580cf7a0 0x7f30580caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 -- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30580cf7a0 msgr2=0x7f30580caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30580cf7a0 0x7f30580caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 -- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30580b9b70 msgr2=0x7f30580cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30580b9b70 0x7f30580cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.039+0000 7f3056d9d640 1 -- 172.21.15.22:0/2918648910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30580cb350 con 0x7f30580d2ed0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.040+0000 7f3056d9d640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580b8a90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f305000d5f0 tx=0x7f305000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.040+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050017070 con 0x7f30580d2ed0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.040+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3050005330 con 0x7f30580d2ed0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.040+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30580cb850 con 0x7f30580d2ed0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.040+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500047a0 con 0x7f30580d2ed0 2024-09-21T05:47:04.043 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.040+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30580c8620 con 0x7f30580d2ed0 2024-09-21T05:47:04.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.041+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30580cb5a0 con 0x7f30580d2ed0 2024-09-21T05:47:04.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.042+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3050004940 con 0x7f30580d2ed0 2024-09-21T05:47:04.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.042+0000 7f304effd640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3038078b90 0x7f303807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:04.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.042+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f305009abf0 con 0x7f30580d2ed0 2024-09-21T05:47:04.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.043+0000 7f305659c640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3038078b90 0x7f303807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:04.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.043+0000 7f305659c640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3038078b90 0x7f303807b0a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3048035000 tx=0x7f3048004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:04.049 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.047+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3050062aa0 con 0x7f30580d2ed0 2024-09-21T05:47:04.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.274+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30581634d0 con 0x7f30580d2ed0 2024-09-21T05:47:04.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.274+0000 7f304effd640 1 -- 172.21.15.22:0/2918648910 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5225 (secure 0 0 0) 0x7f30500674d0 con 0x7f30580d2ed0 2024-09-21T05:47:04.276 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:04.277 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":89,"btime":"2024-09-21T05:47:04:116400+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:03.119344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7755":{"gid":7755,"name":"c","rank":0,"incarnation":87,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.22:6834/3179586532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3179586532},{"type":"v1","addr":"172.21.15.22:6834","nonce":3179586532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T05:47:04.277 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 89 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.276+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3038078b90 msgr2=0x7f303807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.276+0000 7f305ca69640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3038078b90 0x7f303807b0a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3048035000 tx=0x7f3048004ab0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.276+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 msgr2=0x7f30580b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580b8a90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f305000d5f0 tx=0x7f305000dab0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 shutdown_connections 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f30580d2ed0 0x7f30580b8a90 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3038078b90 0x7f303807b0a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f30580cf7a0 0x7f30580caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 --2- 172.21.15.22:0/2918648910 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f30580b9b70 0x7f30580cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 >> 172.21.15.22:0/2918648910 conn(0x7f305808bfc0 msgr2=0x7f305806e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 shutdown_connections 2024-09-21T05:47:04.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:04.277+0000 7f305ca69640 1 -- 172.21.15.22:0/2918648910 wait complete. 2024-09-21T05:47:04.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:46:17.567401+0000', 'modified': '2024-09-21T05:47:03.119344+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7755}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7755': {'gid': 7755, 'name': 'c', 'rank': 0, 'incarnation': 87, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.22:6834/3179586532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3179586532}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3179586532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:47:04.692 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7755:up:rejoin 2024-09-21T05:47:05.692 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 -- 172.21.15.22:0/3492668802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0b9b70 msgr2=0x7f417c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 --2- 172.21.15.22:0/3492668802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0b9b70 0x7f417c0cf090 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f417c08d4e0 tx=0x7f416402a560 comp rx=0 tx=0).stop 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 -- 172.21.15.22:0/3492668802 shutdown_connections 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 --2- 172.21.15.22:0/3492668802 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f417c0cf5d0 0x7f417c0cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 --2- 172.21.15.22:0/3492668802 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0b9b70 0x7f417c0cf090 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 --2- 172.21.15.22:0/3492668802 >> v2:172.21.15.134:3300/0 conn(0x7f417c0d2ed0 0x7f417c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:05.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.822+0000 7f41812bd640 1 -- 172.21.15.22:0/3492668802 >> 172.21.15.22:0/3492668802 conn(0x7f417c08bfc0 msgr2=0x7f417c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.823+0000 7f41812bd640 1 -- 172.21.15.22:0/3492668802 shutdown_connections 2024-09-21T05:47:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.823+0000 7f41812bd640 1 -- 172.21.15.22:0/3492668802 wait complete. 2024-09-21T05:47:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.824+0000 7f41812bd640 1 Processor -- start 2024-09-21T05:47:05.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.825+0000 7f41812bd640 1 -- start start 2024-09-21T05:47:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f41812bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f417c0b9b70 0x7f417c15f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f41812bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 0x7f417c1f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f41812bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f417c0d2ed0 0x7f417c1f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f41812bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f417c0d45c0 con 0x7f417c0cf5d0 2024-09-21T05:47:05.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f41812bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f417c0d4440 con 0x7f417c0d2ed0 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f41812bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f417c0d4740 con 0x7f417c0b9b70 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 0x7f417c1f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 0x7f417c1f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34722/0 (socket says 172.21.15.22:34722) 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 -- 172.21.15.22:0/261286229 learned_addr learned my addr 172.21.15.22:0/261286229 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417affd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f417c0b9b70 0x7f417c15f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417b7fe640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f417c0d2ed0 0x7f417c1f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 -- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f417c0b9b70 msgr2=0x7f417c15f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f417c0b9b70 0x7f417c15f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 -- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f417c0d2ed0 msgr2=0x7f417c1f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f417c0d2ed0 0x7f417c1f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.826+0000 7f417a7fc640 1 -- 172.21.15.22:0/261286229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417c0c9920 con 0x7f417c0cf5d0 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.827+0000 7f417a7fc640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 0x7f417c1f9450 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4164034040 tx=0x7f41640039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:05.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.827+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416403e030 con 0x7f417c0cf5d0 2024-09-21T05:47:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.827+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41640047a0 con 0x7f417c0cf5d0 2024-09-21T05:47:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.827+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417c0c9c00 con 0x7f417c0cf5d0 2024-09-21T05:47:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.827+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4164035040 con 0x7f417c0cf5d0 2024-09-21T05:47:05.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.827+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f417c0ca1c0 con 0x7f417c0cf5d0 2024-09-21T05:47:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.828+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f417c0cf9e0 con 0x7f417c0cf5d0 2024-09-21T05:47:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.828+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4164041030 con 0x7f417c0cf5d0 2024-09-21T05:47:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.829+0000 7f41737fe640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f415c078b90 0x7f415c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.829+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f41640bb5d0 con 0x7f417c0cf5d0 2024-09-21T05:47:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.829+0000 7f417affd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f415c078b90 0x7f415c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:05.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.830+0000 7f417affd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f415c078b90 0x7f415c07b0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f416c00da90 tx=0x7f416c00d470 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:05.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:05.834+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4164083930 con 0x7f417c0cf5d0 2024-09-21T05:47:06.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.062+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f417c0b47e0 con 0x7f417c0cf5d0 2024-09-21T05:47:06.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.063+0000 7f41737fe640 1 -- 172.21.15.22:0/261286229 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5232 (secure 0 0 0) 0x7f4164083230 con 0x7f417c0cf5d0 2024-09-21T05:47:06.065 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:06.065 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":90,"btime":"2024-09-21T05:47:05:118286+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:05.118274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7755},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7755":{"gid":7755,"name":"c","rank":0,"incarnation":87,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6834/3179586532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3179586532},{"type":"v1","addr":"172.21.15.22:6834","nonce":3179586532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7755,"qdb_cluster":[7755]},"id":7}]} 2024-09-21T05:47:06.065 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 90 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.065+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f415c078b90 msgr2=0x7f415c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.065+0000 7f41812bd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f415c078b90 0x7f415c07b0a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f416c00da90 tx=0x7f416c00d470 comp rx=0 tx=0).stop 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.065+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 msgr2=0x7f417c1f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.065+0000 7f41812bd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 0x7f417c1f9450 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f4164034040 tx=0x7f41640039e0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 shutdown_connections 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f417c0d2ed0 0x7f417c1f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f417c0cf5d0 0x7f417c1f9450 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.068 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f415c078b90 0x7f415c07b0a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 --2- 172.21.15.22:0/261286229 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f417c0b9b70 0x7f417c15f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 >> 172.21.15.22:0/261286229 conn(0x7f417c08bfc0 msgr2=0x7f417c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 shutdown_connections 2024-09-21T05:47:06.069 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.066+0000 7f41812bd640 1 -- 172.21.15.22:0/261286229 wait complete. 2024-09-21T05:47:06.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:46:17.567401+0000', 'modified': '2024-09-21T05:47:05.118274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7755}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7755': {'gid': 7755, 'name': 'c', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.22:6834/3179586532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6832', 'nonce': 3179586532}, {'type': 'v1', 'addr': '172.21.15.22:6834', 'nonce': 3179586532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7755, 'qdb_cluster': [7755]} 2024-09-21T05:47:06.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:47:06.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T05:47:06.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 -- 172.21.15.22:0/1318348250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 msgr2=0x7fb1d0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 --2- 172.21.15.22:0/1318348250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1d0004c10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc0016c0 tx=0x7fb1d0005830 comp rx=0 tx=0).stop 2024-09-21T05:47:06.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 -- 172.21.15.22:0/1318348250 shutdown_connections 2024-09-21T05:47:06.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 --2- 172.21.15.22:0/1318348250 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1d0004c10 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 -- 172.21.15.22:0/1318348250 >> 172.21.15.22:0/1318348250 conn(0x7fb1e809d7f0 msgr2=0x7fb1e80d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 -- 172.21.15.22:0/1318348250 shutdown_connections 2024-09-21T05:47:06.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.599+0000 7fb1efb24640 1 -- 172.21.15.22:0/1318348250 wait complete. 2024-09-21T05:47:06.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.601+0000 7fb1efb24640 1 Processor -- start 2024-09-21T05:47:06.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.601+0000 7fb1efb24640 1 -- start start 2024-09-21T05:47:06.605 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1efb24640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1e815fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1efb24640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e81f6040 0x7fb1e80cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1efb24640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e815d930 0x7fb1e80ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1efb24640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b6120 con 0x7fb1d0004800 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1efb24640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b5fa0 con 0x7fb1e815d930 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1efb24640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80b62a0 con 0x7fb1e81f6040 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1e815fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1e815fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34744/0 (socket says 172.21.15.22:34744) 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3496175761 learned_addr learned my addr 172.21.15.22:0/3496175761 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e81f6040 0x7fb1e80cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.602+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e815d930 0x7fb1e80ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e81f6040 msgr2=0x7fb1e80cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e81f6040 0x7fb1e80cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e815d930 msgr2=0x7fb1e80ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e815d930 0x7fb1e80ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3496175761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e81604a0 con 0x7fb1d0004800 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1e815fca0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8002230 tx=0x7fb1d800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1e67fc640 1 -- 172.21.15.22:0/3496175761 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8015070 con 0x7fb1d0004800 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1e67fc640 1 -- 172.21.15.22:0/3496175761 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1d800cce0 con 0x7fb1d0004800 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1e67fc640 1 -- 172.21.15.22:0/3496175761 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d80116c0 con 0x7fb1d0004800 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e81601e0 con 0x7fb1d0004800 2024-09-21T05:47:06.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.603+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e8161e20 con 0x7fb1d0004800 2024-09-21T05:47:06.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.605+0000 7fb1e67fc640 1 -- 172.21.15.22:0/3496175761 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1d8018070 con 0x7fb1d0004800 2024-09-21T05:47:06.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.606+0000 7fb1e67fc640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb1d4078b90 0x7fb1d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.606+0000 7fb1e67fc640 1 -- 172.21.15.22:0/3496175761 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fb1d809b4c0 con 0x7fb1d0004800 2024-09-21T05:47:06.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.606+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb1d4078b90 0x7fb1d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.608 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.607+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb1d4078b90 0x7fb1d407b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc032000 tx=0x7fb1dc02c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.607+0000 7fb1f6a8e740 1 Processor -- start 2024-09-21T05:47:06.609 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.607+0000 7fb1f6a8e740 1 -- start start 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d422fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56302d36b820 con 0x56302d051270 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56302d35ac50 con 0x56302d0283b0 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56302d35ea60 con 0x56302d030590 2024-09-21T05:47:06.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d422fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d422fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34748/0 (socket says 172.21.15.22:34748) 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed0bf640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3831034844 learned_addr learned my addr 172.21.15.22:0/3831034844 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 msgr2=0x56302d4403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4403e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 msgr2=0x56302d4234e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.608+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4234e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.609+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/3831034844 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302d0bc850 con 0x56302d051270 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.609+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d422fa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8060570 tx=0x7fb1d8062ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df4981f528e852c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.609+0000 7fb1cf7fe640 1 -- 172.21.15.22:0/3831034844 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d801f040 con 0x56302d051270 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.609+0000 7fb1cf7fe640 1 -- 172.21.15.22:0/3831034844 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1d806cbe0 con 0x56302d051270 2024-09-21T05:47:06.612 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.609+0000 7fb1cf7fe640 1 -- 172.21.15.22:0/3831034844 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d809b0d0 con 0x56302d051270 2024-09-21T05:47:06.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3831034844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 msgr2=0x56302d422fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.613 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d422fa0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8060570 tx=0x7fb1d8062ba0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3831034844 shutdown_connections 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3831034844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d422fa0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3831034844 >> 172.21.15.22:0/3831034844 conn(0x56302d0453c0 msgr2=0x56302d3f9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3831034844 shutdown_connections 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.610+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3831034844 wait complete. 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.611+0000 7fb1f6a8e740 1 -- start start 2024-09-21T05:47:06.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.611+0000 7fb1f6a8e740 1 Processor -- start 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d411670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d438ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d421e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56302d0cb620 con 0x56302d0283b0 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56302d3005f0 con 0x56302d051270 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56302d423a20 con 0x56302d030590 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1f6a8e740 10 client.0 _ll_register_callbacks cb 0x56302d3040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d411670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d411670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34752/0 (socket says 172.21.15.22:34752) 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.612+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/1183980119 learned_addr learned my addr 172.21.15.22:0/1183980119 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d438ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d421e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 msgr2=0x56302d438ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d438ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 msgr2=0x56302d421e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d421e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/1183980119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302d1336b0 con 0x56302d0283b0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d438ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d421e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d411670 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8099940 tx=0x7fb1d806a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780ca00ea98fb4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ccff9640 1 -- 172.21.15.22:0/1183980119 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8082060 con 0x56302d0283b0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ccff9640 1 -- 172.21.15.22:0/1183980119 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fb1d8062560 con 0x56302d0283b0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d806cbe0 con 0x56302d0283b0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ccff9640 1 -- 172.21.15.22:0/1183980119 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb1d804e070 con 0x56302d0283b0 2024-09-21T05:47:06.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1f6a8e740 10 client.7824 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x56302d44d9b0 con 0x56302d0283b0 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.613+0000 7fb1ccff9640 1 -- 172.21.15.22:0/1183980119 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1d806cbe0 con 0x56302d0283b0 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1ccff9640 1 -- 172.21.15.22:0/1183980119 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7fb1d805ec50 con 0x56302d0283b0 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1f6a8e740 10 client.7824 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1f6a8e740 10 client.7824 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1f6a8e740 20 client.7824 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d00fe40 0x56302d450490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1f6a8e740 4 client.7824 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56302d36d770 con 0x56302d00fe40 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d00fe40 0x56302d450490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1ccff9640 10 client.7824 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.614+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d00fe40 0x56302d450490 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591382c73b914623 server_cookie=953bc1e8b782acf9 in_seq=0 out_seq=0 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1ccff9640 1 -- 172.21.15.22:0/1183980119 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56302d36d770 con 0x56302d00fe40 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1ccff9640 10 client.7824 handle_command_reply: tid=0 2024-09-21T05:47:06.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 client.7824 shutdown 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 msgr2=0x56302d411670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d411670 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8099940 tx=0x7fb1d806a2f0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 shutdown_connections 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d00fe40 0x56302d450490 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d421e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d438ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1ccff9640 0 client.7824 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/1183980119 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d411670 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 >> 172.21.15.22:0/1183980119 conn(0x56302d0453c0 msgr2=0x56302d43f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 shutdown_connections 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/1183980119 wait complete. 2024-09-21T05:47:06.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.615+0000 7fb1f6a8e740 20 client.7824 trim_cache size 0 max 16384 2024-09-21T05:47:06.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.634+0000 7fb1f6a8e740 1 Processor -- start 2024-09-21T05:47:06.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.634+0000 7fb1f6a8e740 1 -- start start 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d412e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d43c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8201110 con 0x56302d051270 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e81f9120 con 0x56302d0283b0 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e8200ac0 con 0x56302d030590 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1ee0c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d43c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1ee0c1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d43c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34762/0 (socket says 172.21.15.22:34762) 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.635+0000 7fb1ee0c1640 1 -- 172.21.15.22:0/3722567469 learned_addr learned my addr 172.21.15.22:0/3722567469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.640 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d412e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ee0c1640 1 -- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 msgr2=0x56302d412e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d412e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ee0c1640 1 -- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 msgr2=0x56302d4128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ee0c1640 1 -- 172.21.15.22:0/3722567469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302d4362e0 con 0x56302d051270 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d43c840 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009c80 tx=0x7fb1e0014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eb052899959c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ceffd640 1 -- 172.21.15.22:0/3722567469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0009e30 con 0x56302d051270 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ceffd640 1 -- 172.21.15.22:0/3722567469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e0004030 con 0x56302d051270 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.636+0000 7fb1ceffd640 1 -- 172.21.15.22:0/3722567469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0004480 con 0x56302d051270 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3722567469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 msgr2=0x56302d43c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d43c840 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0009c80 tx=0x7fb1e0014840 comp rx=0 tx=0).stop 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3722567469 shutdown_connections 2024-09-21T05:47:06.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d051270 0x56302d43c840 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d412e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/3722567469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d0283b0 0x56302d4128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3722567469 >> 172.21.15.22:0/3722567469 conn(0x56302d0453c0 msgr2=0x56302d3ee920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3722567469 shutdown_connections 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.637+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/3722567469 wait complete. 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.638+0000 7fb1f6a8e740 1 -- start start 2024-09-21T05:47:06.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.638+0000 7fb1f6a8e740 1 Processor -- start 2024-09-21T05:47:06.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d4baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4bddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d4be310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56302d0cb620 con 0x56302d0283b0 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x56302d3005f0 con 0x56302d051270 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56302d44cef0 con 0x56302d030590 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1f6a8e740 10 client.0 _ll_register_callbacks cb 0x7fb1e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d4baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1ed8c0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d4baca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34764/0 (socket says 172.21.15.22:34764) 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.639+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/2781600610 learned_addr learned my addr 172.21.15.22:0/2781600610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4bddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ee0c1640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d4be310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 msgr2=0x56302d4bddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4bddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 msgr2=0x56302d4be310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d4be310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed8c0640 1 -- 172.21.15.22:0/2781600610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302d43f540 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ed8c0640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d4baca0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8063290 tx=0x7fb1d80997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede2577f9638d97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ccff9640 1 -- 172.21.15.22:0/2781600610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d801f040 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ccff9640 1 -- 172.21.15.22:0/2781600610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fb1d8065c90 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e0004030 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ccff9640 1 -- 172.21.15.22:0/2781600610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb1d8065e90 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1f6a8e740 10 client.7830 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x56302d43a5f0 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.640+0000 7fb1ccff9640 1 -- 172.21.15.22:0/2781600610 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e0004030 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1ccff9640 1 -- 172.21.15.22:0/2781600610 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7fb1d80853a0 con 0x56302d0283b0 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 10 client.7830 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 10 client.7830 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 20 client.7830 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:06.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d4366c0 0x56302d436ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 4 client.7830 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "status"}) -- 0x56302d369220 con 0x56302d4366c0 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d4366c0 0x56302d436ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1ccff9640 10 client.7830 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1ed0bf640 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d4366c0 0x56302d436ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cacd9277d16bd74 server_cookie=2ba09681c97075fb in_seq=0 out_seq=0 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1ccff9640 1 -- 172.21.15.22:0/2781600610 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56302d369220 con 0x56302d4366c0 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1ccff9640 10 client.7830 handle_command_reply: tid=0 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.641+0000 7fb1f6a8e740 1 client.7830 shutdown 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 msgr2=0x56302d4baca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d4baca0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8063290 tx=0x7fb1d80997c0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 shutdown_connections 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x56302d4366c0 0x56302d436ad0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x56302d051270 0x56302d4be310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56302d030590 0x56302d4bddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1ccff9640 0 client.7830 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 --2- 172.21.15.22:0/2781600610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56302d0283b0 0x56302d4baca0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 >> 172.21.15.22:0/2781600610 conn(0x56302d0453c0 msgr2=0x56302d439360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 shutdown_connections 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 1 -- 172.21.15.22:0/2781600610 wait complete. 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1f6a8e740 20 client.7830 trim_cache size 0 max 16384 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 7755, 2024-09-21T05:47:06.649 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 4.5266573530000001, 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 90, 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 98, 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 98, 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 5.4614919430000004 2024-09-21T05:47:06.650 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb1d4078b90 msgr2=0x7fb1d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb1d4078b90 0x7fb1d407b0a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc032000 tx=0x7fb1dc02c040 comp rx=0 tx=0).stop 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 msgr2=0x7fb1e815fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1e815fca0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8002230 tx=0x7fb1d800edb0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 shutdown_connections 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb1e815d930 0x7fb1e80ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb1d4078b90 0x7fb1d407b0a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb1e81f6040 0x7fb1e80cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 --2- 172.21.15.22:0/3496175761 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb1d0004800 0x7fb1e815fca0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.642+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 >> 172.21.15.22:0/3496175761 conn(0x7fb1e809d7f0 msgr2=0x7fb1e815e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.643+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 shutdown_connections 2024-09-21T05:47:06.651 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.643+0000 7fb1efb24640 1 -- 172.21.15.22:0/3496175761 wait complete. 2024-09-21T05:47:06.656 DEBUG:teuthology.orchestra.run.smithi022:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-21T05:47:06.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:47:06.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-09-21T05:47:06.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 -- 172.21.15.22:0/3124615621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b38f0 msgr2=0x7f51280b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 --2- 172.21.15.22:0/3124615621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b38f0 0x7f51280b59e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f51280cb860 tx=0x7f511002a560 comp rx=0 tx=0).stop 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 -- 172.21.15.22:0/3124615621 shutdown_connections 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 --2- 172.21.15.22:0/3124615621 >> v2:172.21.15.134:3300/0 conn(0x7f51280b40e0 0x7f51280b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 --2- 172.21.15.22:0/3124615621 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b38f0 0x7f51280b59e0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 -- 172.21.15.22:0/3124615621 >> 172.21.15.22:0/3124615621 conn(0x7f5128074190 msgr2=0x7f51280b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 -- 172.21.15.22:0/3124615621 shutdown_connections 2024-09-21T05:47:06.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.777+0000 7f512f00a640 1 -- 172.21.15.22:0/3124615621 wait complete. 2024-09-21T05:47:06.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.779+0000 7f512f00a640 1 Processor -- start 2024-09-21T05:47:06.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.779+0000 7f512f00a640 1 -- start start 2024-09-21T05:47:06.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.780+0000 7f512f00a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512806cde0 0x7f51281f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.780+0000 7f512f00a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51280b38f0 0x7f512814aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.780+0000 7f512f00a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 0x7f512809c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.780+0000 7f512f00a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51280b3650 con 0x7f51280b40e0 2024-09-21T05:47:06.784 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512f00a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f51280b34d0 con 0x7f51280b38f0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512f00a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51280b4a30 con 0x7f512806cde0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 0x7f512809c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 0x7f512809c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34776/0 (socket says 172.21.15.22:34776) 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 -- 172.21.15.22:0/3714460144 learned_addr learned my addr 172.21.15.22:0/3714460144 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512cda6640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512806cde0 0x7f51281f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f5127fff640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51280b38f0 0x7f512814aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 -- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512806cde0 msgr2=0x7f51281f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512806cde0 0x7f51281f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 -- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51280b38f0 msgr2=0x7f512814aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51280b38f0 0x7f512814aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 -- 172.21.15.22:0/3714460144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512809b790 con 0x7f51280b40e0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f5127fff640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51280b38f0 0x7f512814aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512d5a7640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 0x7f512809c830 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f512000c8f0 tx=0x7f512000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f51257fa640 1 -- 172.21.15.22:0/3714460144 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120007bb0 con 0x7f51280b40e0 2024-09-21T05:47:06.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f51257fa640 1 -- 172.21.15.22:0/3714460144 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f512000ae40 con 0x7f51280b40e0 2024-09-21T05:47:06.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512809ba70 con 0x7f51280b40e0 2024-09-21T05:47:06.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f51257fa640 1 -- 172.21.15.22:0/3714460144 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120005280 con 0x7f51280b40e0 2024-09-21T05:47:06.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.781+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512809c030 con 0x7f51280b40e0 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.783+0000 7f51257fa640 1 -- 172.21.15.22:0/3714460144 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5120005420 con 0x7f51280b40e0 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.784+0000 7f51257fa640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5108078bc0 0x7f510807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.784+0000 7f51257fa640 1 -- 172.21.15.22:0/3714460144 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f512009b1d0 con 0x7f51280b40e0 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.784+0000 7f512cda6640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5108078bc0 0x7f510807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.785+0000 7f5136050740 1 Processor -- start 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.785+0000 7f5136050740 1 -- start start 2024-09-21T05:47:06.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.785+0000 7f512cda6640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5108078bc0 0x7f510807b0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5110033000 tx=0x7f5110004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc182f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc18306be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5136050740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc18307120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5136050740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cc18252120 con 0x55cc17f16730 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cc18241550 con 0x55cc17f0e550 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55cc18245360 con 0x55cc17f37410 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f512d5a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc18307120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f512d5a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc18307120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34788/0 (socket says 172.21.15.22:34788) 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f512d5a7640 1 -- 172.21.15.22:0/641612508 learned_addr learned my addr 172.21.15.22:0/641612508 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f512cda6640 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc182f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.786+0000 7f5127fff640 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc18306be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f512d5a7640 1 -- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 msgr2=0x55cc182f86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f512d5a7640 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc182f86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f512d5a7640 1 -- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 msgr2=0x55cc18306be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f512d5a7640 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc18306be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f512d5a7640 1 -- 172.21.15.22:0/641612508 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc17f33470 con 0x55cc17f16730 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f512d5a7640 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc18307120 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5120099610 tx=0x7f5120062770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7487423065f9b34b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f511e7fc640 1 -- 172.21.15.22:0/641612508 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512001d040 con 0x55cc17f16730 2024-09-21T05:47:06.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f511e7fc640 1 -- 172.21.15.22:0/641612508 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5120062f40 con 0x55cc17f16730 2024-09-21T05:47:06.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.787+0000 7f511e7fc640 1 -- 172.21.15.22:0/641612508 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120089220 con 0x55cc17f16730 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 -- 172.21.15.22:0/641612508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 msgr2=0x55cc18307120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc18307120 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f5120099610 tx=0x7f5120062770 comp rx=0 tx=0).stop 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 -- 172.21.15.22:0/641612508 shutdown_connections 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc18307120 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc18306be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 --2- 172.21.15.22:0/641612508 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc182f86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 -- 172.21.15.22:0/641612508 >> 172.21.15.22:0/641612508 conn(0x55cc17f2b560 msgr2=0x55cc182dffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 -- 172.21.15.22:0/641612508 shutdown_connections 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.788+0000 7f5136050740 1 -- 172.21.15.22:0/641612508 wait complete. 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.789+0000 7f5136050740 1 -- start start 2024-09-21T05:47:06.792 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.789+0000 7f5136050740 1 Processor -- start 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc1831fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc1833bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc1833c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55cc17fb17c0 con 0x55cc17f16730 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cc181e6890 con 0x55cc17f0e550 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55cc1831e170 con 0x55cc17f37410 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.790+0000 7f5136050740 10 client.0 _ll_register_callbacks cb 0x55cc1823ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:06.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc1833bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc1833bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34804/0 (socket says 172.21.15.22:34804) 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 -- 172.21.15.22:0/2787587112 learned_addr learned my addr 172.21.15.22:0/2787587112 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f512d5a7640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc1833c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f512cda6640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc1831fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 -- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 msgr2=0x55cc1833c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc1833c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 -- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 msgr2=0x55cc1831fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc1831fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 -- 172.21.15.22:0/2787587112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc180177e0 con 0x55cc17f16730 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5127fff640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc1833bac0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f511800fb50 tx=0x7f511800d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4595d1ec47bbc031 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f51077fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f51077fe640 1 -- 172.21.15.22:0/2787587112 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118010620 con 0x55cc17f16730 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f51077fe640 1 -- 172.21.15.22:0/2787587112 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f51180107c0 con 0x55cc17f16730 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5120062f40 con 0x55cc17f16730 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f51077fe640 1 -- 172.21.15.22:0/2787587112 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5118022050 con 0x55cc17f16730 2024-09-21T05:47:06.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5136050740 10 client.7842 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.791+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55cc18077f70 con 0x55cc17f16730 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f51077fe640 1 -- 172.21.15.22:0/2787587112 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5120062f40 con 0x55cc17f16730 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f51077fe640 1 -- 172.21.15.22:0/2787587112 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7f5118025020 con 0x55cc17f16730 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f5136050740 10 client.7842 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f5136050740 10 client.7842 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f5136050740 20 client.7842 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f5136050740 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ef61f0 0x55cc1831cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f5136050740 4 client.7842 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc18254070 con 0x55cc17ef61f0 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f512cda6640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ef61f0 0x55cc1831cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f51077fe640 10 client.7842 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.792+0000 7f512cda6640 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ef61f0 0x55cc1831cf10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8f0b692a31ad83 server_cookie=c87fcdfb5b5c59b4 in_seq=0 out_seq=0 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.793+0000 7f51077fe640 1 -- 172.21.15.22:0/2787587112 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cc18254070 con 0x55cc17ef61f0 2024-09-21T05:47:06.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.793+0000 7f51077fe640 10 client.7842 handle_command_reply: tid=0 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.793+0000 7f5136050740 1 client.7842 shutdown 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.793+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 msgr2=0x55cc1833bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.793+0000 7f5136050740 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc1833bac0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f511800fb50 tx=0x7f511800d700 comp rx=0 tx=0).stop 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 shutdown_connections 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ef61f0 0x55cc1831cf10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f51077fe640 0 client.7842 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17f37410 0x55cc1833c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f16730 0x55cc1833bac0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 --2- 172.21.15.22:0/2787587112 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f0e550 0x55cc1831fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 >> 172.21.15.22:0/2787587112 conn(0x55cc17f2b560 msgr2=0x55cc182f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 shutdown_connections 2024-09-21T05:47:06.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 1 -- 172.21.15.22:0/2787587112 wait complete. 2024-09-21T05:47:06.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.794+0000 7f5136050740 20 client.7842 trim_cache size 0 max 16384 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.800+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3750943334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ccc004800 msgr2=0x7f4ccc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.800+0000 7f4ce79ed640 1 --2- 172.21.15.22:0/3750943334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ccc004800 0x7f4ccc004c10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00016c0 tx=0x7f4ccc005830 comp rx=0 tx=0).stop 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.800+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3750943334 shutdown_connections 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.800+0000 7f4ce79ed640 1 --2- 172.21.15.22:0/3750943334 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ccc004800 0x7f4ccc004c10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.800+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3750943334 >> 172.21.15.22:0/3750943334 conn(0x7f4ce00cdd20 msgr2=0x7f4ce00ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.801+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3750943334 shutdown_connections 2024-09-21T05:47:06.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.801+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3750943334 wait complete. 2024-09-21T05:47:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.802+0000 7f4ce79ed640 1 Processor -- start 2024-09-21T05:47:06.804 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.802+0000 7f4ce79ed640 1 -- start start 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce79ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc004800 0x7f4ce01f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce79ed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ce014ae30 0x7f4ce015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce79ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015a770 0x7f4ce01f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce79ed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00722f0 con 0x7f4ce015a770 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce79ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce00cf390 con 0x7f4ccc004800 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce79ed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce0072470 con 0x7f4ce014ae30 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015a770 0x7f4ce01f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015a770 0x7f4ce01f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34828/0 (socket says 172.21.15.22:34828) 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 -- 172.21.15.22:0/3959761464 learned_addr learned my addr 172.21.15.22:0/3959761464 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce4f88640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ce014ae30 0x7f4ce015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5789640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc004800 0x7f4ce01f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 -- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ce014ae30 msgr2=0x7f4ce015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4ce014ae30 0x7f4ce015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 -- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc004800 msgr2=0x7f4ce01f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4ccc004800 0x7f4ce01f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 -- 172.21.15.22:0/3959761464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce014b6d0 con 0x7f4ce015a770 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4ce5f8a640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4ce015a770 0x7f4ce01f6580 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8005fd0 tx=0x7f4cd800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.804+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8003d80 con 0x7f4ce015a770 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.805+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cd8004b20 con 0x7f4ce015a770 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.805+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3959761464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce015ab80 con 0x7f4ce015a770 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.805+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8004030 con 0x7f4ce015a770 2024-09-21T05:47:06.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.805+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3959761464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce0148190 con 0x7f4ce015a770 2024-09-21T05:47:06.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.806+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3959761464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4ce00715b0 con 0x7f4ce015a770 2024-09-21T05:47:06.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.807+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4cd80041d0 con 0x7f4ce015a770 2024-09-21T05:47:06.809 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.807+0000 7f4cd5ffb640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.807+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f4cd809b920 con 0x7f4ce015a770 2024-09-21T05:47:06.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.808+0000 7f4ce5789640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.808+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7f4cd8020030 con 0x7f4ce015a770 2024-09-21T05:47:06.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.808+0000 7f4ce79ed640 1 -- 172.21.15.22:0/3959761464 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f4ce00b2010 con 0x7f4ce015a770 2024-09-21T05:47:06.810 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.808+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 7 ==== log(2 entries from seq 890 at 2024-09-21T05:47:05.118358+0000) ==== 389+0+0 (secure 0 0 0) 0x7f4cd8099f00 con 0x7f4ce015a770 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: cluster: 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: id: cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: health: HEALTH_OK 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: services: 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: mgr: x(active, since 8m), standbys: y 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: mds: 1/1 daemons up, 3 standby 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: osd: 8 osds: 8 up (since 7m), 8 in (since 8m) 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: data: 2024-09-21T05:47:06.811 INFO:teuthology.orchestra.run.smithi022.stdout: volumes: 1/1 healthy 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: pools: 3 pools, 97 pgs 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: objects: 30 objects, 19 MiB 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: usage: 306 MiB used, 720 GiB / 720 GiB avail 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: pgs: 97 active+clean 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: io: 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: client: 6.6 KiB/s rd, 102 B/s wr, 6 op/s rd, 0 op/s wr 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:05.118358+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:05.118395+0000 mon.a [INF] Cluster is now healthy 2024-09-21T05:47:06.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.808+0000 7f4ce5789640 1 --2- 172.21.15.22:0/3959761464 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4cc4078bc0 0x7f4cc407b0d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0052910 tx=0x7f4cdc0067a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.815+0000 7f5136050740 1 Processor -- start 2024-09-21T05:47:06.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.816+0000 7f5136050740 1 -- start start 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc182d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5136050740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc1834c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc1833b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5136050740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51281fe210 con 0x55cc17f0e550 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f512814cb50 con 0x55cc17f16730 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f51280d0300 con 0x55cc17ef61f0 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc1834c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc1834c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34832/0 (socket says 172.21.15.22:34832) 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 -- 172.21.15.22:0/2954615902 learned_addr learned my addr 172.21.15.22:0/2954615902 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f512cda6640 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc182d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f512d5a7640 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc1833b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 -- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 msgr2=0x55cc182d5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc182d5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 -- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 msgr2=0x55cc1833b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc1833b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 -- 172.21.15.22:0/2954615902 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc17f80720 con 0x55cc17f0e550 2024-09-21T05:47:06.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.817+0000 7f5127fff640 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc1834c2d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5118002230 tx=0x7f51180049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2ad4a6cf33ae4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.818+0000 7f511dffb640 1 -- 172.21.15.22:0/2954615902 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511802c070 con 0x55cc17f0e550 2024-09-21T05:47:06.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.818+0000 7f511dffb640 1 -- 172.21.15.22:0/2954615902 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511800dc00 con 0x55cc17f0e550 2024-09-21T05:47:06.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.818+0000 7f511dffb640 1 -- 172.21.15.22:0/2954615902 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5118010420 con 0x55cc17f0e550 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.818+0000 7f5136050740 1 -- 172.21.15.22:0/2954615902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 msgr2=0x55cc1834c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.818+0000 7f5136050740 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc1834c2d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f5118002230 tx=0x7f51180049c0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 -- 172.21.15.22:0/2954615902 shutdown_connections 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc1833b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc1834c2d0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 --2- 172.21.15.22:0/2954615902 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc182d5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 -- 172.21.15.22:0/2954615902 >> 172.21.15.22:0/2954615902 conn(0x55cc17f2b560 msgr2=0x55cc1831e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 -- 172.21.15.22:0/2954615902 shutdown_connections 2024-09-21T05:47:06.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 -- 172.21.15.22:0/2954615902 wait complete. 2024-09-21T05:47:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.819+0000 7f5136050740 1 -- start start 2024-09-21T05:47:06.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.820+0000 7f5136050740 1 Processor -- start 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc1839beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc183091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc18309720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f51280d0950 con 0x55cc17f0e550 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55cc18306aa0 con 0x55cc17f16730 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55cc1831b160 con 0x55cc17ef61f0 2024-09-21T05:47:06.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5136050740 10 client.0 _ll_register_callbacks cb 0x7f5118007aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc183091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc183091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34842/0 (socket says 172.21.15.22:34842) 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 -- 172.21.15.22:0/3325241294 learned_addr learned my addr 172.21.15.22:0/3325241294 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f512d5a7640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc18309720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 -- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 msgr2=0x55cc1839beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc1839beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 -- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 msgr2=0x55cc18309720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc18309720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f5127fff640 1 -- 172.21.15.22:0/3325241294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1831af50 con 0x55cc17f0e550 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.821+0000 7f512d5a7640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc18309720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5127fff640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc183091e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5118007a70 tx=0x7f511800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a9ca98b1c34ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f51077fe640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f51077fe640 1 -- 172.21.15.22:0/3325241294 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511802c070 con 0x55cc17f0e550 2024-09-21T05:47:06.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f51077fe640 1 -- 172.21.15.22:0/3325241294 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f5118010970 con 0x55cc17f0e550 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511800dc00 con 0x55cc17f0e550 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f51077fe640 1 -- 172.21.15.22:0/3325241294 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5118041050 con 0x55cc17f0e550 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 10 client.7854 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55cc1839f100 con 0x55cc17f0e550 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f51077fe640 1 -- 172.21.15.22:0/3325241294 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f511800dc00 con 0x55cc17f0e550 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f51077fe640 1 -- 172.21.15.22:0/3325241294 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7f5118044020 con 0x55cc17f0e550 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 10 client.7854 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 10 client.7854 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 20 client.7854 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ecb630 0x55cc183264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 4 client.7854 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55cc1824fb20 con 0x55cc17ecb630 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.822+0000 7f512cda6640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ecb630 0x55cc183264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.823+0000 7f51077fe640 10 client.7854 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.823+0000 7f512cda6640 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ecb630 0x55cc183264a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f412cec8312c49 server_cookie=2c3d2df81835e25d in_seq=0 out_seq=0 2024-09-21T05:47:06.828 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-09-21T05:47:06.828 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-09-21T05:47:06.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f51077fe640 1 -- 172.21.15.22:0/3325241294 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cc1824fb20 con 0x55cc17ecb630 2024-09-21T05:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f51077fe640 10 client.7854 handle_command_reply: tid=0 2024-09-21T05:47:06.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 client.7854 shutdown 2024-09-21T05:47:06.829 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-09-21T05:47:06.829 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 msgr2=0x55cc183091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc183091e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5118007a70 tx=0x7f511800f7f0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 shutdown_connections 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55cc17ecb630 0x55cc183264a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55cc17f16730 0x55cc18309720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55cc17f0e550 0x55cc183091e0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f51077fe640 0 client.7854 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 --2- 172.21.15.22:0/3325241294 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55cc17ef61f0 0x55cc1839beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 >> 172.21.15.22:0/3325241294 conn(0x55cc17f2b560 msgr2=0x55cc182e0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.825+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 shutdown_connections 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f5136050740 1 -- 172.21.15.22:0/3325241294 wait complete. 2024-09-21T05:47:06.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f5136050740 20 client.7854 trim_cache size 0 max 16384 2024-09-21T05:47:06.831 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0, 2024-09-21T05:47:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_tag": "e63d795c-851d-415f-8ea3-1e40fd0e3fe9", 2024-09-21T05:47:06.831 INFO:teuthology.orchestra.run.smithi022.stdout: "mode": "asynchronous" 2024-09-21T05:47:06.831 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5108078bc0 msgr2=0x7f510807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5108078bc0 0x7f510807b0d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5110033000 tx=0x7f5110004730 comp rx=0 tx=0).stop 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 msgr2=0x7f512809c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 0x7f512809c830 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f512000c8f0 tx=0x7f512000cdb0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 shutdown_connections 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51280b40e0 0x7f512809c830 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51280b38f0 0x7f512814aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5108078bc0 0x7f510807b0d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 --2- 172.21.15.22:0/3714460144 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f512806cde0 0x7f51281f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.826+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 >> 172.21.15.22:0/3714460144 conn(0x7f5128074190 msgr2=0x7f51280cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.827+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 shutdown_connections 2024-09-21T05:47:06.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.827+0000 7f512f00a640 1 -- 172.21.15.22:0/3714460144 wait complete. 2024-09-21T05:47:06.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:47:06.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T05:47:06.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 -- 172.21.15.22:0/3470553347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900d1490 msgr2=0x7f52900cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 --2- 172.21.15.22:0/3470553347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900d1490 0x7f52900cebf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5290073f50 tx=0x7f528802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 -- 172.21.15.22:0/3470553347 shutdown_connections 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 --2- 172.21.15.22:0/3470553347 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900d1490 0x7f52900cebf0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 --2- 172.21.15.22:0/3470553347 >> v2:172.21.15.134:3300/0 conn(0x7f52900b51c0 0x7f529015c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 -- 172.21.15.22:0/3470553347 >> 172.21.15.22:0/3470553347 conn(0x7f5290072ae0 msgr2=0x7f5290072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 -- 172.21.15.22:0/3470553347 shutdown_connections 2024-09-21T05:47:06.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.955+0000 7f5295d6e640 1 -- 172.21.15.22:0/3470553347 wait complete. 2024-09-21T05:47:06.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.957+0000 7f5295d6e640 1 Processor -- start 2024-09-21T05:47:06.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.957+0000 7f5295d6e640 1 -- start start 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5295d6e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900b51c0 0x7f5290148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5295d6e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 0x7f529014abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5295d6e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52900d1490 0x7f529015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5295d6e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52900cf2b0 con 0x7f52900ccbd0 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5295d6e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52900cf130 con 0x7f52900b51c0 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5295d6e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f52900cf430 con 0x7f52900d1490 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 0x7f529014abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 0x7f529014abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34870/0 (socket says 172.21.15.22:34870) 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 -- 172.21.15.22:0/856701519 learned_addr learned my addr 172.21.15.22:0/856701519 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528ffff640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52900d1490 0x7f529015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528f7fe640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900b51c0 0x7f5290148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.962 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 -- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52900d1490 msgr2=0x7f529015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52900d1490 0x7f529015ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 -- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900b51c0 msgr2=0x7f5290148290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900b51c0 0x7f5290148290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 -- 172.21.15.22:0/856701519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52901487d0 con 0x7f52900ccbd0 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f528effd640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 0x7f529014abd0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f527800c3a0 tx=0x7f527800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.959+0000 7f5287fff640 1 -- 172.21.15.22:0/856701519 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278011790 con 0x7f52900ccbd0 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.960+0000 7f5287fff640 1 -- 172.21.15.22:0/856701519 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5278017070 con 0x7f52900ccbd0 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.960+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529014b940 con 0x7f52900ccbd0 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.960+0000 7f5287fff640 1 -- 172.21.15.22:0/856701519 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278011c80 con 0x7f52900ccbd0 2024-09-21T05:47:06.963 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.960+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901f70c0 con 0x7f52900ccbd0 2024-09-21T05:47:06.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.961+0000 7f5287fff640 1 -- 172.21.15.22:0/856701519 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f527801c460 con 0x7f52900ccbd0 2024-09-21T05:47:06.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.962+0000 7f5287fff640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5270078bc0 0x7f527007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.962+0000 7f5287fff640 1 -- 172.21.15.22:0/856701519 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f527809aa70 con 0x7f52900ccbd0 2024-09-21T05:47:06.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.962+0000 7f528f7fe640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5270078bc0 0x7f527007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.963+0000 7f528f7fe640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5270078bc0 0x7f527007b0d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5280005fd0 tx=0x7f5280005850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.963+0000 7f529cdbd740 1 Processor -- start 2024-09-21T05:47:06.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.963+0000 7f529cdbd740 1 -- start start 2024-09-21T05:47:06.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.964+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.967 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a2c1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f529cdbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55642a21aa60 con 0x556429edf6b0 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55642a209e90 con 0x556429f00390 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55642a20dca0 con 0x556429ed74d0 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34886/0 (socket says 172.21.15.22:34886) 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 -- 172.21.15.22:0/1377981121 learned_addr learned my addr 172.21.15.22:0/1377981121 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528effd640 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a2c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528f7fe640 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 -- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 msgr2=0x55642a2c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a2c1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 -- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 msgr2=0x55642a2eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 -- 172.21.15.22:0/1377981121 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556429fe27d0 con 0x556429edf6b0 2024-09-21T05:47:06.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f528ffff640 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2c1da0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5288005fd0 tx=0x7f5288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25632b948e2ac4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f52857fa640 1 -- 172.21.15.22:0/1377981121 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288004040 con 0x556429edf6b0 2024-09-21T05:47:06.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f52857fa640 1 -- 172.21.15.22:0/1377981121 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52880042d0 con 0x556429edf6b0 2024-09-21T05:47:06.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.965+0000 7f52857fa640 1 -- 172.21.15.22:0/1377981121 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5288004680 con 0x556429edf6b0 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 -- 172.21.15.22:0/1377981121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 msgr2=0x55642a2c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2c1da0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f5288005fd0 tx=0x7f5288004a40 comp rx=0 tx=0).stop 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 -- 172.21.15.22:0/1377981121 shutdown_connections 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2c1da0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a2c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 --2- 172.21.15.22:0/1377981121 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 -- 172.21.15.22:0/1377981121 >> 172.21.15.22:0/1377981121 conn(0x556429ef44e0 msgr2=0x55642a2a7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.966+0000 7f529cdbd740 1 -- 172.21.15.22:0/1377981121 shutdown_connections 2024-09-21T05:47:06.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.967+0000 7f529cdbd740 1 -- 172.21.15.22:0/1377981121 wait complete. 2024-09-21T05:47:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.967+0000 7f529cdbd740 1 -- start start 2024-09-21T05:47:06.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.967+0000 7f529cdbd740 1 Processor -- start 2024-09-21T05:47:06.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 0x55642a2e3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429edf6b0 0x55642a2e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2e6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556429f7a740 con 0x556429ed74d0 2024-09-21T05:47:06.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55642a1af970 con 0x556429f00390 2024-09-21T05:47:06.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55642a2aa970 con 0x556429edf6b0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f529cdbd740 10 client.0 _ll_register_callbacks cb 0x55642a1be780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 0x55642a2e3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 0x55642a2e3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34896/0 (socket says 172.21.15.22:34896) 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 -- 172.21.15.22:0/287443819 learned_addr learned my addr 172.21.15.22:0/287443819 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528effd640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429edf6b0 0x55642a2e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528ffff640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2e6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 -- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429edf6b0 msgr2=0x55642a2e9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429edf6b0 0x55642a2e9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 -- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 msgr2=0x55642a2e6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2e6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 -- 172.21.15.22:0/287443819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556429f6b970 con 0x556429ed74d0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528effd640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429edf6b0 0x55642a2e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f528f7fe640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 0x55642a2e3ab0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f52800116c0 tx=0x7f52800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd85d6c0e5aecb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.969+0000 7f526e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:06.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f526e7fc640 1 -- 172.21.15.22:0/287443819 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528001f840 con 0x556429ed74d0 2024-09-21T05:47:06.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f526e7fc640 1 -- 172.21.15.22:0/287443819 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f528001f9e0 con 0x556429ed74d0 2024-09-21T05:47:06.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52880042d0 con 0x556429ed74d0 2024-09-21T05:47:06.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f526e7fc640 1 -- 172.21.15.22:0/287443819 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f528001fbe0 con 0x556429ed74d0 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 10 client.7869 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x556429f496a0 con 0x556429ed74d0 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f526e7fc640 1 -- 172.21.15.22:0/287443819 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52880042d0 con 0x556429ed74d0 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f526e7fc640 1 -- 172.21.15.22:0/287443819 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7f5280033080 con 0x556429ed74d0 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 10 client.7869 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 10 client.7869 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 20 client.7869 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x556429ebef60 0x55642a2fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 4 client.7869 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55642a21c9b0 con 0x556429ebef60 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.970+0000 7f528effd640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x556429ebef60 0x55642a2fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f526e7fc640 10 client.7869 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f528effd640 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x556429ebef60 0x55642a2fef60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed453935824caf7b server_cookie=392ba1bf5a627dcc in_seq=0 out_seq=0 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f526e7fc640 1 -- 172.21.15.22:0/287443819 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55642a21c9b0 con 0x556429ebef60 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f526e7fc640 10 client.7869 handle_command_reply: tid=0 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 client.7869 shutdown 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 msgr2=0x55642a2e3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 0x55642a2e3ab0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f52800116c0 tx=0x7f52800046c0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 shutdown_connections 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x556429ebef60 0x55642a2fef60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429f00390 0x55642a2e6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429edf6b0 0x55642a2e9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f526e7fc640 0 client.7869 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:06.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 --2- 172.21.15.22:0/287443819 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429ed74d0 0x55642a2e3ab0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.971+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 >> 172.21.15.22:0/287443819 conn(0x556429ef44e0 msgr2=0x55642a2ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.972+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 shutdown_connections 2024-09-21T05:47:06.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.972+0000 7f529cdbd740 1 -- 172.21.15.22:0/287443819 wait complete. 2024-09-21T05:47:06.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.972+0000 7f529cdbd740 20 client.7869 trim_cache size 0 max 16384 2024-09-21T05:47:06.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.989+0000 7f529cdbd740 1 Processor -- start 2024-09-21T05:47:06.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.990+0000 7f529cdbd740 1 -- start start 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429ed74d0 0x55642a310610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429f00390 0x55642a2e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f529cdbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f52901fff80 con 0x556429edf6b0 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f52901fefa0 con 0x556429ed74d0 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f52901599c0 con 0x556429f00390 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34908/0 (socket says 172.21.15.22:34908) 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 -- 172.21.15.22:0/2007540473 learned_addr learned my addr 172.21.15.22:0/2007540473 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:06.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 -- 172.21.15.22:0/2007540473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429f00390 msgr2=0x55642a2e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429f00390 0x55642a2e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 -- 172.21.15.22:0/2007540473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429ed74d0 msgr2=0x55642a310610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429ed74d0 0x55642a310610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.991+0000 7f528effd640 1 -- 172.21.15.22:0/2007540473 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55642a2a6ea0 con 0x556429edf6b0 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.992+0000 7f528effd640 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2a2c50 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f527800cd60 tx=0x7f527801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8bee572f6c7c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.992+0000 7f5284ff9640 1 -- 172.21.15.22:0/2007540473 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278081070 con 0x556429edf6b0 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.992+0000 7f5284ff9640 1 -- 172.21.15.22:0/2007540473 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5278062750 con 0x556429edf6b0 2024-09-21T05:47:06.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.992+0000 7f5284ff9640 1 -- 172.21.15.22:0/2007540473 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527807c070 con 0x556429edf6b0 2024-09-21T05:47:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 -- 172.21.15.22:0/2007540473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 msgr2=0x55642a2a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2a2c50 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f527800cd60 tx=0x7f527801e040 comp rx=0 tx=0).stop 2024-09-21T05:47:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 -- 172.21.15.22:0/2007540473 shutdown_connections 2024-09-21T05:47:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429f00390 0x55642a2e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429edf6b0 0x55642a2a2c50 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 --2- 172.21.15.22:0/2007540473 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429ed74d0 0x55642a310610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 -- 172.21.15.22:0/2007540473 >> 172.21.15.22:0/2007540473 conn(0x556429ef44e0 msgr2=0x55642a2e3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 -- 172.21.15.22:0/2007540473 shutdown_connections 2024-09-21T05:47:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.993+0000 7f529cdbd740 1 -- 172.21.15.22:0/2007540473 wait complete. 2024-09-21T05:47:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.994+0000 7f529cdbd740 1 -- start start 2024-09-21T05:47:06.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.994+0000 7f529cdbd740 1 Processor -- start 2024-09-21T05:47:07.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a35ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:07.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429edf6b0 0x55642a2e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:07.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 0x55642a2d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:07.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55642a2e5c20 con 0x556429f00390 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f529015a010 con 0x556429edf6b0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55642a21c9b0 con 0x556429ed74d0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f529cdbd740 10 client.0 _ll_register_callbacks cb 0x7f5280005cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 0x55642a2d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 0x55642a2d20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:34920/0 (socket says 172.21.15.22:34920) 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 -- 172.21.15.22:0/879445577 learned_addr learned my addr 172.21.15.22:0/879445577 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528f7fe640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a35ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528effd640 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429edf6b0 0x55642a2e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 -- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 msgr2=0x55642a35ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a35ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 -- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429edf6b0 msgr2=0x55642a2e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429edf6b0 0x55642a2e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.995+0000 7f528ffff640 1 -- 172.21.15.22:0/879445577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55642a2e38b0 con 0x556429f00390 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f528ffff640 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 0x55642a2d20f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f528802aea0 tx=0x7f5288005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d8eb3b9913a8eed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:07.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f526e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f526e7fc640 1 -- 172.21.15.22:0/879445577 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52880044e0 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f526e7fc640 1 -- 172.21.15.22:0/879445577 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f5288004680 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5278062750 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f526e7fc640 1 -- 172.21.15.22:0/879445577 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f5288045440 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 10 client.7875 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x55642a2fe3e0 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f526e7fc640 1 -- 172.21.15.22:0/879445577 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5278062750 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f526e7fc640 1 -- 172.21.15.22:0/879445577 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7f5288043020 con 0x556429f00390 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 10 client.7875 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 10 client.7875 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 20 client.7875 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55642a2fd910 0x55642a2fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 4 client.7875 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55642a218460 con 0x55642a2fd910 2024-09-21T05:47:07.002 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.996+0000 7f528f7fe640 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55642a2fd910 0x55642a2fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f526e7fc640 10 client.7875 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f528f7fe640 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55642a2fd910 0x55642a2fdd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd5973ef823a579 server_cookie=2f16a1d8b15d45ae in_seq=0 out_seq=0 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f526e7fc640 1 -- 172.21.15.22:0/879445577 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x55642a218460 con 0x55642a2fd910 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f526e7fc640 10 client.7875 handle_command_reply: tid=0 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 client.7875 shutdown 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 msgr2=0x55642a2d20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 0x55642a2d20f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f528802aea0 tx=0x7f5288005bd0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 shutdown_connections 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x55642a2fd910 0x55642a2fdd20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556429f00390 0x55642a2d20f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556429edf6b0 0x55642a2e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f526e7fc640 0 client.7875 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 --2- 172.21.15.22:0/879445577 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556429ed74d0 0x55642a35ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 >> 172.21.15.22:0/879445577 conn(0x556429ef44e0 msgr2=0x55642a2fac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 shutdown_connections 2024-09-21T05:47:07.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 1 -- 172.21.15.22:0/879445577 wait complete. 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.997+0000 7f529cdbd740 20 client.7875 trim_cache size 0 max 16384 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": { 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "e63d795c-851d-415f-8ea3-1e40fd0e3fe9": { 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "tag": "e63d795c-851d-415f-8ea3-1e40fd0e3fe9", 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "options": "recursive,repair" 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:07.004 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "e63d795c-851d-415f-8ea3-1e40fd0e3fe9": { 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:47:06.823781+0000", 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:07.005 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5270078bc0 msgr2=0x7f527007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5270078bc0 0x7f527007b0d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5280005fd0 tx=0x7f5280005850 comp rx=0 tx=0).stop 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 msgr2=0x7f529014abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 0x7f529014abd0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f527800c3a0 tx=0x7f527800c860 comp rx=0 tx=0).stop 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 shutdown_connections 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5270078bc0 0x7f527007b0d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f52900d1490 0x7f529015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f52900ccbd0 0x7f529014abd0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 --2- 172.21.15.22:0/856701519 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f52900b51c0 0x7f5290148290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 >> 172.21.15.22:0/856701519 conn(0x7f5290072ae0 msgr2=0x7f52900d1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 shutdown_connections 2024-09-21T05:47:07.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:06.998+0000 7f5295d6e640 1 -- 172.21.15.22:0/856701519 wait complete. 2024-09-21T05:47:07.011 INFO:tasks.cephfs.filesystem:scrub status for tag:e63d795c-851d-415f-8ea3-1e40fd0e3fe9 - {'path': '/', 'tag': 'e63d795c-851d-415f-8ea3-1e40fd0e3fe9', 'options': 'recursive,repair'} 2024-09-21T05:47:08.117 INFO:tasks.ceph.mds.c.smithi022.stderr:2024-09-21T05:47:08.116+0000 7f06f82b9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:47:08.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:08.137+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 8 ==== log(8 entries from seq 1 at 2024-09-21T05:47:06.824459+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f4cd805f880 con 0x7f4ce015a770 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.824459+0000 mds.c [INF] scrub queued for path: / 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.824465+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.824473+0000 mds.c [INF] scrub summary: active paths [/] 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.826361+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.826370+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.826441+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.826447+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-09-21T05:47:08.140 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:06.827297+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-21T05:47:13.147 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:13.144+0000 7f4cd5ffb640 1 -- 172.21.15.22:0/3959761464 <== mon.0 v2:172.21.15.22:3300/0 9 ==== log(1 entries from seq 9 at 2024-09-21T05:47:11.183154+0000) ==== 189+0+0 (secure 0 0 0) 0x7f4cd8063870 con 0x7f4ce015a770 2024-09-21T05:47:13.147 INFO:teuthology.orchestra.run.smithi022.stdout:2024-09-21T05:47:11.183154+0000 mds.c [INF] scrub summary: idle 2024-09-21T05:47:22.649 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:47:22.652 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:47:37.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:47:37.012 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-21T05:47:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 -- 172.21.15.22:0/3804573138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180b4d90 msgr2=0x7f9518072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 --2- 172.21.15.22:0/3804573138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180b4d90 0x7f9518072e30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f950800a940 tx=0x7f950802b850 comp rx=0 tx=0).stop 2024-09-21T05:47:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 -- 172.21.15.22:0/3804573138 shutdown_connections 2024-09-21T05:47:37.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 --2- 172.21.15.22:0/3804573138 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180b4d90 0x7f9518072e30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 -- 172.21.15.22:0/3804573138 >> 172.21.15.22:0/3804573138 conn(0x7f951806d7d0 msgr2=0x7f95180b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 -- 172.21.15.22:0/3804573138 shutdown_connections 2024-09-21T05:47:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.146+0000 7f951d2cc640 1 -- 172.21.15.22:0/3804573138 wait complete. 2024-09-21T05:47:37.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.148+0000 7f951d2cc640 1 Processor -- start 2024-09-21T05:47:37.150 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.148+0000 7f951d2cc640 1 -- start start 2024-09-21T05:47:37.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f951d2cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9518070390 0x7f95181e5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f951d2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 0x7f95181f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f951d2cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181474a0 0x7f9518159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f951d2cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f95180ccbc0 con 0x7f95180cf940 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f951d2cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f95180709b0 con 0x7f95181474a0 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f951d2cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95180ccd40 con 0x7f9518070390 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 0x7f95181f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 0x7f95181f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41144/0 (socket says 172.21.15.22:41144) 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 -- 172.21.15.22:0/1354732032 learned_addr learned my addr 172.21.15.22:0/1354732032 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f95177fe640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181474a0 0x7f9518159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f9516ffd640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9518070390 0x7f95181e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 -- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9518070390 msgr2=0x7f95181e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9518070390 0x7f95181e5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 -- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181474a0 msgr2=0x7f9518159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181474a0 0x7f9518159a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f950ffff640 1 -- 172.21.15.22:0/1354732032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95181f7560 con 0x7f95180cf940 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.149+0000 7f9516ffd640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9518070390 0x7f95181e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.150+0000 7f950ffff640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 0x7f95181f6aa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f950000c730 tx=0x7f950000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.150+0000 7f950f7fe640 1 -- 172.21.15.22:0/1354732032 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500006890 con 0x7f95180cf940 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.150+0000 7f950f7fe640 1 -- 172.21.15.22:0/1354732032 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9500017070 con 0x7f95180cf940 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.150+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95181f7840 con 0x7f95180cf940 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.150+0000 7f950f7fe640 1 -- 172.21.15.22:0/1354732032 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500006e00 con 0x7f95180cf940 2024-09-21T05:47:37.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.150+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181e5770 con 0x7f95180cf940 2024-09-21T05:47:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.152+0000 7f950f7fe640 1 -- 172.21.15.22:0/1354732032 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9500007050 con 0x7f95180cf940 2024-09-21T05:47:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.152+0000 7f950f7fe640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f94f8078bc0 0x7f94f807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.153+0000 7f950f7fe640 1 -- 172.21.15.22:0/1354732032 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f950009a9e0 con 0x7f95180cf940 2024-09-21T05:47:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.153+0000 7f9516ffd640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f94f8078bc0 0x7f94f807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.153+0000 7f9524238740 1 Processor -- start 2024-09-21T05:47:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.153+0000 7f9524238740 1 -- start start 2024-09-21T05:47:37.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.154+0000 7f9516ffd640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f94f8078bc0 0x7f94f807b0d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9508036000 tx=0x7f9508004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dcfe6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfbb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9524238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9524238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591dcf14a60 con 0x5591dcbd96b0 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591dcf03e90 con 0x5591dcbfa390 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591dcf07ca0 con 0x5591dcbd14d0 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfbbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41148/0 (socket says 172.21.15.22:41148) 2024-09-21T05:47:37.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f950ffff640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfbb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 -- 172.21.15.22:0/4021544063 learned_addr learned my addr 172.21.15.22:0/4021544063 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f9516ffd640 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dcfe6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 -- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 msgr2=0x5591dcfbb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfbb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 -- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 msgr2=0x5591dcfe6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dcfe6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.155+0000 7f95177fe640 1 -- 172.21.15.22:0/4021544063 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591dccdc7d0 con 0x5591dcbd96b0 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.156+0000 7f95177fe640 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfbbda0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9510009f30 tx=0x7f951000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371526603ec13532 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.156+0000 7f950cff9640 1 -- 172.21.15.22:0/4021544063 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951001a070 con 0x5591dcbd96b0 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.156+0000 7f950cff9640 1 -- 172.21.15.22:0/4021544063 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f951000a5f0 con 0x5591dcbd96b0 2024-09-21T05:47:37.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.156+0000 7f950cff9640 1 -- 172.21.15.22:0/4021544063 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951000aa00 con 0x5591dcbd96b0 2024-09-21T05:47:37.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 -- 172.21.15.22:0/4021544063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 msgr2=0x5591dcfbbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfbbda0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f9510009f30 tx=0x7f951000c890 comp rx=0 tx=0).stop 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 -- 172.21.15.22:0/4021544063 shutdown_connections 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfbbda0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfbb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 --2- 172.21.15.22:0/4021544063 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dcfe6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 -- 172.21.15.22:0/4021544063 >> 172.21.15.22:0/4021544063 conn(0x5591dcbee4e0 msgr2=0x5591dcfa1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 -- 172.21.15.22:0/4021544063 shutdown_connections 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.157+0000 7f9524238740 1 -- 172.21.15.22:0/4021544063 wait complete. 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.158+0000 7f9524238740 1 -- start start 2024-09-21T05:47:37.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.158+0000 7f9524238740 1 Processor -- start 2024-09-21T05:47:37.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd14d0 0x5591dcfcc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfa0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbfa390 0x5591dcfa0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591dcc74740 con 0x5591dcbd96b0 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5591dcea9970 con 0x5591dcbd14d0 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591dcfa1460 con 0x5591dcbfa390 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f9524238740 10 client.0 _ll_register_callbacks cb 0x5591dceb8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f950ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfa0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f950ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfa0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41154/0 (socket says 172.21.15.22:41154) 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.159+0000 7f950ffff640 1 -- 172.21.15.22:0/3054187152 learned_addr learned my addr 172.21.15.22:0/3054187152 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f95177fe640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbfa390 0x5591dcfa0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f9516ffd640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd14d0 0x5591dcfcc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f950ffff640 1 -- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbfa390 msgr2=0x5591dcfa0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f950ffff640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbfa390 0x5591dcfa0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f950ffff640 1 -- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd14d0 msgr2=0x5591dcfcc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f950ffff640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd14d0 0x5591dcfcc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f950ffff640 1 -- 172.21.15.22:0/3054187152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591dcc65970 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f9516ffd640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd14d0 0x5591dcfcc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f950ffff640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfa0350 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9500001090 tx=0x7f9500062e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a6ded5b2465f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f94f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f94f5ffb640 1 -- 172.21.15.22:0/3054187152 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000627e0 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f94f5ffb640 1 -- 172.21.15.22:0/3054187152 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f9500007340 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951000a5f0 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f94f5ffb640 1 -- 172.21.15.22:0/3054187152 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f95000672c0 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f9524238740 10 client.7890 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x5591dcc436a0 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.160+0000 7f94f5ffb640 1 -- 172.21.15.22:0/3054187152 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f951000a5f0 con 0x5591dcbd96b0 2024-09-21T05:47:37.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f94f5ffb640 1 -- 172.21.15.22:0/3054187152 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7f9500063b10 con 0x5591dcbd96b0 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9524238740 10 client.7890 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9524238740 10 client.7890 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9524238740 20 client.7890 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9524238740 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcbb8f60 0x5591dcff9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9524238740 4 client.7890 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591dcf169b0 con 0x5591dcbb8f60 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9516ffd640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcbb8f60 0x5591dcff9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f94f5ffb640 10 client.7890 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.161+0000 7f9516ffd640 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcbb8f60 0x5591dcff9380 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a988d546083a04 server_cookie=84fc5eae1568bc40 in_seq=0 out_seq=0 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f94f5ffb640 1 -- 172.21.15.22:0/3054187152 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591dcf169b0 con 0x5591dcbb8f60 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f94f5ffb640 10 client.7890 handle_command_reply: tid=0 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 client.7890 shutdown 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 msgr2=0x5591dcfa0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfa0350 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f9500001090 tx=0x7f9500062e00 comp rx=0 tx=0).stop 2024-09-21T05:47:37.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 shutdown_connections 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcbb8f60 0x5591dcff9380 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbfa390 0x5591dcfa0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfa0350 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f94f5ffb640 0 client.7890 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 --2- 172.21.15.22:0/3054187152 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd14d0 0x5591dcfcc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 >> 172.21.15.22:0/3054187152 conn(0x5591dcbee4e0 msgr2=0x5591dcfc8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 shutdown_connections 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 1 -- 172.21.15.22:0/3054187152 wait complete. 2024-09-21T05:47:37.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.162+0000 7f9524238740 20 client.7890 trim_cache size 0 max 16384 2024-09-21T05:47:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.180+0000 7f9524238740 1 Processor -- start 2024-09-21T05:47:37.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.181+0000 7f9524238740 1 -- start start 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd96b0 0x5591dcfff500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9524238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 0x5591dcfffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9524238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5591dcfa33b0 con 0x5591dcbfa390 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f9518200160 con 0x5591dcbd96b0 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f95181ff160 con 0x5591dcbd14d0 2024-09-21T05:47:37.186 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 0x5591dcfffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 0x5591dcfffa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41170/0 (socket says 172.21.15.22:41170) 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 -- 172.21.15.22:0/2021224005 learned_addr learned my addr 172.21.15.22:0/2021224005 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f950ffff640 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd96b0 0x5591dcfff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f9516ffd640 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 -- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 msgr2=0x5591dcfe2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 -- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd96b0 msgr2=0x5591dcfff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd96b0 0x5591dcfff500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 -- 172.21.15.22:0/2021224005 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591dcf9f490 con 0x5591dcbfa390 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.183+0000 7f95177fe640 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 0x5591dcfffa40 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f951000cd90 tx=0x7f9510004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78d96ade30c57f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.184+0000 7f94f7fff640 1 -- 172.21.15.22:0/2021224005 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9510024020 con 0x5591dcbfa390 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.184+0000 7f94f7fff640 1 -- 172.21.15.22:0/2021224005 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9510003a60 con 0x5591dcbfa390 2024-09-21T05:47:37.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.184+0000 7f94f7fff640 1 -- 172.21.15.22:0/2021224005 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951001a040 con 0x5591dcbfa390 2024-09-21T05:47:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 -- 172.21.15.22:0/2021224005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 msgr2=0x5591dcfffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 0x5591dcfffa40 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f951000cd90 tx=0x7f9510004a70 comp rx=0 tx=0).stop 2024-09-21T05:47:37.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 -- 172.21.15.22:0/2021224005 shutdown_connections 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbfa390 0x5591dcfffa40 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbd96b0 0x5591dcfff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 --2- 172.21.15.22:0/2021224005 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 -- 172.21.15.22:0/2021224005 >> 172.21.15.22:0/2021224005 conn(0x5591dcbee4e0 msgr2=0x5591dcfc8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 -- 172.21.15.22:0/2021224005 shutdown_connections 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 -- 172.21.15.22:0/2021224005 wait complete. 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.185+0000 7f9524238740 1 -- start start 2024-09-21T05:47:37.189 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.186+0000 7f9524238740 1 Processor -- start 2024-09-21T05:47:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfb9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dd05fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f951809b260 con 0x5591dcbd96b0 2024-09-21T05:47:37.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f951814abf0 con 0x5591dcbfa390 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5591dd066750 con 0x5591dcbd14d0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9524238740 10 client.0 _ll_register_callbacks cb 0x7f95000625b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f950ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfb9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f950ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfb9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41176/0 (socket says 172.21.15.22:41176) 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f950ffff640 1 -- 172.21.15.22:0/2560403140 learned_addr learned my addr 172.21.15.22:0/2560403140 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f9516ffd640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.187+0000 7f95177fe640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dd05fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f950ffff640 1 -- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 msgr2=0x5591dcfe6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f950ffff640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f950ffff640 1 -- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 msgr2=0x5591dd05fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f950ffff640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dd05fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f950ffff640 1 -- 172.21.15.22:0/2560403140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591dcfe6480 con 0x5591dcbd96b0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f95177fe640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dd05fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f950ffff640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfb9700 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f950005ed60 tx=0x7f950005e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db11952f526fdf6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f94f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f94f5ffb640 1 -- 172.21.15.22:0/2560403140 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95000627e0 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f94f5ffb640 1 -- 172.21.15.22:0/2560403140 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7f950005e130 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9510003a60 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f94f5ffb640 1 -- 172.21.15.22:0/2560403140 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f950005ee30 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 10 client.7896 fetch_fsmap learned FSMap version 90 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) -- 0x5591dcfe6480 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f94f5ffb640 1 -- 172.21.15.22:0/2560403140 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9510003a60 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f94f5ffb640 1 -- 172.21.15.22:0/2560403140 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 90) ==== 3502+0+0 (secure 0 0 0) 0x7f950005e330 con 0x5591dcbd96b0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 10 client.7896 fetch_fsmap finished waiting for FSMap version 90 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 10 client.7896 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 20 client.7896 populate_metadata read hostname 'smithi022' 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcfe6f80 0x5591dd060040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.188+0000 7f9524238740 4 client.7896 mds_command: new command op to 7755 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 --> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591dcf12460 con 0x5591dcfe6f80 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f9516ffd640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcfe6f80 0x5591dd060040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f94f5ffb640 10 client.7896 ms_handle_connect on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f9516ffd640 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcfe6f80 0x5591dd060040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ac54c43d30f452e server_cookie=eeea21d7df0317a4 in_seq=0 out_seq=0 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f94f5ffb640 1 -- 172.21.15.22:0/2560403140 <== mds.0 v2:172.21.15.22:6832/3179586532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x5591dcf12460 con 0x5591dcfe6f80 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f94f5ffb640 10 client.7896 handle_command_reply: tid=0 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.189+0000 7f9524238740 1 client.7896 shutdown 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 msgr2=0x5591dcfb9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfb9700 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f950005ed60 tx=0x7f950005e780 comp rx=0 tx=0).stop 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 shutdown_connections 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:6832/3179586532,v1:172.21.15.22:6834/3179586532] conn(0x5591dcfe6f80 0x5591dd060040 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5591dcbfa390 0x5591dd05fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5591dcbd96b0 0x5591dcfb9700 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 --2- 172.21.15.22:0/2560403140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5591dcbd14d0 0x5591dcfe6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f94f5ffb640 0 client.7896 ms_handle_reset on v2:172.21.15.22:6832/3179586532 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 >> 172.21.15.22:0/2560403140 conn(0x5591dcbee4e0 msgr2=0x5591dcfcbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 shutdown_connections 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 1 -- 172.21.15.22:0/2560403140 wait complete. 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f9524238740 20 client.7896 trim_cache size 0 max 16384 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "status": "no active scrubs running", 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "scrubs": {}, 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "scrub_stats": { 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "e63d795c-851d-415f-8ea3-1e40fd0e3fe9": { 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "start_time": "2024-09-21T05:47:06.823781+0000", 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "path": "/", 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_started": 0, 2024-09-21T05:47:37.197 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_passed": 0, 2024-09-21T05:47:37.198 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_failed": 0, 2024-09-21T05:47:37.198 INFO:teuthology.orchestra.run.smithi022.stdout: "uninline_skipped": 0 2024-09-21T05:47:37.198 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:37.198 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:37.198 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:37.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f94f8078bc0 msgr2=0x7f94f807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f951d2cc640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f94f8078bc0 0x7f94f807b0d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9508036000 tx=0x7f9508004730 comp rx=0 tx=0).stop 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 msgr2=0x7f95181f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.190+0000 7f951d2cc640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 0x7f95181f6aa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f950000c730 tx=0x7f950000cbf0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 shutdown_connections 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f95181474a0 0x7f9518159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f95180cf940 0x7f95181f6aa0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f94f8078bc0 0x7f94f807b0d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 --2- 172.21.15.22:0/1354732032 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f9518070390 0x7f95181e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 >> 172.21.15.22:0/1354732032 conn(0x7f951806d7d0 msgr2=0x7f9518073c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 shutdown_connections 2024-09-21T05:47:37.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.191+0000 7f951d2cc640 1 -- 172.21.15.22:0/1354732032 wait complete. 2024-09-21T05:47:37.204 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T05:47:37.306 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:47:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 -- 172.21.15.22:0/2831389450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806e0a0 msgr2=0x7fa6780b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 --2- 172.21.15.22:0/2831389450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806e0a0 0x7fa6780b97e0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa6780cebb0 tx=0x7fa66802a860 comp rx=0 tx=0).stop 2024-09-21T05:47:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 -- 172.21.15.22:0/2831389450 shutdown_connections 2024-09-21T05:47:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 --2- 172.21.15.22:0/2831389450 >> v2:172.21.15.134:3300/0 conn(0x7fa67806f6e0 0x7fa67806e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 --2- 172.21.15.22:0/2831389450 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806e0a0 0x7fa6780b97e0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 -- 172.21.15.22:0/2831389450 >> 172.21.15.22:0/2831389450 conn(0x7fa67809d840 msgr2=0x7fa67808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 -- 172.21.15.22:0/2831389450 shutdown_connections 2024-09-21T05:47:37.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.421+0000 7fa67c863640 1 -- 172.21.15.22:0/2831389450 wait complete. 2024-09-21T05:47:37.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.423+0000 7fa67c863640 1 Processor -- start 2024-09-21T05:47:37.425 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.423+0000 7fa67c863640 1 -- start start 2024-09-21T05:47:37.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa67c863640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67806e0a0 0x7fa6781f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa67c863640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 0x7fa67815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa67c863640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6780d0e50 0x7fa6780ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa67c863640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6780cfea0 con 0x7fa67806f6e0 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa67c863640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6780cfd20 con 0x7fa67806e0a0 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa67c863640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6780d0020 con 0x7fa6780d0e50 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa675d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 0x7fa67815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.424+0000 7fa675d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 0x7fa67815cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41184/0 (socket says 172.21.15.22:41184) 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 -- 172.21.15.22:0/3985128169 learned_addr learned my addr 172.21.15.22:0/3985128169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa67659c640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67806e0a0 0x7fa6781f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa676d9d640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6780d0e50 0x7fa6780ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 -- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6780d0e50 msgr2=0x7fa6780ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6780d0e50 0x7fa6780ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 -- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67806e0a0 msgr2=0x7fa6781f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67806e0a0 0x7fa6781f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 -- 172.21.15.22:0/3985128169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6781fa1b0 con 0x7fa67806f6e0 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa675d9b640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 0x7fa67815cb40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00d6a0 tx=0x7fa66c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c017070 con 0x7fa67806f6e0 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa66c004310 con 0x7fa67806f6e0 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6781fa490 con 0x7fa67806f6e0 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c009a80 con 0x7fa67806f6e0 2024-09-21T05:47:37.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.425+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6780a1d30 con 0x7fa67806f6e0 2024-09-21T05:47:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.426+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6780d0460 con 0x7fa67806f6e0 2024-09-21T05:47:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.427+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa66c009c20 con 0x7fa67806f6e0 2024-09-21T05:47:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.428+0000 7fa666ffd640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa658078bc0 0x7fa65807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.428+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5795+0+0 (secure 0 0 0) 0x7fa66c09abc0 con 0x7fa67806f6e0 2024-09-21T05:47:37.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.428+0000 7fa67659c640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa658078bc0 0x7fa65807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.429+0000 7fa67659c640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa658078bc0 0x7fa65807b0d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa668035000 tx=0x7fa668004ab0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.435 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.433+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa66c062a70 con 0x7fa67806f6e0 2024-09-21T05:47:37.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.660+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa6780d1970 con 0x7fa67806f6e0 2024-09-21T05:47:37.663 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:47:37.661+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:47:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.662+0000 7fa666ffd640 1 -- 172.21.15.22:0/3985128169 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7fa66c0674a0 con 0x7fa67806f6e0 2024-09-21T05:47:37.664 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:47:37.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa658078bc0 msgr2=0x7fa65807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa658078bc0 0x7fa65807b0d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa668035000 tx=0x7fa668004ab0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 msgr2=0x7fa67815cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 0x7fa67815cb40 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa66c00d6a0 tx=0x7fa66c00db60 comp rx=0 tx=0).stop 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 shutdown_connections 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa658078bc0 0x7fa65807b0d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6780d0e50 0x7fa6780ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa67806f6e0 0x7fa67815cb40 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 --2- 172.21.15.22:0/3985128169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa67806e0a0 0x7fa6781f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 >> 172.21.15.22:0/3985128169 conn(0x7fa67809d840 msgr2=0x7fa678161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 shutdown_connections 2024-09-21T05:47:37.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.665+0000 7fa67c863640 1 -- 172.21.15.22:0/3985128169 wait complete. 2024-09-21T05:47:37.670 INFO:tasks.ceph.mds.c.smithi022.stderr: -1223> 2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-09-21T05:47:37.670 INFO:tasks.ceph.mds.c.smithi022.stderr: -1220> 2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-09-21T05:47:37.670 INFO:tasks.ceph.mds.c.smithi022.stderr: -1202> 2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-09-21T05:47:37.670 INFO:tasks.ceph.mds.c.smithi022.stderr: -1199> 2024-09-21T05:47:06.825+0000 7f06f42b1640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-09-21T05:47:37.670 INFO:tasks.ceph.mds.c.smithi022.stderr: -1022> 2024-09-21T05:47:08.116+0000 7f06f82b9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:47:37.680 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-09-21T05:47:37.729 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:47:37.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.734+0000 7fde8ee00280 1 -- start start 2024-09-21T05:47:37.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.734+0000 7fde8ee00280 1 Processor -- start 2024-09-21T05:47:37.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c35c9a80 0x55b6c34f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 0x55b6c34f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218000 0x55b6c34f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8ee00280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3510780 con 0x55b6c35c9180 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8ee00280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3510600 con 0x55b6c35c9a80 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8ee00280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b6c3510480 con 0x55b6c4218000 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8cda4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 0x55b6c34f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8cda4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 0x55b6c34f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41220/0 (socket says 172.21.15.22:41220) 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8cda4640 1 -- 172.21.15.22:0/1600838039 learned_addr learned my addr 172.21.15.22:0/1600838039 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8dda6640 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218000 0x55b6c34f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.735+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c35c9a80 0x55b6c34f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8cda4640 1 -- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218000 msgr2=0x55b6c34f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8cda4640 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218000 0x55b6c34f0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8cda4640 1 -- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c35c9a80 msgr2=0x55b6c34f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8cda4640 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c35c9a80 0x55b6c34f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8cda4640 1 -- 172.21.15.22:0/1600838039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c3472780 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8cda4640 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 0x55b6c34f0580 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x55b6c425f890 tx=0x55b6c35309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce09a7647f8af9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ada0640 1 -- 172.21.15.22:0/1600838039 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6bf42aea0 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ada0640 1 -- 172.21.15.22:0/1600838039 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b6bf431180 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ada0640 1 -- 172.21.15.22:0/1600838039 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6bf42b380 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c34734a0 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b6c3472000 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ee00280 4 waiting for MDS map... 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ada0640 1 -- 172.21.15.22:0/1600838039 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x55b6c4316600 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ada0640 1 -- 172.21.15.22:0/1600838039 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 91) ==== 2893+0+0 (secure 0 0 0) 0x55b6c348f180 con 0x55b6c35c9180 2024-09-21T05:47:37.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ee00280 4 Got MDS map 91 2024-09-21T05:47:37.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.736+0000 7fde8ee00280 10 main: main 2024-09-21T05:47:37.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.737+0000 7fde8ee00280 4 main: connecting to RADOS... 2024-09-21T05:47:37.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.737+0000 7fde8ee00280 1 Processor -- start 2024-09-21T05:47:37.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.737+0000 7fde8ee00280 1 -- start start 2024-09-21T05:47:37.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.738+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.738+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.747 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8ee00280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3510300 con 0x55b6c4218d80 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8ee00280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3510000 con 0x55b6c4219200 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8ee00280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b6c3497800 con 0x55b6c4218900 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41230/0 (socket says 172.21.15.22:41230) 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 -- 172.21.15.22:0/1470182221 learned_addr learned my addr 172.21.15.22:0/1470182221 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8cda4640 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8dda6640 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 -- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218900 msgr2=0x55b6c34f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 -- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4219200 msgr2=0x55b6c34f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:47:37.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 -- 172.21.15.22:0/1470182221 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c34723c0 con 0x55b6c4218d80 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8cda4640 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x55b6c4320bd0 tx=0x55b6c4382000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d4693aeb40472f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde88d9c640 1 -- 172.21.15.22:0/1470182221 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6bf42a340 con 0x55b6c4218d80 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde88d9c640 1 -- 172.21.15.22:0/1470182221 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b6bf430fc0 con 0x55b6c4218d80 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.739+0000 7fde88d9c640 1 -- 172.21.15.22:0/1470182221 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6bf42aea0 con 0x55b6c4218d80 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 -- 172.21.15.22:0/1470182221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 msgr2=0x55b6c34f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x55b6c4320bd0 tx=0x55b6c4382000 comp rx=0 tx=0).stop 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 -- 172.21.15.22:0/1470182221 shutdown_connections 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1470182221 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 -- 172.21.15.22:0/1470182221 >> 172.21.15.22:0/1470182221 conn(0x55b6c4218480 msgr2=0x55b6c354f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 -- 172.21.15.22:0/1470182221 shutdown_connections 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.740+0000 7fde8ee00280 1 -- 172.21.15.22:0/1470182221 wait complete. 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.741+0000 7fde8ee00280 1 Processor -- start 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.741+0000 7fde8ee00280 1 -- start start 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8ee00280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8ee00280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3511380 con 0x55b6c4218900 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8ee00280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55b6c3510000 con 0x55b6c4218d80 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8ee00280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55b6c3511500 con 0x55b6c4219200 2024-09-21T05:47:37.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218900 0x55b6c34f1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41236/0 (socket says 172.21.15.22:41236) 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 -- 172.21.15.22:0/2733540326 learned_addr learned my addr 172.21.15.22:0/2733540326 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8cda4640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 -- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4219200 msgr2=0x55b6c34f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4219200 0x55b6c34f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 -- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4218d80 msgr2=0x55b6c34f1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c4218d80 0x55b6c34f1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 -- 172.21.15.22:0/2733540326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c3472780 con 0x55b6c4218900 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.742+0000 7fde8dda6640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218900 0x55b6c34f1080 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x55b6c4321a10 tx=0x55b6c43822a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.743+0000 7fde87599640 1 -- 172.21.15.22:0/2733540326 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6bf42a340 con 0x55b6c4218900 2024-09-21T05:47:37.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.743+0000 7fde87599640 1 -- 172.21.15.22:0/2733540326 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b6bf430380 con 0x55b6c4218900 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.743+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b6c34734a0 con 0x55b6c4218900 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.743+0000 7fde87599640 1 -- 172.21.15.22:0/2733540326 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b6bf42aea0 con 0x55b6c4218900 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.743+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c3472960 con 0x55b6c4218900 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.743+0000 7fde8ee00280 4 main: resolving pool 14 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.744+0000 7fde87599640 1 -- 172.21.15.22:0/2733540326 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x55b6c4238c00 con 0x55b6c4218900 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.744+0000 7fde87599640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b6c421b600 0x55b6c34f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde87599640 1 -- 172.21.15.22:0/2733540326 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x55b6c4240a00 con 0x55b6c4218900 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8ee00280 4 main: creating IoCtx.. 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8ee00280 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b6c4219680 0x55b6c34f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) -- 0x55b6c4384800 con 0x55b6c4219680 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b6c421b600 0x55b6c34f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8cda4640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b6c4219680 0x55b6c34f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:37.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8cda4640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b6c4219680 0x55b6c34f2680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.745+0000 7fde8d5a5640 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b6c421b600 0x55b6c34f2100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55b6c4320bd0 tx=0x55b6c34f4300 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:37.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8cda4640 1 -- 172.21.15.22:0/2733540326 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55b6c352c000 con 0x55b6c4219680 2024-09-21T05:47:37.753 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:37.753 INFO:teuthology.orchestra.run.smithi022.stdout: "0": { 2024-09-21T05:47:37.753 INFO:teuthology.orchestra.run.smithi022.stdout: "data": { 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: "version": 5, 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: "inotable": { 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: "projected_free": [ 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: "start": 1099511627779, 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: "len": 1099511627773 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: ], 2024-09-21T05:47:37.754 INFO:teuthology.orchestra.run.smithi022.stdout: "free": [ 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: "start": 1099511627779, 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: "len": 1099511627773 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: "result": 0 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:37.755 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:37.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b6c4219680 msgr2=0x55b6c34f2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x55b6c4219680 0x55b6c34f2680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b6c421b600 msgr2=0x55b6c34f2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x55b6c421b600 0x55b6c34f2100 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x55b6c4320bd0 tx=0x55b6c34f4300 comp rx=0 tx=0).stop 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218900 msgr2=0x55b6c34f1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/2733540326 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c4218900 0x55b6c34f1080 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x55b6c4321a10 tx=0x55b6c43822a0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8d5a5640 1 -- 172.21.15.22:0/2733540326 reap_dead start 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 shutdown_connections 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 >> 172.21.15.22:0/2733540326 conn(0x55b6c4218480 msgr2=0x55b6c354f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 shutdown_connections 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/2733540326 wait complete. 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 msgr2=0x55b6c34f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:37.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 0x55b6c34f0580 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x55b6c425f890 tx=0x55b6c35309c0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 shutdown_connections 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55b6c4218000 0x55b6c34f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55b6c35c9180 0x55b6c34f0580 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 --2- 172.21.15.22:0/1600838039 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55b6c35c9a80 0x55b6c34f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 >> 172.21.15.22:0/1600838039 conn(0x55b6c35c9600 msgr2=0x55b6c354f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 shutdown_connections 2024-09-21T05:47:37.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:37.748+0000 7fde8ee00280 1 -- 172.21.15.22:0/1600838039 wait complete. 2024-09-21T05:47:37.759 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.079056, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-09-21T05:47:37.760 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:47:37.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:47:37.800 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:47:37.846 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:47:37.846 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:47:37.846 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:47:37.846 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:37.863 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:47:37.863 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:47:37.863 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:47:37.863 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:37.881 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:47:37.881 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:47:37.881 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:47:37.881 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:37.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:47:37.919 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:47:37.919 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:47:37.919 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:37.939 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:47:37.939 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 -- 172.21.15.22:0/2160320591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0d2ed0 msgr2=0x7f6d3c0d1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 --2- 172.21.15.22:0/2160320591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0d1a80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009820 tx=0x7f6d2c02a510 comp rx=0 tx=0).stop 2024-09-21T05:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 -- 172.21.15.22:0/2160320591 shutdown_connections 2024-09-21T05:47:38.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 --2- 172.21.15.22:0/2160320591 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0cf7a0 0x7f6d3c1626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 --2- 172.21.15.22:0/2160320591 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0d1a80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 -- 172.21.15.22:0/2160320591 >> 172.21.15.22:0/2160320591 conn(0x7f6d3c08bfc0 msgr2=0x7f6d3c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 -- 172.21.15.22:0/2160320591 shutdown_connections 2024-09-21T05:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.073+0000 7f6d423f1640 1 -- 172.21.15.22:0/2160320591 wait complete. 2024-09-21T05:47:38.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.075+0000 7f6d423f1640 1 Processor -- start 2024-09-21T05:47:38.077 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.075+0000 7f6d423f1640 1 -- start start 2024-09-21T05:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d423f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0b9b70 0x7f6d3c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d423f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0cf7a0 0x7f6d3c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d423f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d423f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c162da0 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d423f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3c162c20 con 0x7f6d3c0cf7a0 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d423f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6d3c162f20 con 0x7f6d3c0b9b70 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d4098e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d4098e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41240/0 (socket says 172.21.15.22:41240) 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d4098e640 1 -- 172.21.15.22:0/1446303763 learned_addr learned my addr 172.21.15.22:0/1446303763 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d3bfff640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0b9b70 0x7f6d3c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.076+0000 7f6d3b7fe640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0cf7a0 0x7f6d3c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d4098e640 1 -- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0b9b70 msgr2=0x7f6d3c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d4098e640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0b9b70 0x7f6d3c151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d4098e640 1 -- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0cf7a0 msgr2=0x7f6d3c0cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d4098e640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0cf7a0 0x7f6d3c0cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d4098e640 1 -- 172.21.15.22:0/1446303763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3c1f70e0 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d4098e640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0cbfd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400efc0 tx=0x7f6d3400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3401b070 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d34003b80 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3c0cb4f0 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34004030 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.077+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3c0cd060 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.082 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.078+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3c1f6e50 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.079+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d3401e070 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.079+0000 7f6d38ff9640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6d28078bc0 0x7f6d2807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.080+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f6d3409b3c0 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.080+0000 7f6d3bfff640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6d28078bc0 0x7f6d2807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.080+0000 7f6d3bfff640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6d28078bc0 0x7f6d2807b0d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c034000 tx=0x7f6d2c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:38.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.084+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6d340630e0 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.314+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d3c0cd2c0 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.314+0000 7f6d38ff9640 1 -- 172.21.15.22:0/1446303763 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+4392 (secure 0 0 0) 0x7f6d34067b10 con 0x7f6d3c0d2ed0 2024-09-21T05:47:38.316 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:38.316 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":91,"btime":"2024-09-21T05:47:37:661878+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:37.661874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T05:47:38.317 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 91 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6d28078bc0 msgr2=0x7f6d2807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6d28078bc0 0x7f6d2807b0d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c034000 tx=0x7f6d2c004730 comp rx=0 tx=0).stop 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 msgr2=0x7f6d3c0cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0cbfd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400efc0 tx=0x7f6d3400c800 comp rx=0 tx=0).stop 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 shutdown_connections 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6d3c0d2ed0 0x7f6d3c0cbfd0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6d3c0cf7a0 0x7f6d3c0cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6d28078bc0 0x7f6d2807b0d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 --2- 172.21.15.22:0/1446303763 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6d3c0b9b70 0x7f6d3c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 >> 172.21.15.22:0/1446303763 conn(0x7f6d3c08bfc0 msgr2=0x7f6d3c0d3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:38.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 shutdown_connections 2024-09-21T05:47:38.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.316+0000 7f6d423f1640 1 -- 172.21.15.22:0/1446303763 wait complete. 2024-09-21T05:47:38.330 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 -- 172.21.15.22:0/3488052352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc1570e0 msgr2=0x7f84cc0d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/3488052352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc1570e0 0x7f84cc0d1f20 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f84cc074e30 tx=0x7f84bc02a5f0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 -- 172.21.15.22:0/3488052352 shutdown_connections 2024-09-21T05:47:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/3488052352 >> v2:172.21.15.134:3300/0 conn(0x7f84cc15d840 0x7f84cc0b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/3488052352 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc1570e0 0x7f84cc0d1f20 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 -- 172.21.15.22:0/3488052352 >> 172.21.15.22:0/3488052352 conn(0x7f84cc06d550 msgr2=0x7f84cc071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 -- 172.21.15.22:0/3488052352 shutdown_connections 2024-09-21T05:47:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.445+0000 7f84d1fa1640 1 -- 172.21.15.22:0/3488052352 wait complete. 2024-09-21T05:47:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.446+0000 7f84d1fa1640 1 Processor -- start 2024-09-21T05:47:38.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.447+0000 7f84d1fa1640 1 -- start start 2024-09-21T05:47:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84d1fa1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 0x7f84cc0b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84d1fa1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84cc1570e0 0x7f84cc0b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84d1fa1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84cc15d840 0x7f84cc151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84d1fa1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc0b7d60 con 0x7f84cc0d2460 2024-09-21T05:47:38.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84d1fa1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc0b7be0 con 0x7f84cc15d840 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84d1fa1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f84cc0b7ee0 con 0x7f84cc1570e0 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 0x7f84cc0b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 0x7f84cc0b4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41250/0 (socket says 172.21.15.22:41250) 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 -- 172.21.15.22:0/2097804183 learned_addr learned my addr 172.21.15.22:0/2097804183 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84caffd640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84cc1570e0 0x7f84cc0b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cbfff640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84cc15d840 0x7f84cc151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 -- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84cc1570e0 msgr2=0x7f84cc0b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84cc1570e0 0x7f84cc0b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 -- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84cc15d840 msgr2=0x7f84cc151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84cc15d840 0x7f84cc151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.448+0000 7f84cb7fe640 1 -- 172.21.15.22:0/2097804183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc153c70 con 0x7f84cc0d2460 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.449+0000 7f84cb7fe640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 0x7f84cc0b4900 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f84bc034040 tx=0x7f84bc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:38.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.449+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc03e030 con 0x7f84cc0d2460 2024-09-21T05:47:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.449+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84bc0047a0 con 0x7f84cc0d2460 2024-09-21T05:47:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.449+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc035040 con 0x7f84cc0d2460 2024-09-21T05:47:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.449+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc151340 con 0x7f84cc0d2460 2024-09-21T05:47:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.449+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84cc151900 con 0x7f84cc0d2460 2024-09-21T05:47:38.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.450+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84cc0d1f20 con 0x7f84cc0d2460 2024-09-21T05:47:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.451+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f84bc041030 con 0x7f84cc0d2460 2024-09-21T05:47:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.451+0000 7f84b3fff640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84b8078b90 0x7f84b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.451+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f84bc0bb4a0 con 0x7f84cc0d2460 2024-09-21T05:47:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.451+0000 7f84caffd640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84b8078b90 0x7f84b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.452+0000 7f84caffd640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84b8078b90 0x7f84b807b0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0b4430 tx=0x7f84c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:38.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.456+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f84bc083270 con 0x7f84cc0d2460 2024-09-21T05:47:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.684+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84cc15f370 con 0x7f84cc0d2460 2024-09-21T05:47:38.686 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.684+0000 7f84b3fff640 1 -- 172.21.15.22:0/2097804183 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5214 (secure 0 0 0) 0x7f84bc087ca0 con 0x7f84cc0d2460 2024-09-21T05:47:38.686 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:38.687 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"btime":"2024-09-21T05:47:38:663520+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7908,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/199773408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":199773408},{"type":"v1","addr":"172.21.15.22:6834","nonce":199773408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:37.661874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T05:47:38.687 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 92 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.686+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84b8078b90 msgr2=0x7f84b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.686+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84b8078b90 0x7f84b807b0a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0b4430 tx=0x7f84c0009040 comp rx=0 tx=0).stop 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.686+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 msgr2=0x7f84cc0b4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.686+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 0x7f84cc0b4900 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f84bc034040 tx=0x7f84bc0039e0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 shutdown_connections 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f84cc15d840 0x7f84cc151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84b8078b90 0x7f84b807b0a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f84cc1570e0 0x7f84cc0b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 --2- 172.21.15.22:0/2097804183 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f84cc0d2460 0x7f84cc0b4900 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 >> 172.21.15.22:0/2097804183 conn(0x7f84cc06d550 msgr2=0x7f84cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 shutdown_connections 2024-09-21T05:47:38.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.687+0000 7f84d1fa1640 1 -- 172.21.15.22:0/2097804183 wait complete. 2024-09-21T05:47:38.700 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 -- 172.21.15.22:0/2653216675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00b7cc0 msgr2=0x7f05e00d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 --2- 172.21.15.22:0/2653216675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00b7cc0 0x7f05e00d0980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005fd0 tx=0x7f05d802aa90 comp rx=0 tx=0).stop 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 -- 172.21.15.22:0/2653216675 shutdown_connections 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 --2- 172.21.15.22:0/2653216675 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00b7cc0 0x7f05e00d0980 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 --2- 172.21.15.22:0/2653216675 >> v2:172.21.15.134:3300/0 conn(0x7f05e00cf320 0x7f05e00ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 -- 172.21.15.22:0/2653216675 >> 172.21.15.22:0/2653216675 conn(0x7f05e006d4f0 msgr2=0x7f05e0071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 -- 172.21.15.22:0/2653216675 shutdown_connections 2024-09-21T05:47:38.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.815+0000 7f05e5cde640 1 -- 172.21.15.22:0/2653216675 wait complete. 2024-09-21T05:47:38.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.816+0000 7f05e5cde640 1 Processor -- start 2024-09-21T05:47:38.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.817+0000 7f05e5cde640 1 -- start start 2024-09-21T05:47:38.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05e5cde640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 0x7f05e00cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05e5cde640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00cf320 0x7f05e00cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05e5cde640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e01600b0 0x7f05e0148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05e5cde640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f05e00d13c0 con 0x7f05e00b7cc0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05e5cde640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f05e00d1240 con 0x7f05e01600b0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05e5cde640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f05e00d1540 con 0x7f05e00cf320 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 0x7f05e00cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 0x7f05e00cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41260/0 (socket says 172.21.15.22:41260) 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 -- 172.21.15.22:0/595652932 learned_addr learned my addr 172.21.15.22:0/595652932 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05deffd640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00cf320 0x7f05e00cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05dffff640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e01600b0 0x7f05e0148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 -- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00cf320 msgr2=0x7f05e00cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00cf320 0x7f05e00cc400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 -- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e01600b0 msgr2=0x7f05e0148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.818+0000 7f05df7fe640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e01600b0 0x7f05e0148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05df7fe640 1 -- 172.21.15.22:0/595652932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e00cc940 con 0x7f05e00b7cc0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05df7fe640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 0x7f05e00cbec0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c310 tx=0x7f05d000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:38.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d00052d0 con 0x7f05e00b7cc0 2024-09-21T05:47:38.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05d0007020 con 0x7f05e00b7cc0 2024-09-21T05:47:38.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d0007470 con 0x7f05e00b7cc0 2024-09-21T05:47:38.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e015e3b0 con 0x7f05e00b7cc0 2024-09-21T05:47:38.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.819+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e015e970 con 0x7f05e00b7cc0 2024-09-21T05:47:38.823 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.820+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e00d1ad0 con 0x7f05e00b7cc0 2024-09-21T05:47:38.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.821+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05d00067d0 con 0x7f05e00b7cc0 2024-09-21T05:47:38.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.822+0000 7f05c7fff640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f05cc078bc0 0x7f05cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:38.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.822+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f05d009a1f0 con 0x7f05e00b7cc0 2024-09-21T05:47:38.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.822+0000 7f05deffd640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f05cc078bc0 0x7f05cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:38.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.822+0000 7f05deffd640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f05cc078bc0 0x7f05cc07b0d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f05e0051010 tx=0x7f05d4005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:38.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:38.826+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05d0061f30 con 0x7f05e00b7cc0 2024-09-21T05:47:39.016 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.014+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05e00b57a0 con 0x7f05e00b7cc0 2024-09-21T05:47:39.018 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.016+0000 7f05c7fff640 1 -- 172.21.15.22:0/595652932 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16915 (secure 0 0 0) 0x7f05d0066960 con 0x7f05e00b7cc0 2024-09-21T05:47:39.018 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:39.018 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":99,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:47:37.661699+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:46:15.254551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T05:46:16.522802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6832/3179586532":"2024-09-22T05:47:37.661668+0000","172.21.15.22:6834/1062944206":"2024-09-22T05:47:01.104934+0000","172.21.15.22:6834/3179586532":"2024-09-22T05:47:37.661668+0000","172.21.15.22:6832/1062944206":"2024-09-22T05:47:01.104934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:47:39.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f05cc078bc0 msgr2=0x7f05cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f05cc078bc0 0x7f05cc07b0d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f05e0051010 tx=0x7f05d4005f90 comp rx=0 tx=0).stop 2024-09-21T05:47:39.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 msgr2=0x7f05e00cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 0x7f05e00cbec0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c310 tx=0x7f05d000c7d0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.020 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 shutdown_connections 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f05e01600b0 0x7f05e0148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f05cc078bc0 0x7f05cc07b0d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f05e00cf320 0x7f05e00cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 --2- 172.21.15.22:0/595652932 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f05e00b7cc0 0x7f05e00cbec0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 >> 172.21.15.22:0/595652932 conn(0x7f05e006d4f0 msgr2=0x7f05e015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 shutdown_connections 2024-09-21T05:47:39.021 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.018+0000 7f05e5cde640 1 -- 172.21.15.22:0/595652932 wait complete. 2024-09-21T05:47:39.031 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:47:39.032 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.145+0000 7f92d6027640 1 -- 172.21.15.22:0/497787581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d00ccba0 msgr2=0x7f92d00ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.145+0000 7f92d6027640 1 --2- 172.21.15.22:0/497787581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d00ccba0 0x7f92d00ccfb0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f92c8005fd0 tx=0x7f92c802aa90 comp rx=0 tx=0).stop 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.146+0000 7f92d6027640 1 -- 172.21.15.22:0/497787581 shutdown_connections 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.146+0000 7f92d6027640 1 --2- 172.21.15.22:0/497787581 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d00ccba0 0x7f92d00ccfb0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.146+0000 7f92d6027640 1 --2- 172.21.15.22:0/497787581 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d00d1d40 0x7f92d00b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.146+0000 7f92d6027640 1 -- 172.21.15.22:0/497787581 >> 172.21.15.22:0/497787581 conn(0x7f92d00b3f10 msgr2=0x7f92d00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.146+0000 7f92d6027640 1 -- 172.21.15.22:0/497787581 shutdown_connections 2024-09-21T05:47:39.148 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.146+0000 7f92d6027640 1 -- 172.21.15.22:0/497787581 wait complete. 2024-09-21T05:47:39.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.147+0000 7f92d6027640 1 Processor -- start 2024-09-21T05:47:39.149 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.148+0000 7f92d6027640 1 -- start start 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92d6027640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d00ccba0 0x7f92d0157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92d6027640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d00d1d40 0x7f92d00cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92d6027640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 0x7f92d015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92d6027640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00d1300 con 0x7f92d015c6f0 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92d6027640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f92d00d1180 con 0x7f92d00d1d40 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92d6027640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f92d00d1480 con 0x7f92d00ccba0 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 0x7f92d015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 0x7f92d015a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41278/0 (socket says 172.21.15.22:41278) 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 -- 172.21.15.22:0/2189969564 learned_addr learned my addr 172.21.15.22:0/2189969564 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cf7fe640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d00ccba0 0x7f92d0157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92ceffd640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d00d1d40 0x7f92d00cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 -- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d00ccba0 msgr2=0x7f92d0157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d00ccba0 0x7f92d0157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 -- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d00d1d40 msgr2=0x7f92d00cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d00d1d40 0x7f92d00cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.149+0000 7f92cffff640 1 -- 172.21.15.22:0/2189969564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d01f45d0 con 0x7f92d015c6f0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.150+0000 7f92cffff640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 0x7f92d015a120 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009d90 tx=0x7f92c8003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.150+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8044070 con 0x7f92d015c6f0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.150+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92c8004250 con 0x7f92d015c6f0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.150+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d00ca800 con 0x7f92d015c6f0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.150+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004660 con 0x7f92d015c6f0 2024-09-21T05:47:39.153 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.150+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d00cadc0 con 0x7f92d015c6f0 2024-09-21T05:47:39.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.151+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92d00d1930 con 0x7f92d015c6f0 2024-09-21T05:47:39.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.152+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f92c8047070 con 0x7f92d015c6f0 2024-09-21T05:47:39.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.152+0000 7f92b7fff640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f92bc078b90 0x7f92bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.152+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f92c80bbda0 con 0x7f92d015c6f0 2024-09-21T05:47:39.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.152+0000 7f92cf7fe640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f92bc078b90 0x7f92bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.153+0000 7f92cf7fe640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f92bc078b90 0x7f92bc07b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f92c000bb50 tx=0x7f92c000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:39.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.157+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92c8032030 con 0x7f92d015c6f0 2024-09-21T05:47:39.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.386+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92d00b38b0 con 0x7f92d015c6f0 2024-09-21T05:47:39.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.387+0000 7f92b7fff640 1 -- 172.21.15.22:0/2189969564 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5214 (secure 0 0 0) 0x7f92c8083ae0 con 0x7f92d015c6f0 2024-09-21T05:47:39.389 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:39.389 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"btime":"2024-09-21T05:47:38:663520+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7908,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/199773408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":199773408},{"type":"v1","addr":"172.21.15.22:6834","nonce":199773408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:37.661874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T05:47:39.390 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 92 2024-09-21T05:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f92bc078b90 msgr2=0x7f92bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f92bc078b90 0x7f92bc07b0a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f92c000bb50 tx=0x7f92c000b2e0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 msgr2=0x7f92d015a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 0x7f92d015a120 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009d90 tx=0x7f92c8003ce0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 shutdown_connections 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f92d015c6f0 0x7f92d015a120 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f92d00d1d40 0x7f92d00cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f92bc078b90 0x7f92bc07b0a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 --2- 172.21.15.22:0/2189969564 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f92d00ccba0 0x7f92d0157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 >> 172.21.15.22:0/2189969564 conn(0x7f92d00b3f10 msgr2=0x7f92d00b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 shutdown_connections 2024-09-21T05:47:39.392 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.389+0000 7f92d6027640 1 -- 172.21.15.22:0/2189969564 wait complete. 2024-09-21T05:47:39.403 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.516+0000 7f4e7d739640 1 -- 172.21.15.22:0/373703009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e780736e0 msgr2=0x7f4e780caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.516+0000 7f4e7d739640 1 --2- 172.21.15.22:0/373703009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e780736e0 0x7f4e780caad0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f4e70005fd0 tx=0x7f4e7002aa90 comp rx=0 tx=0).stop 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.517+0000 7f4e7d739640 1 -- 172.21.15.22:0/373703009 shutdown_connections 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.517+0000 7f4e7d739640 1 --2- 172.21.15.22:0/373703009 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e780736e0 0x7f4e780caad0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.517+0000 7f4e7d739640 1 --2- 172.21.15.22:0/373703009 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e78156c50 0x7f4e780731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.517+0000 7f4e7d739640 1 -- 172.21.15.22:0/373703009 >> 172.21.15.22:0/373703009 conn(0x7f4e7806cde0 msgr2=0x7f4e7806d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:39.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.517+0000 7f4e7d739640 1 -- 172.21.15.22:0/373703009 shutdown_connections 2024-09-21T05:47:39.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.517+0000 7f4e7d739640 1 -- 172.21.15.22:0/373703009 wait complete. 2024-09-21T05:47:39.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.518+0000 7f4e7d739640 1 Processor -- start 2024-09-21T05:47:39.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.519+0000 7f4e7d739640 1 -- start start 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e7d739640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 0x7f4e7814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e7d739640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780736e0 0x7f4e781f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e7d739640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78156c50 0x7f4e781f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e7d739640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e780cb1d0 con 0x7f4e7806fad0 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e7d739640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e780cb050 con 0x7f4e78156c50 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e7d739640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4e780cb350 con 0x7f4e780736e0 2024-09-21T05:47:39.523 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 0x7f4e7814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 0x7f4e7814f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41304/0 (socket says 172.21.15.22:41304) 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 -- 172.21.15.22:0/2703260335 learned_addr learned my addr 172.21.15.22:0/2703260335 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e767fc640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780736e0 0x7f4e781f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e777fe640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78156c50 0x7f4e781f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 -- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780736e0 msgr2=0x7f4e781f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780736e0 0x7f4e781f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 -- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78156c50 msgr2=0x7f4e781f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78156c50 0x7f4e781f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 -- 172.21.15.22:0/2703260335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7814fa90 con 0x7f4e7806fad0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.520+0000 7f4e76ffd640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 0x7f4e7814f550 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c3a0 tx=0x7f4e6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.521+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68017070 con 0x7f4e7806fad0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.521+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e680053c0 con 0x7f4e7806fad0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.521+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e78151250 con 0x7f4e7806fad0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.521+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68003850 con 0x7f4e7806fad0 2024-09-21T05:47:39.524 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.521+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e78151810 con 0x7f4e7806fad0 2024-09-21T05:47:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.522+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e780cb800 con 0x7f4e7806fad0 2024-09-21T05:47:39.525 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.523+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e6801a070 con 0x7f4e7806fad0 2024-09-21T05:47:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.523+0000 7f4e5f7fe640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e64078bc0 0x7f4e6407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.523+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f4e68099fe0 con 0x7f4e7806fad0 2024-09-21T05:47:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.524+0000 7f4e767fc640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e64078bc0 0x7f4e6407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.526 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.524+0000 7f4e767fc640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e64078bc0 0x7f4e6407b0d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4e781438e0 tx=0x7f4e6c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:39.530 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.528+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e78073af0 con 0x7f4e7806fad0 2024-09-21T05:47:39.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.755+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e78073ff0 con 0x7f4e7806fad0 2024-09-21T05:47:39.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.755+0000 7f4e5f7fe640 1 -- 172.21.15.22:0/2703260335 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5214 (secure 0 0 0) 0x7f4e68061db0 con 0x7f4e7806fad0 2024-09-21T05:47:39.758 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:39.758 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":92,"btime":"2024-09-21T05:47:38:663520+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7908,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/199773408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":199773408},{"type":"v1","addr":"172.21.15.22:6834","nonce":199773408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":91,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:46:17.567401+0000","modified":"2024-09-21T05:47:37.661874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-21T05:47:39.758 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 92 2024-09-21T05:47:39.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.757+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e64078bc0 msgr2=0x7f4e6407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.757+0000 7f4e7d739640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e64078bc0 0x7f4e6407b0d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4e781438e0 tx=0x7f4e6c00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.757+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 msgr2=0x7f4e7814f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.757+0000 7f4e7d739640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 0x7f4e7814f550 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c3a0 tx=0x7f4e6800c860 comp rx=0 tx=0).stop 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 shutdown_connections 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e78156c50 0x7f4e781f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e64078bc0 0x7f4e6407b0d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e780736e0 0x7f4e781f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 --2- 172.21.15.22:0/2703260335 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e7806fad0 0x7f4e7814f550 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 >> 172.21.15.22:0/2703260335 conn(0x7f4e7806cde0 msgr2=0x7f4e780c58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 shutdown_connections 2024-09-21T05:47:39.762 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.758+0000 7f4e7d739640 1 -- 172.21.15.22:0/2703260335 wait complete. 2024-09-21T05:47:39.771 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 -- 172.21.15.22:0/452595051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd80d0e50 msgr2=0x7f3dd80d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 --2- 172.21.15.22:0/452595051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd80d0e50 0x7f3dd80d1260 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0005fd0 tx=0x7f3dd002a980 comp rx=0 tx=0).stop 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 -- 172.21.15.22:0/452595051 shutdown_connections 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 --2- 172.21.15.22:0/452595051 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd80d0e50 0x7f3dd80d1260 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 --2- 172.21.15.22:0/452595051 >> v2:172.21.15.134:3300/0 conn(0x7f3dd80b97e0 0x7f3dd806f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 -- 172.21.15.22:0/452595051 >> 172.21.15.22:0/452595051 conn(0x7f3dd809d840 msgr2=0x7f3dd808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 -- 172.21.15.22:0/452595051 shutdown_connections 2024-09-21T05:47:39.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.885+0000 7f3ddefad640 1 -- 172.21.15.22:0/452595051 wait complete. 2024-09-21T05:47:39.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.887+0000 7f3ddefad640 1 Processor -- start 2024-09-21T05:47:39.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.887+0000 7f3ddefad640 1 -- start start 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddefad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd806e0a0 0x7f3dd8160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddefad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 0x7f3dd80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddefad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd80d0e50 0x7f3dd80b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddefad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80cfea0 con 0x7f3dd80b97e0 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddefad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd80cfd20 con 0x7f3dd80d0e50 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddefad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd80d0020 con 0x7f3dd806e0a0 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 0x7f3dd80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 0x7f3dd80b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41320/0 (socket says 172.21.15.22:41320) 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 -- 172.21.15.22:0/1768499722 learned_addr learned my addr 172.21.15.22:0/1768499722 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddcd49640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd806e0a0 0x7f3dd8160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3ddd54a640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd80d0e50 0x7f3dd80b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 -- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd806e0a0 msgr2=0x7f3dd8160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd806e0a0 0x7f3dd8160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 -- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd80d0e50 msgr2=0x7f3dd80b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd80d0e50 0x7f3dd80b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 -- 172.21.15.22:0/1768499722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd815a3e0 con 0x7f3dd80b97e0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.889+0000 7f3dd7fff640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 0x7f3dd80b4ee0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00e7c0 tx=0x7f3dcc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.890+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00ccc0 con 0x7f3dd80b97e0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.890+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dcc00ee70 con 0x7f3dd80b97e0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.890+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd815a8e0 con 0x7f3dd80b97e0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.890+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00ce60 con 0x7f3dd80b97e0 2024-09-21T05:47:39.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.890+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd815cf90 con 0x7f3dd80b97e0 2024-09-21T05:47:39.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.891+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd815a630 con 0x7f3dd80b97e0 2024-09-21T05:47:39.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.892+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3dcc004260 con 0x7f3dd80b97e0 2024-09-21T05:47:39.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.892+0000 7f3dd57fa640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc4078b90 0x7f3dc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:39.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.892+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f3dcc09b650 con 0x7f3dd80b97e0 2024-09-21T05:47:39.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.892+0000 7f3ddcd49640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc4078b90 0x7f3dc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:39.895 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.893+0000 7f3ddcd49640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc4078b90 0x7f3dc407b0a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8007fd0 tx=0x7f3dc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:39.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:39.897+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3dcc063310 con 0x7f3dd80b97e0 2024-09-21T05:47:40.088 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.086+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dd815caa0 con 0x7f3dd80b97e0 2024-09-21T05:47:40.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.087+0000 7f3dd57fa640 1 -- 172.21.15.22:0/1768499722 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+16915 (secure 0 0 0) 0x7f3dcc067d40 con 0x7f3dd80b97e0 2024-09-21T05:47:40.089 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:40.089 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":99,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:47:37.661699+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:46:15.254551+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-21T05:46:16.522802+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6832/3179586532":"2024-09-22T05:47:37.661668+0000","172.21.15.22:6834/1062944206":"2024-09-22T05:47:01.104934+0000","172.21.15.22:6834/3179586532":"2024-09-22T05:47:37.661668+0000","172.21.15.22:6832/1062944206":"2024-09-22T05:47:01.104934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:47:40.091 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc4078b90 msgr2=0x7f3dc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc4078b90 0x7f3dc407b0a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8007fd0 tx=0x7f3dc8009040 comp rx=0 tx=0).stop 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 msgr2=0x7f3dd80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 0x7f3dd80b4ee0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00e7c0 tx=0x7f3dcc00ec80 comp rx=0 tx=0).stop 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 shutdown_connections 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3dd80d0e50 0x7f3dd80b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3dd80b97e0 0x7f3dd80b4ee0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3dc4078b90 0x7f3dc407b0a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 --2- 172.21.15.22:0/1768499722 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3dd806e0a0 0x7f3dd8160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 >> 172.21.15.22:0/1768499722 conn(0x7f3dd809d840 msgr2=0x7f3dd8068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 shutdown_connections 2024-09-21T05:47:40.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.089+0000 7f3ddefad640 1 -- 172.21.15.22:0/1768499722 wait complete. 2024-09-21T05:47:40.103 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:47:40.220 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 -- 172.21.15.22:0/1403951191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240ccbd0 msgr2=0x7fe2240d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/1403951191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240ccbd0 0x7fe2240d0d80 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fe224073f50 tx=0x7fe21802a560 comp rx=0 tx=0).stop 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 -- 172.21.15.22:0/1403951191 shutdown_connections 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/1403951191 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240ccbd0 0x7fe2240d0d80 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/1403951191 >> v2:172.21.15.134:3300/0 conn(0x7fe2240b51c0 0x7fe22415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 -- 172.21.15.22:0/1403951191 >> 172.21.15.22:0/1403951191 conn(0x7fe224072ae0 msgr2=0x7fe224072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 -- 172.21.15.22:0/1403951191 shutdown_connections 2024-09-21T05:47:40.221 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.218+0000 7fe22bfc2640 1 -- 172.21.15.22:0/1403951191 wait complete. 2024-09-21T05:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.220+0000 7fe22bfc2640 1 Processor -- start 2024-09-21T05:47:40.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.220+0000 7fe22bfc2640 1 -- start start 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.221+0000 7fe22bfc2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 0x7fe2240c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.221+0000 7fe22bfc2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2240ccbd0 0x7fe22415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.221+0000 7fe22bfc2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2240d12c0 0x7fe2240c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.221+0000 7fe22bfc2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe2240d19f0 con 0x7fe2240b51c0 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.221+0000 7fe22bfc2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe224160a60 con 0x7fe2240ccbd0 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.221+0000 7fe22bfc2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe2240ca620 con 0x7fe2240d12c0 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 0x7fe2240c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:40.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 0x7fe2240c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41342/0 (socket says 172.21.15.22:41342) 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 -- 172.21.15.22:0/3099207877 learned_addr learned my addr 172.21.15.22:0/3099207877 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe22955d640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2240ccbd0 0x7fe22415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe22a55f640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2240d12c0 0x7fe2240c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 -- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2240d12c0 msgr2=0x7fe2240c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2240d12c0 0x7fe2240c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 -- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2240ccbd0 msgr2=0x7fe22415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2240ccbd0 0x7fe22415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 -- 172.21.15.22:0/3099207877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2240b1130 con 0x7fe2240b51c0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe229d5e640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 0x7fe2240c9d20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d770 tx=0x7fe21c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c017070 con 0x7fe2240b51c0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe21c009d20 con 0x7fe2240b51c0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21c0106c0 con 0x7fe2240b51c0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.222+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2240b1410 con 0x7fe2240b51c0 2024-09-21T05:47:40.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.223+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2240ca360 con 0x7fe2240b51c0 2024-09-21T05:47:40.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.224+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2240b5960 con 0x7fe2240b51c0 2024-09-21T05:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.224+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe21c01a070 con 0x7fe2240b51c0 2024-09-21T05:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.225+0000 7fe2227fc640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe210078b90 0x7fe21007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.225+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7fe21c09afb0 con 0x7fe2240b51c0 2024-09-21T05:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.225+0000 7fe22955d640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe210078b90 0x7fe21007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:40.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.226+0000 7fe22955d640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe210078b90 0x7fe21007b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe218033000 tx=0x7fe218004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:40.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.230+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe2240b5960 con 0x7fe2240b51c0 2024-09-21T05:47:40.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:40.454+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe224160f10 con 0x7fe2240b51c0 2024-09-21T05:47:41.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.197+0000 7fe2227fc640 1 -- 172.21.15.22:0/3099207877 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7fe21c062c50 con 0x7fe2240b51c0 2024-09-21T05:47:41.199 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.199+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe210078b90 msgr2=0x7fe21007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.199+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe210078b90 0x7fe21007b0a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe218033000 tx=0x7fe218004730 comp rx=0 tx=0).stop 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 msgr2=0x7fe2240c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 0x7fe2240c9d20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fe21c00d770 tx=0x7fe21c00dc30 comp rx=0 tx=0).stop 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 shutdown_connections 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe210078b90 0x7fe21007b0a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe2240d12c0 0x7fe2240c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.202 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe2240ccbd0 0x7fe22415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 --2- 172.21.15.22:0/3099207877 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe2240b51c0 0x7fe2240c9d20 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 >> 172.21.15.22:0/3099207877 conn(0x7fe224072ae0 msgr2=0x7fe2240b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:41.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 shutdown_connections 2024-09-21T05:47:41.203 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.200+0000 7fe22bfc2640 1 -- 172.21.15.22:0/3099207877 wait complete. 2024-09-21T05:47:41.215 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:47:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 -- 172.21.15.22:0/3518107435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b9b60 msgr2=0x7f09f80d0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 --2- 172.21.15.22:0/3518107435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b9b60 0x7f09f80d0fb0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009830 tx=0x7f09e802a480 comp rx=0 tx=0).stop 2024-09-21T05:47:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 -- 172.21.15.22:0/3518107435 shutdown_connections 2024-09-21T05:47:41.341 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 --2- 172.21.15.22:0/3518107435 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f80b3a10 0x7f09f80d14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 --2- 172.21.15.22:0/3518107435 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b9b60 0x7f09f80d0fb0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 -- 172.21.15.22:0/3518107435 >> 172.21.15.22:0/3518107435 conn(0x7f09f809d8f0 msgr2=0x7f09f80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 -- 172.21.15.22:0/3518107435 shutdown_connections 2024-09-21T05:47:41.342 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.339+0000 7f09fde9b640 1 -- 172.21.15.22:0/3518107435 wait complete. 2024-09-21T05:47:41.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.341+0000 7f09fde9b640 1 Processor -- start 2024-09-21T05:47:41.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.341+0000 7f09fde9b640 1 -- start start 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09fde9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 0x7f09f80b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09fde9b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f80b9b60 0x7f09f81576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09fde9b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8162430 0x7f09f815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09fde9b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f09f81630d0 con 0x7f09f80b3a10 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09fde9b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f09f8162f50 con 0x7f09f8162430 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09fde9b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f09f8163250 con 0x7f09f80b9b60 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 0x7f09f80b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09f77fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 0x7f09f80b66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41372/0 (socket says 172.21.15.22:41372) 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.342+0000 7f09f77fe640 1 -- 172.21.15.22:0/1445473292 learned_addr learned my addr 172.21.15.22:0/1445473292 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f7fff640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8162430 0x7f09f815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:41.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f6ffd640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f80b9b60 0x7f09f81576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f77fe640 1 -- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f80b9b60 msgr2=0x7f09f81576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f77fe640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f80b9b60 0x7f09f81576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f77fe640 1 -- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8162430 msgr2=0x7f09f815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f77fe640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8162430 0x7f09f815e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f77fe640 1 -- 172.21.15.22:0/1445473292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f8157ed0 con 0x7f09f80b3a10 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09f77fe640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 0x7f09f80b66e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f09e8000c00 tx=0x7f09e8033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e8005c90 con 0x7f09f80b3a10 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e8031e50 con 0x7f09f80b3a10 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e80332f0 con 0x7f09f80b3a10 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f80b6c20 con 0x7f09f80b3a10 2024-09-21T05:47:41.347 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.343+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f81f9930 con 0x7f09f80b3a10 2024-09-21T05:47:41.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.344+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f81580a0 con 0x7f09f80b3a10 2024-09-21T05:47:41.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.345+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09e8033490 con 0x7f09f80b3a10 2024-09-21T05:47:41.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.346+0000 7f09dffff640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f09e4078b90 0x7f09e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:41.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.346+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f09e80bc210 con 0x7f09f80b3a10 2024-09-21T05:47:41.348 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.346+0000 7f09f6ffd640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f09e4078b90 0x7f09e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:41.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.347+0000 7f09f6ffd640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f09e4078b90 0x7f09e407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f09f80cea30 tx=0x7f09ec00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:41.354 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.350+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09e803f030 con 0x7f09f80b3a10 2024-09-21T05:47:41.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:41.577+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f09f8050fd0 con 0x7f09f80b3a10 2024-09-21T05:47:42.204 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.202+0000 7f09dffff640 1 -- 172.21.15.22:0/1445473292 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7f09f8050fd0 con 0x7f09f80b3a10 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f09e4078b90 msgr2=0x7f09e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f09e4078b90 0x7f09e407b0a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f09f80cea30 tx=0x7f09ec00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 msgr2=0x7f09f80b66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 0x7f09f80b66e0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f09e8000c00 tx=0x7f09e8033c90 comp rx=0 tx=0).stop 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 shutdown_connections 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f09f8162430 0x7f09f815e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.207 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f09e4078b90 0x7f09e407b0a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f09f80b9b60 0x7f09f81576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 --2- 172.21.15.22:0/1445473292 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f09f80b3a10 0x7f09f80b66e0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 >> 172.21.15.22:0/1445473292 conn(0x7f09f809d8f0 msgr2=0x7f09f80b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 shutdown_connections 2024-09-21T05:47:42.208 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.205+0000 7f09fde9b640 1 -- 172.21.15.22:0/1445473292 wait complete. 2024-09-21T05:47:42.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:47:42.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.348+0000 7f1d3029b640 1 -- 172.21.15.22:0/3669334171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280cf540 msgr2=0x7f1d2815e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:42.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.348+0000 7f1d3029b640 1 --2- 172.21.15.22:0/3669334171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280cf540 0x7f1d2815e870 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f1d280cff70 tx=0x7f1d1802a560 comp rx=0 tx=0).stop 2024-09-21T05:47:42.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.349+0000 7f1d3029b640 1 -- 172.21.15.22:0/3669334171 shutdown_connections 2024-09-21T05:47:42.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.349+0000 7f1d3029b640 1 --2- 172.21.15.22:0/3669334171 >> v2:172.21.15.134:3300/0 conn(0x7f1d280d0ef0 0x7f1d280d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.349+0000 7f1d3029b640 1 --2- 172.21.15.22:0/3669334171 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280cf540 0x7f1d2815e870 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.351 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.349+0000 7f1d3029b640 1 -- 172.21.15.22:0/3669334171 >> 172.21.15.22:0/3669334171 conn(0x7f1d2808dc50 msgr2=0x7f1d2809d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.349+0000 7f1d3029b640 1 -- 172.21.15.22:0/3669334171 shutdown_connections 2024-09-21T05:47:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.349+0000 7f1d3029b640 1 -- 172.21.15.22:0/3669334171 wait complete. 2024-09-21T05:47:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.350+0000 7f1d3029b640 1 Processor -- start 2024-09-21T05:47:42.352 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.351+0000 7f1d3029b640 1 -- start start 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d3029b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d2806fa70 0x7f1d2815c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d3029b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d280cf540 0x7f1d28158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d3029b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 0x7f1d280cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d3029b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28162b20 con 0x7f1d280d0ef0 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d3029b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f1d281629a0 con 0x7f1d280cf540 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d3029b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f1d28162ca0 con 0x7f1d2806fa70 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 0x7f1d280cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 0x7f1d280cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41392/0 (socket says 172.21.15.22:41392) 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 -- 172.21.15.22:0/494915301 learned_addr learned my addr 172.21.15.22:0/494915301 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2d836640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d280cf540 0x7f1d28158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:42.355 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e037640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d2806fa70 0x7f1d2815c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 -- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d2806fa70 msgr2=0x7f1d2815c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d2806fa70 0x7f1d2815c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 -- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d280cf540 msgr2=0x7f1d28158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d280cf540 0x7f1d28158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.352+0000 7f1d2e838640 1 -- 172.21.15.22:0/494915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d280a1b50 con 0x7f1d280d0ef0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.353+0000 7f1d2e838640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 0x7f1d280cca80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000e960 tx=0x7f1d2000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.353+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004d20 con 0x7f1d280d0ef0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.353+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d2000cb20 con 0x7f1d280d0ef0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.353+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d280a1e30 con 0x7f1d280d0ef0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.353+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20005020 con 0x7f1d280d0ef0 2024-09-21T05:47:42.356 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.353+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d28158dc0 con 0x7f1d280d0ef0 2024-09-21T05:47:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.354+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d28163170 con 0x7f1d280d0ef0 2024-09-21T05:47:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.354+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1d200051c0 con 0x7f1d280d0ef0 2024-09-21T05:47:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.355+0000 7f1d1effd640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d0c078b90 0x7f1d0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.355+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5881+0+0 (secure 0 0 0) 0x7f1d2009b660 con 0x7f1d280d0ef0 2024-09-21T05:47:42.357 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.355+0000 7f1d2e037640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d0c078b90 0x7f1d0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:42.358 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.356+0000 7f1d2e037640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d0c078b90 0x7f1d0c07b0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1d24001720 tx=0x7f1d24009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:42.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.360+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d20016030 con 0x7f1d280d0ef0 2024-09-21T05:47:42.552 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:42.550+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1d2806e3b0 con 0x7f1d280d0ef0 2024-09-21T05:47:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.240+0000 7f1d1effd640 1 -- 172.21.15.22:0/494915301 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) ==== 181+0+0 (secure 0 0 0) 0x7f1d20063440 con 0x7f1d280d0ef0 2024-09-21T05:47:43.242 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d0c078b90 msgr2=0x7f1d0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d0c078b90 0x7f1d0c07b0a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1d24001720 tx=0x7f1d24009040 comp rx=0 tx=0).stop 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 msgr2=0x7f1d280cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 0x7f1d280cca80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000e960 tx=0x7f1d2000ee20 comp rx=0 tx=0).stop 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 shutdown_connections 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1d280d0ef0 0x7f1d280cca80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f1d280cf540 0x7f1d28158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1d0c078b90 0x7f1d0c07b0a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 --2- 172.21.15.22:0/494915301 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f1d2806fa70 0x7f1d2815c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.244+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 >> 172.21.15.22:0/494915301 conn(0x7f1d2808dc50 msgr2=0x7f1d280b9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:43.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.245+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 shutdown_connections 2024-09-21T05:47:43.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.245+0000 7f1d3029b640 1 -- 172.21.15.22:0/494915301 wait complete. 2024-09-21T05:47:43.261 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1400786029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 msgr2=0x7fd6e80d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1400786029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 0x7fd6e80d1b60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8005fd0 tx=0x7fd6d802a5d0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1400786029 shutdown_connections 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1400786029 >> v2:172.21.15.134:3300/0 conn(0x7fd6e80cff30 0x7fd6e815c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1400786029 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 0x7fd6e80d1b60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1400786029 >> 172.21.15.22:0/1400786029 conn(0x7fd6e8072a90 msgr2=0x7fd6e8072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.387+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1400786029 shutdown_connections 2024-09-21T05:47:43.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.388+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1400786029 wait complete. 2024-09-21T05:47:43.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.389+0000 7fd6ecbad640 1 Processor -- start 2024-09-21T05:47:43.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.389+0000 7fd6ecbad640 1 -- start start 2024-09-21T05:47:43.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6ecbad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6e80cff30 0x7fd6e8150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:43.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6ecbad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 0x7fd6e80b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:43.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6ecbad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 0x7fd6e80b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:43.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6ecbad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e815d460 con 0x7fd6e80d0340 2024-09-21T05:47:43.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6ecbad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e815d2e0 con 0x7fd6e80cff30 2024-09-21T05:47:43.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6ecbad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e815d5e0 con 0x7fd6e81601c0 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 0x7fd6e80b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 0x7fd6e80b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:41398/0 (socket says 172.21.15.22:41398) 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 -- 172.21.15.22:0/1899200604 learned_addr learned my addr 172.21.15.22:0/1899200604 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e659c640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6e80cff30 0x7fd6e8150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e6d9d640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 0x7fd6e80b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 -- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 msgr2=0x7fd6e80b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 0x7fd6e80b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 -- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6e80cff30 msgr2=0x7fd6e8150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6e80cff30 0x7fd6e8150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.391+0000 7fd6e5d9b640 1 -- 172.21.15.22:0/1899200604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e80b2d60 con 0x7fd6e80d0340 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.392+0000 7fd6e5d9b640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 0x7fd6e80b66c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009fd0 tx=0x7fd6dc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.392+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00ccc0 con 0x7fd6e80d0340 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.392+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6dc00ee70 con 0x7fd6e80d0340 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.392+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00ce60 con 0x7fd6e80d0340 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.392+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e80b2150 con 0x7fd6e80d0340 2024-09-21T05:47:43.396 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.392+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e80b2710 con 0x7fd6e80d0340 2024-09-21T05:47:43.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.393+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e815da90 con 0x7fd6e80d0340 2024-09-21T05:47:43.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.394+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd6dc004280 con 0x7fd6e80d0340 2024-09-21T05:47:43.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.394+0000 7fd6d6ffd640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd6c8078b90 0x7fd6c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:43.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.394+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5443+0+0 (secure 0 0 0) 0x7fd6dc09b4f0 con 0x7fd6e80d0340 2024-09-21T05:47:43.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.395+0000 7fd6e659c640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd6c8078b90 0x7fd6c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:43.397 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.395+0000 7fd6e659c640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd6c8078b90 0x7fd6c807b0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8033000 tx=0x7fd6d8004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:43.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.399+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6dc063360 con 0x7fd6e80d0340 2024-09-21T05:47:43.589 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:43.587+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd6e815ea30 con 0x7fd6e80d0340 2024-09-21T05:47:44.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.248+0000 7fd6d6ffd640 1 -- 172.21.15.22:0/1899200604 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) ==== 169+0+0 (secure 0 0 0) 0x7fd6dc067d90 con 0x7fd6e80d0340 2024-09-21T05:47:44.250 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.252+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd6c8078b90 msgr2=0x7fd6c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.252+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd6c8078b90 0x7fd6c807b0a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8033000 tx=0x7fd6d8004730 comp rx=0 tx=0).stop 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 msgr2=0x7fd6e80b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 0x7fd6e80b66c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009fd0 tx=0x7fd6dc00ec20 comp rx=0 tx=0).stop 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 shutdown_connections 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd6c8078b90 0x7fd6c807b0a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd6e81601c0 0x7fd6e80b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd6e80d0340 0x7fd6e80b66c0 secure :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009fd0 tx=0x7fd6dc00ec20 comp rx=0 tx=0).stop 2024-09-21T05:47:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 --2- 172.21.15.22:0/1899200604 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd6e80cff30 0x7fd6e8150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 >> 172.21.15.22:0/1899200604 conn(0x7fd6e8072a90 msgr2=0x7fd6e80c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 shutdown_connections 2024-09-21T05:47:44.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.253+0000 7fd6ecbad640 1 -- 172.21.15.22:0/1899200604 wait complete. 2024-09-21T05:47:44.270 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:47:44.393 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 -- 172.21.15.22:0/3955895538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b3a10 msgr2=0x7f53b80d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/3955895538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b3a10 0x7f53b80d14f0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f53b80d4470 tx=0x7f53ac02a4d0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 -- 172.21.15.22:0/3955895538 shutdown_connections 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/3955895538 >> v2:172.21.15.134:3300/0 conn(0x7f53b8162430 0x7f53b8162840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/3955895538 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b3a10 0x7f53b80d14f0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/3955895538 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b80b9b60 0x7f53b80d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 -- 172.21.15.22:0/3955895538 >> 172.21.15.22:0/3955895538 conn(0x7f53b809d8f0 msgr2=0x7f53b80d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 -- 172.21.15.22:0/3955895538 shutdown_connections 2024-09-21T05:47:44.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.391+0000 7f53bdaa9640 1 -- 172.21.15.22:0/3955895538 wait complete. 2024-09-21T05:47:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.393+0000 7f53bdaa9640 1 Processor -- start 2024-09-21T05:47:44.395 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.393+0000 7f53bdaa9640 1 -- start start 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53bdaa9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80b3a10 0x7f53b80ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53bdaa9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 0x7f53b81576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53bdaa9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b8162430 0x7f53b80c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53bdaa9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53b8163110 con 0x7f53b80b9b60 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53bdaa9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53b8162f90 con 0x7f53b80b3a10 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53bdaa9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53b8163290 con 0x7f53b8162430 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 0x7f53b81576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.398 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 0x7f53b81576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33670/0 (socket says 172.21.15.22:33670) 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 -- 172.21.15.22:0/1372339061 learned_addr learned my addr 172.21.15.22:0/1372339061 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b7fff640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b8162430 0x7f53b80c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b77fe640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80b3a10 0x7f53b80ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 -- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b8162430 msgr2=0x7f53b80c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b8162430 0x7f53b80c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 -- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80b3a10 msgr2=0x7f53b80ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80b3a10 0x7f53b80ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 -- 172.21.15.22:0/1372339061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b81f9460 con 0x7f53b80b9b60 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b7fff640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b8162430 0x7f53b80c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b77fe640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80b3a10 0x7f53b80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f53b6ffd640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 0x7f53b81576b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f53ac034040 tx=0x7f53ac0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:44.399 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac03e030 con 0x7f53b80b9b60 2024-09-21T05:47:44.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.395+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53ac0047a0 con 0x7f53b80b9b60 2024-09-21T05:47:44.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.396+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b81f9740 con 0x7f53b80b9b60 2024-09-21T05:47:44.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.396+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac035040 con 0x7f53b80b9b60 2024-09-21T05:47:44.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.396+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b81f9d00 con 0x7f53b80b9b60 2024-09-21T05:47:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.396+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b8162840 con 0x7f53b80b9b60 2024-09-21T05:47:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.397+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53ac041030 con 0x7f53b80b9b60 2024-09-21T05:47:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.398+0000 7f539ffff640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53a4078b90 0x7f53a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.398+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f53ac0bb320 con 0x7f53b80b9b60 2024-09-21T05:47:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.398+0000 7f53b77fe640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53a4078b90 0x7f53a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.399+0000 7f53b77fe640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53a4078b90 0x7f53a407b0a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f53a800ba00 tx=0x7f53a8005c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:44.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.403+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53ac083a10 con 0x7f53b80b9b60 2024-09-21T05:47:44.595 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.593+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f53b80ba0d0 con 0x7f53b80b9b60 2024-09-21T05:47:44.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.597+0000 7f539ffff640 1 -- 172.21.15.22:0/1372339061 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f53ac083310 con 0x7f53b80b9b60 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53a4078b90 msgr2=0x7f53a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53a4078b90 0x7f53a407b0a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f53a800ba00 tx=0x7f53a8005c80 comp rx=0 tx=0).stop 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 msgr2=0x7f53b81576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 0x7f53b81576b0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f53ac034040 tx=0x7f53ac0039e0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 shutdown_connections 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53a4078b90 0x7f53a407b0a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53b8162430 0x7f53b80c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53b80b9b60 0x7f53b81576b0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 --2- 172.21.15.22:0/1372339061 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53b80b3a10 0x7f53b80ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 >> 172.21.15.22:0/1372339061 conn(0x7f53b809d8f0 msgr2=0x7f53b80b3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:44.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 shutdown_connections 2024-09-21T05:47:44.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.599+0000 7f53bdaa9640 1 -- 172.21.15.22:0/1372339061 wait complete. 2024-09-21T05:47:44.613 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-09-21T05:47:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 -- 172.21.15.22:0/2199034577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38140d4d80 msgr2=0x7f38140cf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 --2- 172.21.15.22:0/2199034577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38140d4d80 0x7f38140cf540 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f380c009860 tx=0x7f380c02a420 comp rx=0 tx=0).stop 2024-09-21T05:47:44.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 -- 172.21.15.22:0/2199034577 shutdown_connections 2024-09-21T05:47:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 --2- 172.21.15.22:0/2199034577 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38140d4d80 0x7f38140cf540 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 -- 172.21.15.22:0/2199034577 >> 172.21.15.22:0/2199034577 conn(0x7f381409d7f0 msgr2=0x7f38140d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 -- 172.21.15.22:0/2199034577 shutdown_connections 2024-09-21T05:47:44.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.727+0000 7f38196de640 1 -- 172.21.15.22:0/2199034577 wait complete. 2024-09-21T05:47:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.729+0000 7f38196de640 1 Processor -- start 2024-09-21T05:47:44.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.729+0000 7f38196de640 1 -- start start 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38196de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38140b9b60 0x7f38140ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38196de640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38140d4d80 0x7f3814150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38196de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 0x7f38140b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38196de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f381406f7f0 con 0x7f38141588c0 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38196de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f381406f670 con 0x7f38140b9b60 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38196de640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f381406f970 con 0x7f38140d4d80 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 0x7f38140b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38137fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 0x7f38140b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33690/0 (socket says 172.21.15.22:33690) 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38137fe640 1 -- 172.21.15.22:0/3303842762 learned_addr learned my addr 172.21.15.22:0/3303842762 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f3812ffd640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38140b9b60 0x7f38140ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.730+0000 7f38127fc640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38140d4d80 0x7f3814150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38137fe640 1 -- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38140d4d80 msgr2=0x7f3814150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38137fe640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38140d4d80 0x7f3814150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38137fe640 1 -- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38140b9b60 msgr2=0x7f38140ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38137fe640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38140b9b60 0x7f38140ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38137fe640 1 -- 172.21.15.22:0/3303842762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38141f8d90 con 0x7f38141588c0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38127fc640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38140d4d80 0x7f3814150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f3812ffd640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38140b9b60 0x7f38140ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38137fe640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 0x7f38140b4fc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f380c02aa10 tx=0x7f380c033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c031d60 con 0x7f38141588c0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f380c0330d0 con 0x7f38141588c0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38141f9070 con 0x7f38141588c0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c033500 con 0x7f38141588c0 2024-09-21T05:47:44.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.731+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38141f9630 con 0x7f38141588c0 2024-09-21T05:47:44.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.732+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38140cf540 con 0x7f38141588c0 2024-09-21T05:47:44.736 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.733+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f380c0035f0 con 0x7f38141588c0 2024-09-21T05:47:44.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.733+0000 7f37fb7fe640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3800078b90 0x7f380007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:44.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.734+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f380c0bc2d0 con 0x7f38141588c0 2024-09-21T05:47:44.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.734+0000 7f3812ffd640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3800078b90 0x7f380007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:44.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.734+0000 7f3812ffd640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3800078b90 0x7f380007b0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3804009fc0 tx=0x7f3804009550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:44.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:44.737+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f380c03f030 con 0x7f38141588c0 2024-09-21T05:47:45.004 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.002+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f38140d3fb0 con 0x7f38141588c0 2024-09-21T05:47:45.251 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.249+0000 7f37fb7fe640 1 -- 172.21.15.22:0/3303842762 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v388) ==== 146+0+0 (secure 0 0 0) 0x7f380c084b20 con 0x7f38141588c0 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3800078b90 msgr2=0x7f380007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3800078b90 0x7f380007b0a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3804009fc0 tx=0x7f3804009550 comp rx=0 tx=0).stop 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 msgr2=0x7f38140b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 0x7f38140b4fc0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f380c02aa10 tx=0x7f380c033c90 comp rx=0 tx=0).stop 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 shutdown_connections 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f38141588c0 0x7f38140b4fc0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3800078b90 0x7f380007b0a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f38140d4d80 0x7f3814150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 --2- 172.21.15.22:0/3303842762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f38140b9b60 0x7f38140ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 >> 172.21.15.22:0/3303842762 conn(0x7f381409d7f0 msgr2=0x7f3814161990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:45.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 shutdown_connections 2024-09-21T05:47:45.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.252+0000 7f38196de640 1 -- 172.21.15.22:0/3303842762 wait complete. 2024-09-21T05:47:45.266 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-21T05:47:45.266 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-21T05:47:45.266 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.381+0000 7f0d566f9640 1 -- 172.21.15.22:0/564710978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c0053f0 msgr2=0x7f0d3c00ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.381+0000 7f0d566f9640 1 --2- 172.21.15.22:0/564710978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c0053f0 0x7f0d3c00ba70 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800c610 tx=0x7f0d3c005830 comp rx=0 tx=0).stop 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.382+0000 7f0d566f9640 1 -- 172.21.15.22:0/564710978 shutdown_connections 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.382+0000 7f0d566f9640 1 --2- 172.21.15.22:0/564710978 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d3c0053f0 0x7f0d3c00ba70 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.382+0000 7f0d566f9640 1 -- 172.21.15.22:0/564710978 >> 172.21.15.22:0/564710978 conn(0x7f0d5009d7f0 msgr2=0x7f0d500d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.382+0000 7f0d566f9640 1 -- 172.21.15.22:0/564710978 shutdown_connections 2024-09-21T05:47:45.384 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.382+0000 7f0d566f9640 1 -- 172.21.15.22:0/564710978 wait complete. 2024-09-21T05:47:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.383+0000 7f0d566f9640 1 Processor -- start 2024-09-21T05:47:45.385 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.383+0000 7f0d566f9640 1 -- start start 2024-09-21T05:47:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d566f9640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d50152a30 0x7f0d501f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d566f9640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d501f6db0 0x7f0d500b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d566f9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 0x7f0d500cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d566f9640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500b6120 con 0x7f0d500b2760 2024-09-21T05:47:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d566f9640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500b5fa0 con 0x7f0d50152a30 2024-09-21T05:47:45.388 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d566f9640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0d500b62a0 con 0x7f0d501f6db0 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 0x7f0d500cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 0x7f0d500cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33714/0 (socket says 172.21.15.22:33714) 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 -- 172.21.15.22:0/2458186925 learned_addr learned my addr 172.21.15.22:0/2458186925 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d47fff640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d501f6db0 0x7f0d500b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d4ffff640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d50152a30 0x7f0d501f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 -- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d501f6db0 msgr2=0x7f0d500b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d501f6db0 0x7f0d500b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 -- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d50152a30 msgr2=0x7f0d501f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d50152a30 0x7f0d501f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 -- 172.21.15.22:0/2458186925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d500b11f0 con 0x7f0d500b2760 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d4ffff640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d50152a30 0x7f0d501f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d54c96640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 0x7f0d500cbce0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f0d48001330 tx=0x7f0d480039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.385+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4802b760 con 0x7f0d500b2760 2024-09-21T05:47:45.389 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.386+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d4803a070 con 0x7f0d500b2760 2024-09-21T05:47:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.386+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d500b2b70 con 0x7f0d500b2760 2024-09-21T05:47:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.386+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4802bcb0 con 0x7f0d500b2760 2024-09-21T05:47:45.390 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.386+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d500b1740 con 0x7f0d500b2760 2024-09-21T05:47:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.387+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d500b67b0 con 0x7f0d500b2760 2024-09-21T05:47:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.387+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d4803c020 con 0x7f0d500b2760 2024-09-21T05:47:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.388+0000 7f0d4d7fa640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d30078b90 0x7f0d3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.388+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f0d480ba4a0 con 0x7f0d500b2760 2024-09-21T05:47:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.388+0000 7f0d4ffff640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d30078b90 0x7f0d3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.391 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.389+0000 7f0d4ffff640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d30078b90 0x7f0d3007b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d501f88d0 tx=0x7f0d40005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:45.394 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.392+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d48082d60 con 0x7f0d500b2760 2024-09-21T05:47:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.580+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d500d3fb0 con 0x7f0d500b2760 2024-09-21T05:47:45.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.581+0000 7f0d4d7fa640 1 -- 172.21.15.22:0/2458186925 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+13152 (secure 0 0 0) 0x7f0d48082660 con 0x7f0d500b2760 2024-09-21T05:47:45.583 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:45.583 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":101,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:47:44.241479+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6832/3179586532":"2024-09-22T05:47:37.661668+0000","172.21.15.22:6834/1062944206":"2024-09-22T05:47:01.104934+0000","172.21.15.22:6834/3179586532":"2024-09-22T05:47:37.661668+0000","172.21.15.22:6832/1062944206":"2024-09-22T05:47:01.104934+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d30078b90 msgr2=0x7f0d3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d30078b90 0x7f0d3007b0a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0d501f88d0 tx=0x7f0d40005f30 comp rx=0 tx=0).stop 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 msgr2=0x7f0d500cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 0x7f0d500cbce0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f0d48001330 tx=0x7f0d480039f0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 shutdown_connections 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0d500b2760 0x7f0d500cbce0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0d30078b90 0x7f0d3007b0a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0d501f6db0 0x7f0d500b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 --2- 172.21.15.22:0/2458186925 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0d50152a30 0x7f0d501f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 >> 172.21.15.22:0/2458186925 conn(0x7f0d5009d7f0 msgr2=0x7f0d5006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:45.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 shutdown_connections 2024-09-21T05:47:45.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.583+0000 7f0d566f9640 1 -- 172.21.15.22:0/2458186925 wait complete. 2024-09-21T05:47:45.596 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T05:39:09.753482+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T05:47:45.596 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T05:47:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 -- 172.21.15.22:0/562345777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 msgr2=0x7f004415bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 --2- 172.21.15.22:0/562345777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f004415bf20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f0034005fd0 tx=0x7f003402b910 comp rx=0 tx=0).stop 2024-09-21T05:47:45.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 -- 172.21.15.22:0/562345777 shutdown_connections 2024-09-21T05:47:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 --2- 172.21.15.22:0/562345777 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00440cb0b0 0x7f004406ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 --2- 172.21.15.22:0/562345777 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f004415bf20 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 -- 172.21.15.22:0/562345777 >> 172.21.15.22:0/562345777 conn(0x7f00440b1cb0 msgr2=0x7f00440b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 -- 172.21.15.22:0/562345777 shutdown_connections 2024-09-21T05:47:45.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.714+0000 7f004a45c640 1 -- 172.21.15.22:0/562345777 wait complete. 2024-09-21T05:47:45.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.716+0000 7f004a45c640 1 Processor -- start 2024-09-21T05:47:45.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.716+0000 7f004a45c640 1 -- start start 2024-09-21T05:47:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f004a45c640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00440cf160 0x7f004415a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f004a45c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f00441e6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f004a45c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00441e7190 0x7f00440c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f004a45c640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f00440ccba0 con 0x7f00440cfb00 2024-09-21T05:47:45.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f004a45c640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f00440cf780 con 0x7f00441e7190 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f004a45c640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f00440ccd20 con 0x7f00440cf160 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f00441e6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f00441e6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33736/0 (socket says 172.21.15.22:33736) 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 -- 172.21.15.22:0/1033947610 learned_addr learned my addr 172.21.15.22:0/1033947610 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f0043fff640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00440cf160 0x7f004415a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00489f9640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00441e7190 0x7f00440c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 -- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00440cf160 msgr2=0x7f004415a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00440cf160 0x7f004415a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 -- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00441e7190 msgr2=0x7f00440c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00441e7190 0x7f00440c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00437fe640 1 -- 172.21.15.22:0/1033947610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441e7640 con 0x7f00440cfb00 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.717+0000 7f00489f9640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00441e7190 0x7f00440c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.718+0000 7f00437fe640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f00441e6c50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f003800c3d0 tx=0x7f003800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:45.721 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.718+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038017070 con 0x7f00440cfb00 2024-09-21T05:47:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.718+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00380053c0 con 0x7f00440cfb00 2024-09-21T05:47:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.718+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00441e7920 con 0x7f00440cfb00 2024-09-21T05:47:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.718+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038003850 con 0x7f00440cfb00 2024-09-21T05:47:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.718+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00441e7ee0 con 0x7f00440cfb00 2024-09-21T05:47:45.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.719+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004415bf20 con 0x7f00440cfb00 2024-09-21T05:47:45.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.720+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f003801a070 con 0x7f00440cfb00 2024-09-21T05:47:45.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.720+0000 7f0040ff9640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0030078bc0 0x7f003007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:45.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.720+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f0038099cf0 con 0x7f00440cfb00 2024-09-21T05:47:45.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.721+0000 7f0043fff640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0030078bc0 0x7f003007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:45.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.721+0000 7f0043fff640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0030078bc0 0x7f003007b0d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0034035000 tx=0x7f0034004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:45.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.725+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f003809e070 con 0x7f00440cfb00 2024-09-21T05:47:45.914 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.912+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f00440b5570 con 0x7f00440cfb00 2024-09-21T05:47:45.915 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.913+0000 7f0040ff9640 1 -- 172.21.15.22:0/1033947610 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f0038061d90 con 0x7f00440cfb00 2024-09-21T05:47:45.915 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-09-21T05:47:45.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0030078bc0 msgr2=0x7f003007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0030078bc0 0x7f003007b0d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0034035000 tx=0x7f0034004730 comp rx=0 tx=0).stop 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 msgr2=0x7f00441e6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f00441e6c50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f003800c3d0 tx=0x7f003800c890 comp rx=0 tx=0).stop 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 shutdown_connections 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f00441e7190 0x7f00440c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f00440cfb00 0x7f00441e6c50 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0030078bc0 0x7f003007b0d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 --2- 172.21.15.22:0/1033947610 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f00440cf160 0x7f004415a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 >> 172.21.15.22:0/1033947610 conn(0x7f00440b1cb0 msgr2=0x7f00440af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 shutdown_connections 2024-09-21T05:47:45.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:45.915+0000 7f004a45c640 1 -- 172.21.15.22:0/1033947610 wait complete. 2024-09-21T05:47:45.929 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.043+0000 7fd0adf58640 1 -- 172.21.15.22:0/2433617984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd090004800 msgr2=0x7fd090004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.043+0000 7fd0adf58640 1 --2- 172.21.15.22:0/2433617984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd090004800 0x7fd090004c10 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0016c0 tx=0x7fd090005830 comp rx=0 tx=0).stop 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.043+0000 7fd0adf58640 1 -- 172.21.15.22:0/2433617984 shutdown_connections 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.043+0000 7fd0adf58640 1 --2- 172.21.15.22:0/2433617984 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd090004800 0x7fd090004c10 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.043+0000 7fd0adf58640 1 -- 172.21.15.22:0/2433617984 >> 172.21.15.22:0/2433617984 conn(0x7fd0a80b3f10 msgr2=0x7fd0a80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.044+0000 7fd0adf58640 1 -- 172.21.15.22:0/2433617984 shutdown_connections 2024-09-21T05:47:46.046 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.044+0000 7fd0adf58640 1 -- 172.21.15.22:0/2433617984 wait complete. 2024-09-21T05:47:46.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.045+0000 7fd0adf58640 1 Processor -- start 2024-09-21T05:47:46.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.046+0000 7fd0adf58640 1 -- start start 2024-09-21T05:47:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0adf58640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd090004800 0x7fd0a8151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0adf58640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 0x7fd0a81f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0adf58640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a81f9600 0x7fd0a806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:46.050 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0adf58640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80ceb50 con 0x7fd0a8152760 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0adf58640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a80b56c0 con 0x7fd0a81f9600 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0adf58640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fd0a80cecd0 con 0x7fd090004800 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 0x7fd0a81f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 0x7fd0a81f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33742/0 (socket says 172.21.15.22:33742) 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/690357394 learned_addr learned my addr 172.21.15.22:0/690357394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a7fff640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a81f9600 0x7fd0a806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd090004800 0x7fd0a8151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd090004800 msgr2=0x7fd0a8151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd090004800 0x7fd0a8151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a81f9600 msgr2=0x7fd0a806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a81f9600 0x7fd0a806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a6ffd640 1 -- 172.21.15.22:0/690357394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a81540e0 con 0x7fd0a8152760 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.047+0000 7fd0a7fff640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a81f9600 0x7fd0a806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:46.051 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.048+0000 7fd0a6ffd640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 0x7fd0a81f90c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd09c02a910 tx=0x7fd09c0063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.048+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c003a00 con 0x7fd0a8152760 2024-09-21T05:47:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.048+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd09c039070 con 0x7fd0a8152760 2024-09-21T05:47:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.048+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a81543a0 con 0x7fd0a8152760 2024-09-21T05:47:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.048+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c033420 con 0x7fd0a8152760 2024-09-21T05:47:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.048+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a80b1060 con 0x7fd0a8152760 2024-09-21T05:47:46.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.049+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0a80ce730 con 0x7fd0a8152760 2024-09-21T05:47:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.050+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd09c044020 con 0x7fd0a8152760 2024-09-21T05:47:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.050+0000 7fd08ffff640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd094078b90 0x7fd09407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.050+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7fd09c0ba420 con 0x7fd0a8152760 2024-09-21T05:47:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.051+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd094078b90 0x7fd09407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:46.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.051+0000 7fd0a77fe640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd094078b90 0x7fd09407b0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8052910 tx=0x7fd098005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:46.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.055+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd0a80ce730 con 0x7fd0a8152760 2024-09-21T05:47:46.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:46.320+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7fd0a81f9f60 con 0x7fd0a8152760 2024-09-21T05:47:47.258 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.256+0000 7fd08ffff640 1 -- 172.21.15.22:0/690357394 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v390) ==== 147+0+0 (secure 0 0 0) 0x7fd09c082b60 con 0x7fd0a8152760 2024-09-21T05:47:47.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd094078b90 msgr2=0x7fd09407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd094078b90 0x7fd09407b0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8052910 tx=0x7fd098005f90 comp rx=0 tx=0).stop 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 msgr2=0x7fd0a81f90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 0x7fd0a81f90c0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd09c02a910 tx=0x7fd09c0063a0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 shutdown_connections 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fd0a81f9600 0x7fd0a806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fd0a8152760 0x7fd0a81f90c0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fd094078b90 0x7fd09407b0a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 --2- 172.21.15.22:0/690357394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fd090004800 0x7fd0a8151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.259+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 >> 172.21.15.22:0/690357394 conn(0x7fd0a80b3f10 msgr2=0x7fd0a80c8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.260+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 shutdown_connections 2024-09-21T05:47:47.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.260+0000 7fd0adf58640 1 -- 172.21.15.22:0/690357394 wait complete. 2024-09-21T05:47:47.276 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.397+0000 7f8942a4a640 1 -- 172.21.15.22:0/2957538365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 msgr2=0x7f893c0d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.397+0000 7f8942a4a640 1 --2- 172.21.15.22:0/2957538365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c0d14f0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f8930005fd0 tx=0x7f893002a540 comp rx=0 tx=0).stop 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.397+0000 7f8942a4a640 1 -- 172.21.15.22:0/2957538365 shutdown_connections 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.397+0000 7f8942a4a640 1 --2- 172.21.15.22:0/2957538365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c0d14f0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.397+0000 7f8942a4a640 1 --2- 172.21.15.22:0/2957538365 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f893c0b9b60 0x7f893c0d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.397+0000 7f8942a4a640 1 -- 172.21.15.22:0/2957538365 >> 172.21.15.22:0/2957538365 conn(0x7f893c09d8f0 msgr2=0x7f893c0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.398+0000 7f8942a4a640 1 -- 172.21.15.22:0/2957538365 shutdown_connections 2024-09-21T05:47:47.400 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.398+0000 7f8942a4a640 1 -- 172.21.15.22:0/2957538365 wait complete. 2024-09-21T05:47:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.399+0000 7f8942a4a640 1 Processor -- start 2024-09-21T05:47:47.401 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.399+0000 7f8942a4a640 1 -- start start 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8942a4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8942a4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f893c0b9b60 0x7f893c153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8942a4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c162430 0x7f893c1f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8942a4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1630d0 con 0x7f893c0b3a10 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8942a4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f893c162f50 con 0x7f893c162430 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8942a4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f893c163250 con 0x7f893c0b9b60 2024-09-21T05:47:47.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33762/0 (socket says 172.21.15.22:33762) 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 -- 172.21.15.22:0/1838067658 learned_addr learned my addr 172.21.15.22:0/1838067658 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893b7fe640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f893c0b9b60 0x7f893c153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f8940fe7640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c162430 0x7f893c1f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 -- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f893c0b9b60 msgr2=0x7f893c153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f893c0b9b60 0x7f893c153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 -- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c162430 msgr2=0x7f893c1f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c162430 0x7f893c1f9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 -- 172.21.15.22:0/1838067658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c14b5d0 con 0x7f893c0b3a10 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.401+0000 7f893bfff640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c155410 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f892c00d770 tx=0x7f892c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.402+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c017070 con 0x7f893c0b3a10 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.402+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f892c009d20 con 0x7f893c0b3a10 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.402+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c0b1380 con 0x7f893c0b3a10 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.402+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0106c0 con 0x7f893c0b3a10 2024-09-21T05:47:47.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.402+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1f9250 con 0x7f893c0b3a10 2024-09-21T05:47:47.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.403+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c155c50 con 0x7f893c0b3a10 2024-09-21T05:47:47.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.404+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f892c01a070 con 0x7f893c0b3a10 2024-09-21T05:47:47.406 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.404+0000 7f8938ff9640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8928078bc0 0x7f892807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.404+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f892c09ab20 con 0x7f893c0b3a10 2024-09-21T05:47:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.405+0000 7f893b7fe640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8928078bc0 0x7f892807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.407 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.405+0000 7f893b7fe640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8928078bc0 0x7f892807b0d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8930033000 tx=0x7f8930004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:47.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.409+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f893c155c50 con 0x7f893c0b3a10 2024-09-21T05:47:47.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.600+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f893c162840 con 0x7f893c0b3a10 2024-09-21T05:47:47.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.601+0000 7f8938ff9640 1 -- 172.21.15.22:0/1838067658 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+2294 (secure 0 0 0) 0x7f892c062bb0 con 0x7f893c0b3a10 2024-09-21T05:47:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T05:47:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T05:47:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced client_mount_timeout 30 2024-09-21T05:47:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_client 20/20 2024-09-21T05:47:47.603 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_mgr 20/20 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:mgr advanced debug_ms 1/1 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4295.166026 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 6060.964502 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4497.352229 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4450.819469 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 5971.638123 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4845.912071 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4687.996217 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6092.703308 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds 20/20 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T05:47:47.604 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced debug_ms 2/2 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_frag true 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_debug_scatterstat true 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds dev mds_verify_scatter true 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced client_mount_timeout 600 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_client 20/20 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced debug_ms 1/1 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T05:47:47.605 INFO:teuthology.orchestra.run.smithi022.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T05:47:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8928078bc0 msgr2=0x7f892807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8928078bc0 0x7f892807b0d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8930033000 tx=0x7f8930004730 comp rx=0 tx=0).stop 2024-09-21T05:47:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 msgr2=0x7f893c155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c155410 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f892c00d770 tx=0x7f892c00dc30 comp rx=0 tx=0).stop 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 shutdown_connections 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f893c162430 0x7f893c1f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8928078bc0 0x7f892807b0d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f893c0b9b60 0x7f893c153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 --2- 172.21.15.22:0/1838067658 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f893c0b3a10 0x7f893c155410 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.603+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 >> 172.21.15.22:0/1838067658 conn(0x7f893c09d8f0 msgr2=0x7f893c0b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.604+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 shutdown_connections 2024-09-21T05:47:47.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.604+0000 7f8942a4a640 1 -- 172.21.15.22:0/1838067658 wait complete. 2024-09-21T05:47:47.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.734+0000 7f54e4958640 1 -- 172.21.15.22:0/1463739641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 msgr2=0x7f54c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.734+0000 7f54e4958640 1 --2- 172.21.15.22:0/1463739641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54c800b9c0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f54e008c9c0 tx=0x7f54c8005830 comp rx=0 tx=0).stop 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.735+0000 7f54e4958640 1 -- 172.21.15.22:0/1463739641 shutdown_connections 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.735+0000 7f54e4958640 1 --2- 172.21.15.22:0/1463739641 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54c800b9c0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.735+0000 7f54e4958640 1 -- 172.21.15.22:0/1463739641 >> 172.21.15.22:0/1463739641 conn(0x7f54e009d830 msgr2=0x7f54e00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.735+0000 7f54e4958640 1 -- 172.21.15.22:0/1463739641 shutdown_connections 2024-09-21T05:47:47.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.735+0000 7f54e4958640 1 -- 172.21.15.22:0/1463739641 wait complete. 2024-09-21T05:47:47.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.737+0000 7f54e4958640 1 Processor -- start 2024-09-21T05:47:47.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.737+0000 7f54e4958640 1 -- start start 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54e4958640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54e01f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54e4958640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e009e720 0x7f54e0157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54e4958640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e015ea00 0x7f54e009fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54e4958640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00b6530 con 0x7f54c8005350 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54e4958640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f54e00b63b0 con 0x7f54e009e720 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54e4958640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f54e00b66b0 con 0x7f54e015ea00 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54e01f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54e01f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33794/0 (socket says 172.21.15.22:33794) 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 -- 172.21.15.22:0/1007111340 learned_addr learned my addr 172.21.15.22:0/1007111340 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54ddd9b640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e009e720 0x7f54e0157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54ded9d640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e015ea00 0x7f54e009fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 -- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e015ea00 msgr2=0x7f54e009fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e015ea00 0x7f54e009fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 -- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e009e720 msgr2=0x7f54e0157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e009e720 0x7f54e0157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.738+0000 7f54de59c640 1 -- 172.21.15.22:0/1007111340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01f58f0 con 0x7f54c8005350 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.739+0000 7f54de59c640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54e01f53b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e590 tx=0x7f54d000ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.739+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0015070 con 0x7f54c8005350 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.739+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54d000c9e0 con 0x7f54c8005350 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.739+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e009ee00 con 0x7f54c8005350 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.739+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d000ce30 con 0x7f54c8005350 2024-09-21T05:47:47.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.739+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e009f3c0 con 0x7f54c8005350 2024-09-21T05:47:47.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.740+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e00d3350 con 0x7f54c8005350 2024-09-21T05:47:47.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.741+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f54d0018070 con 0x7f54c8005350 2024-09-21T05:47:47.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.741+0000 7f54ceffd640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54c0078bc0 0x7f54c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.741+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7f54d009ae40 con 0x7f54c8005350 2024-09-21T05:47:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.741+0000 7f54ddd9b640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54c0078bc0 0x7f54c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:47.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.742+0000 7f54ddd9b640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54c0078bc0 0x7f54c007b0d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54e015e9b0 tx=0x7f54d400b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:47.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.746+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f54d0062ed0 con 0x7f54c8005350 2024-09-21T05:47:47.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.933+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f54e0160d60 con 0x7f54c8005350 2024-09-21T05:47:47.938 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.936+0000 7f54ceffd640 1 -- 172.21.15.22:0/1007111340 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f54d0067900 con 0x7f54c8005350 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54c0078bc0 msgr2=0x7f54c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54c0078bc0 0x7f54c007b0d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f54e015e9b0 tx=0x7f54d400b260 comp rx=0 tx=0).stop 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 msgr2=0x7f54e01f53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54e01f53b0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e590 tx=0x7f54d000ea50 comp rx=0 tx=0).stop 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 shutdown_connections 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f54c0078bc0 0x7f54c007b0d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f54e015ea00 0x7f54e009fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.941 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f54e009e720 0x7f54e0157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 --2- 172.21.15.22:0/1007111340 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f54c8005350 0x7f54e01f53b0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 >> 172.21.15.22:0/1007111340 conn(0x7f54e009d830 msgr2=0x7f54e00d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 shutdown_connections 2024-09-21T05:47:47.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:47.939+0000 7f54e4958640 1 -- 172.21.15.22:0/1007111340 wait complete. 2024-09-21T05:47:47.952 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:47:47.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:47:47.995 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:47:48.043 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:47:48.043 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:47:48.043 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:48.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:47:48.060 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:47:48.060 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:48.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:47:48.078 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:47:48.078 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:48.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:47:48.116 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:47:48.116 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:47:48.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:47:48.136 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T05:47:48.136 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:48.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/3150187683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f40cdbf0 msgr2=0x7fa0f40cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.272 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/3150187683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f40cdbf0 0x7fa0f40cf350 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40d2320 tx=0x7fa0ec02aa30 comp rx=0 tx=0).stop 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/3150187683 shutdown_connections 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/3150187683 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f40cdbf0 0x7fa0f40cf350 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/3150187683 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f409aa30 0x7fa0f40cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/3150187683 >> v2:172.21.15.134:3300/0 conn(0x7fa0f415b900 0x7fa0f40ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/3150187683 >> 172.21.15.22:0/3150187683 conn(0x7fa0f4074170 msgr2=0x7fa0f40b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/3150187683 shutdown_connections 2024-09-21T05:47:48.273 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.270+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/3150187683 wait complete. 2024-09-21T05:47:48.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.272+0000 7fa0fa0f4640 1 Processor -- start 2024-09-21T05:47:48.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.272+0000 7fa0fa0f4640 1 -- start start 2024-09-21T05:47:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.273+0000 7fa0fa0f4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f409aa30 0x7fa0f4144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.273+0000 7fa0fa0f4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40cdbf0 0x7fa0f41444e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.273+0000 7fa0fa0f4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 0x7fa0f4147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.273+0000 7fa0fa0f4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f40cce60 con 0x7fa0f415b900 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.273+0000 7fa0fa0f4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f40ccce0 con 0x7fa0f409aa30 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.273+0000 7fa0fa0f4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f40ccfe0 con 0x7fa0f40cdbf0 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 0x7fa0f4147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 0x7fa0f4147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33814/0 (socket says 172.21.15.22:33814) 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 -- 172.21.15.22:0/1930164624 learned_addr learned my addr 172.21.15.22:0/1930164624 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f37fe640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f409aa30 0x7fa0f4144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f2ffd640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40cdbf0 0x7fa0f41444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 -- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40cdbf0 msgr2=0x7fa0f41444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40cdbf0 0x7fa0f41444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 -- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f409aa30 msgr2=0x7fa0f4144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f409aa30 0x7fa0f4144c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 -- 172.21.15.22:0/1930164624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f4147c70 con 0x7fa0f415b900 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f2ffd640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40cdbf0 0x7fa0f41444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0f3fff640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 0x7fa0f4147730 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec02b020 tx=0x7fa0ec003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec033050 con 0x7fa0f415b900 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0ec004550 con 0x7fa0f415b900 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f4154160 con 0x7fa0f415b900 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec038450 con 0x7fa0f415b900 2024-09-21T05:47:48.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.274+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f4154720 con 0x7fa0f415b900 2024-09-21T05:47:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.275+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f40cd510 con 0x7fa0f415b900 2024-09-21T05:47:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.276+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa0ec036070 con 0x7fa0f415b900 2024-09-21T05:47:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.277+0000 7fa0dbfff640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0e0078bc0 0x7fa0e007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.277+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7fa0ec0bc040 con 0x7fa0f415b900 2024-09-21T05:47:48.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.277+0000 7fa0f37fe640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0e0078bc0 0x7fa0e007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.278+0000 7fa0f37fe640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0e0078bc0 0x7fa0e007b0d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005fd0 tx=0x7fa0e400a250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:48.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.281+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa0f4147c70 con 0x7fa0f415b900 2024-09-21T05:47:48.509 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.508+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0f4153ec0 con 0x7fa0f415b900 2024-09-21T05:47:48.510 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.508+0000 7fa0dbfff640 1 -- 172.21.15.22:0/1930164624 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3914 (secure 0 0 0) 0x7fa0ec084720 con 0x7fa0f415b900 2024-09-21T05:47:48.510 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:48.510 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":94,"btime":"2024-09-21T05:47:42:198164+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6380,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/1255442048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":1255442048},{"type":"v1","addr":"172.21.15.134:6836","nonce":1255442048}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6940,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/4119188147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":4119188147},{"type":"v1","addr":"172.21.15.22:6835","nonce":4119188147}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/2357557694","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":2357557694},{"type":"v1","addr":"172.21.15.134:6837","nonce":2357557694}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7908,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/199773408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":199773408},{"type":"v1","addr":"172.21.15.22:6834","nonce":199773408}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[]} 2024-09-21T05:47:48.511 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 94 2024-09-21T05:47:48.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0e0078bc0 msgr2=0x7fa0e007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0e0078bc0 0x7fa0e007b0d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005fd0 tx=0x7fa0e400a250 comp rx=0 tx=0).stop 2024-09-21T05:47:48.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 msgr2=0x7fa0f4147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 0x7fa0f4147730 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec02b020 tx=0x7fa0ec003f10 comp rx=0 tx=0).stop 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 shutdown_connections 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa0f415b900 0x7fa0f4147730 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa0e0078bc0 0x7fa0e007b0d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa0f40cdbf0 0x7fa0f41444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 --2- 172.21.15.22:0/1930164624 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa0f409aa30 0x7fa0f4144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 >> 172.21.15.22:0/1930164624 conn(0x7fa0f4074170 msgr2=0x7fa0f40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 shutdown_connections 2024-09-21T05:47:48.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.510+0000 7fa0fa0f4640 1 -- 172.21.15.22:0/1930164624 wait complete. 2024-09-21T05:47:48.524 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T05:47:48.524 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T05:47:48.524 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:47:48.524 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:47:48.524 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T05:47:48.524 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T05:47:48.525 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:47:48.525 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:47:48.525 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T05:47:48.525 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T05:47:48.525 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:47:48.525 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:47:48.525 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T05:47:48.525 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T05:47:48.526 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:47:48.526 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:47:48.533 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:47:48.533 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T05:47:48.536 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:47:48.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T05:47:48.537 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:47:48.538 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T05:47:48.538 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:47:48.538 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T05:47:48.578 INFO:tasks.ceph.mds.a:Started 2024-09-21T05:47:48.579 DEBUG:teuthology.parallel:result is None 2024-09-21T05:47:48.580 INFO:tasks.ceph.mds.c:Started 2024-09-21T05:47:48.580 DEBUG:teuthology.parallel:result is None 2024-09-21T05:47:48.581 INFO:tasks.ceph.mds.b:Started 2024-09-21T05:47:48.581 DEBUG:teuthology.parallel:result is None 2024-09-21T05:47:48.581 INFO:tasks.ceph.mds.d:Started 2024-09-21T05:47:48.582 DEBUG:teuthology.parallel:result is None 2024-09-21T05:47:48.582 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T05:47:48.669 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:47:48.670 INFO:tasks.ceph.mds.c.smithi022.stdout:starting mds.c at 2024-09-21T05:47:48.676 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:47:48.676 INFO:tasks.ceph.mds.d.smithi134.stdout:starting mds.d at 2024-09-21T05:47:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe67564d640 1 -- 172.21.15.22:0/1284315242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 msgr2=0x7fe658004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe67564d640 1 --2- 172.21.15.22:0/1284315242 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe658004c10 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fe664009b20 tx=0x7fe658005830 comp rx=0 tx=0).stop 2024-09-21T05:47:48.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe66f7fe640 1 -- 172.21.15.22:0/1284315242 reap_dead start 2024-09-21T05:47:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe67564d640 1 -- 172.21.15.22:0/1284315242 shutdown_connections 2024-09-21T05:47:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe67564d640 1 -- 172.21.15.22:0/1284315242 >> 172.21.15.22:0/1284315242 conn(0x7fe67008bfc0 msgr2=0x7fe67009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe67564d640 1 -- 172.21.15.22:0/1284315242 shutdown_connections 2024-09-21T05:47:48.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.704+0000 7fe67564d640 1 -- 172.21.15.22:0/1284315242 wait complete. 2024-09-21T05:47:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.706+0000 7fe67564d640 1 Processor -- start 2024-09-21T05:47:48.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.706+0000 7fe67564d640 1 -- start start 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.707+0000 7fe67564d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe670148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.707+0000 7fe67564d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6701614a0 0x7fe67014d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe67564d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6701618b0 0x7fe67014dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe67564d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700d28c0 con 0x7fe658004800 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe67564d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe6700d2740 con 0x7fe6701618b0 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe67564d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe6700d2a40 con 0x7fe6701614a0 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe670148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe670148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33872/0 (socket says 172.21.15.22:33872) 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 -- 172.21.15.22:0/1498449972 learned_addr learned my addr 172.21.15.22:0/1498449972 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66e7fc640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6701614a0 0x7fe67014d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66f7fe640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6701618b0 0x7fe67014dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 -- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6701614a0 msgr2=0x7fe67014d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6701614a0 0x7fe67014d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 -- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6701618b0 msgr2=0x7fe67014dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6701618b0 0x7fe67014dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 -- 172.21.15.22:0/1498449972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe670148860 con 0x7fe658004800 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66e7fc640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6701614a0 0x7fe67014d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe66effd640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe670148320 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe660011b10 tx=0x7fe660009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66001e070 con 0x7fe658004800 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe660004050 con 0x7fe658004800 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6701f5cd0 con 0x7fe658004800 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.708+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66000fea0 con 0x7fe658004800 2024-09-21T05:47:48.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.709+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6700c8440 con 0x7fe658004800 2024-09-21T05:47:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.709+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe67006a5e0 con 0x7fe658004800 2024-09-21T05:47:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.710+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe660021070 con 0x7fe658004800 2024-09-21T05:47:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.711+0000 7fe6577fe640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe65c078b90 0x7fe65c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.711+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5029+0+0 (secure 0 0 0) 0x7fe66009b360 con 0x7fe658004800 2024-09-21T05:47:48.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.711+0000 7fe66e7fc640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe65c078b90 0x7fe65c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:48.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.712+0000 7fe66e7fc640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe65c078b90 0x7fe65c07b0a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe664009b20 tx=0x7fe66402d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:48.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.716+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe67006a5e0 con 0x7fe658004800 2024-09-21T05:47:48.918 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:48.916+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe67006f7a0 con 0x7fe658004800 2024-09-21T05:47:49.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.277+0000 7fe6577fe640 1 -- 172.21.15.22:0/1498449972 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) ==== 97+0+0 (secure 0 0 0) 0x7fe6600044a0 con 0x7fe658004800 2024-09-21T05:47:49.279 INFO:teuthology.orchestra.run.smithi022.stderr: removed all blocklist entries 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.279+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe65c078b90 msgr2=0x7fe65c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.279+0000 7fe67564d640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe65c078b90 0x7fe65c07b0a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe664009b20 tx=0x7fe66402d040 comp rx=0 tx=0).stop 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 msgr2=0x7fe670148320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe670148320 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe660011b10 tx=0x7fe660009e90 comp rx=0 tx=0).stop 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 shutdown_connections 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe6701618b0 0x7fe67014dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe65c078b90 0x7fe65c07b0a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe6701614a0 0x7fe67014d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.282 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 --2- 172.21.15.22:0/1498449972 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe658004800 0x7fe670148320 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 >> 172.21.15.22:0/1498449972 conn(0x7fe67008bfc0 msgr2=0x7fe6700b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 shutdown_connections 2024-09-21T05:47:49.283 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.280+0000 7fe67564d640 1 -- 172.21.15.22:0/1498449972 wait complete. 2024-09-21T05:47:49.297 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 -- 172.21.15.22:0/2328197704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0b9b70 msgr2=0x7f563c0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 --2- 172.21.15.22:0/2328197704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0b9b70 0x7f563c0cf090 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f563c08d4e0 tx=0x7f562402a560 comp rx=0 tx=0).stop 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 -- 172.21.15.22:0/2328197704 shutdown_connections 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 --2- 172.21.15.22:0/2328197704 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0b9b70 0x7f563c0cf090 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 --2- 172.21.15.22:0/2328197704 >> v2:172.21.15.134:3300/0 conn(0x7f563c0d2ed0 0x7f563c0d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 -- 172.21.15.22:0/2328197704 >> 172.21.15.22:0/2328197704 conn(0x7f563c08bfc0 msgr2=0x7f563c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 -- 172.21.15.22:0/2328197704 shutdown_connections 2024-09-21T05:47:49.416 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.413+0000 7f5641a4a640 1 -- 172.21.15.22:0/2328197704 wait complete. 2024-09-21T05:47:49.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.415+0000 7f5641a4a640 1 Processor -- start 2024-09-21T05:47:49.417 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.415+0000 7f5641a4a640 1 -- start start 2024-09-21T05:47:49.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f5641a4a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f563c0b9b70 0x7f563c1f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f5641a4a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 0x7f563c1f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f5641a4a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f563c0d2ed0 0x7f563c0b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f5641a4a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f563c0cfe80 con 0x7f563c0cf5d0 2024-09-21T05:47:49.420 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f5641a4a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f563c0cfd00 con 0x7f563c0d2ed0 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f5641a4a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f563c0d4940 con 0x7f563c0b9b70 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 0x7f563c1f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 0x7f563c1f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33884/0 (socket says 172.21.15.22:33884) 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 -- 172.21.15.22:0/1473564820 learned_addr learned my addr 172.21.15.22:0/1473564820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f563affd640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f563c0b9b70 0x7f563c1f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f563b7fe640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f563c0d2ed0 0x7f563c0b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 -- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f563c0b9b70 msgr2=0x7f563c1f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f563c0b9b70 0x7f563c1f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 -- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f563c0d2ed0 msgr2=0x7f563c0b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f563c0d2ed0 0x7f563c0b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 -- 172.21.15.22:0/1473564820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c1f5bd0 con 0x7f563c0cf5d0 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f563affd640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f563c0b9b70 0x7f563c1f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f563b7fe640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f563c0d2ed0 0x7f563c0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.417+0000 7f56327fc640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 0x7f563c1f91c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f56240099c0 tx=0x7f56240039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:49.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.418+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562403e030 con 0x7f563c0cf5d0 2024-09-21T05:47:49.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.418+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c1f9c80 con 0x7f563c0cf5d0 2024-09-21T05:47:49.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.418+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56240047a0 con 0x7f563c0cf5d0 2024-09-21T05:47:49.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.418+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624035040 con 0x7f563c0cf5d0 2024-09-21T05:47:49.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.418+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c1f5e50 con 0x7f563c0cf5d0 2024-09-21T05:47:49.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.419+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c0d4e40 con 0x7f563c0cf5d0 2024-09-21T05:47:49.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.420+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5624041030 con 0x7f563c0cf5d0 2024-09-21T05:47:49.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.420+0000 7f5638ff9640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f561c078b90 0x7f561c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.420+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f56240bb2d0 con 0x7f563c0cf5d0 2024-09-21T05:47:49.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.420+0000 7f563affd640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f561c078b90 0x7f561c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.421+0000 7f563affd640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f561c078b90 0x7f561c07b0a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f562c005fd0 tx=0x7f562c005ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:49.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.425+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f563c1f5bd0 con 0x7f563c0cf5d0 2024-09-21T05:47:49.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.689+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f563c1f9a10 con 0x7f563c0cf5d0 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.690+0000 7f5638ff9640 1 -- 172.21.15.22:0/1473564820 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f56240839e0 con 0x7f563c0cf5d0 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "auth_dump": [ 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.a", 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mSuE3FhAAWxKTXe2o0zwxb4uFdjxFyA==", 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.692 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.b", 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mytFqIxAAykbbq91jWQBtubxD9wFQCw==", 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:47:49.693 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.c", 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5md2n7HRAAqtS/3TX700CG9hiqSSQF8g==", 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mds.d", 2024-09-21T05:47:49.694 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mwdgMKxAAXzcpO7uofJ4AgxoVBBM1Aw==", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.0", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDRW+5mtDsEEhAA3KDpxfwCPQch2KgnuxLM5g==", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.695 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.1", 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDUW+5mLgKjFhAAZjjSIPjHVx7OYsLwTI2CHA==", 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.2", 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDXW+5mpF03GRAAld51yWWjg2hMbv1hlSJIXQ==", 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.696 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.3", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDaW+5msfDYORAAzBPxEfVsd28/LX1blknCWA==", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.4", 2024-09-21T05:47:49.697 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDhW+5meWmvFRAALebDHwpSvYoNnqOku3VEPg==", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.5", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDkW+5mPcBnGBAAXn6kijnbDOHWbmbVL+pnOA==", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.698 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.6", 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDnW+5mZVVxKRAAqT7wff7JTQRHgCWgBL3ExQ==", 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "osd.7", 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDrW+5mcakzDhAAmpmc25/VLlL+ciGmqD/tVQ==", 2024-09-21T05:47:49.699 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow profile osd", 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile osd", 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.0", 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA==", 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow r", 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.700 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.1", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5m6SqANxAAwZe3kiDNTiiR5VI4vuYBJQ==", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.2", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mFLK2MxAAyhhypBhVQeLdQ4UauBctEQ==", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:47:49.701 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.3", 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mCyhIOxAAWX3RHFgMyN5Ck+lSfd6eFw==", 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow", 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow r", 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow rw", 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow rwx" 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.702 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "client.admin", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDOW+5mOrZPNBAAWT3owJTbLFwX3sCHroVqrg==", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mgr": "allow *", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow *", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.x", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mAz1UEhAAsLII4TVTo+7vz+b2sCfhnQ==", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:47:49.703 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: }, 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "entity": "mgr.y", 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "key": "AQDPW+5mLHwHDhAAr81SystYm1K43GcmjXQ4Tw==", 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "caps": { 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mds": "allow *", 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "mon": "allow profile mgr", 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: "osd": "allow *" 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-09-21T05:47:49.704 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f561c078b90 msgr2=0x7f561c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f561c078b90 0x7f561c07b0a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f562c005fd0 tx=0x7f562c005ee0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 msgr2=0x7f563c1f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 0x7f563c1f91c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f56240099c0 tx=0x7f56240039e0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 shutdown_connections 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f563c0d2ed0 0x7f563c0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f563c0cf5d0 0x7f563c1f91c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f561c078b90 0x7f561c07b0a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 --2- 172.21.15.22:0/1473564820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f563c0b9b70 0x7f563c1f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 >> 172.21.15.22:0/1473564820 conn(0x7f563c08bfc0 msgr2=0x7f563c06eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 shutdown_connections 2024-09-21T05:47:49.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.692+0000 7f5641a4a640 1 -- 172.21.15.22:0/1473564820 wait complete. 2024-09-21T05:47:49.707 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T05:47:49.824 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04ee12b640 1 -- 172.21.15.22:0/1236595712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d0004e60 msgr2=0x7f04d00052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1236595712 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04d0004e60 0x7f04d00052d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f04e0005fd0 tx=0x7f04d0005830 comp rx=0 tx=0).stop 2024-09-21T05:47:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04e7fff640 1 -- 172.21.15.22:0/1236595712 reap_dead start 2024-09-21T05:47:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04ee12b640 1 -- 172.21.15.22:0/1236595712 shutdown_connections 2024-09-21T05:47:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04ee12b640 1 -- 172.21.15.22:0/1236595712 >> 172.21.15.22:0/1236595712 conn(0x7f04e80b3f10 msgr2=0x7f04e80b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04ee12b640 1 -- 172.21.15.22:0/1236595712 shutdown_connections 2024-09-21T05:47:49.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.822+0000 7f04ee12b640 1 -- 172.21.15.22:0/1236595712 wait complete. 2024-09-21T05:47:49.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.824+0000 7f04ee12b640 1 Processor -- start 2024-09-21T05:47:49.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.824+0000 7f04ee12b640 1 -- start start 2024-09-21T05:47:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04ee12b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04e814c330 0x7f04e80ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04ee12b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e814c740 0x7f04e80caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04ee12b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 0x7f04e80caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04ee12b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f04e80d0fe0 con 0x7f04e80ad7d0 2024-09-21T05:47:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04ee12b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f04e80d0e60 con 0x7f04e814c740 2024-09-21T05:47:49.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04ee12b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f04e80d1160 con 0x7f04e814c330 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 0x7f04e80caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04e7fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 0x7f04e80caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33900/0 (socket says 172.21.15.22:33900) 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04e7fff640 1 -- 172.21.15.22:0/1888810391 learned_addr learned my addr 172.21.15.22:0/1888810391 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04e6ffd640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e814c740 0x7f04e80caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.825+0000 7f04e77fe640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04e814c330 0x7f04e80ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04e7fff640 1 -- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04e814c330 msgr2=0x7f04e80ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04e7fff640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04e814c330 0x7f04e80ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04e7fff640 1 -- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e814c740 msgr2=0x7f04e80caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04e7fff640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e814c740 0x7f04e80caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04e7fff640 1 -- 172.21.15.22:0/1888810391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e81f6f70 con 0x7f04e80ad7d0 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04e7fff640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 0x7f04e80caf80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f04e0036040 tx=0x7f04e0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:49.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0038040 con 0x7f04e80ad7d0 2024-09-21T05:47:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e0004b20 con 0x7f04e80ad7d0 2024-09-21T05:47:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0003970 con 0x7f04e80ad7d0 2024-09-21T05:47:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e81f7250 con 0x7f04e80ad7d0 2024-09-21T05:47:49.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.826+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81f7810 con 0x7f04e80ad7d0 2024-09-21T05:47:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.827+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e80d16f0 con 0x7f04e80ad7d0 2024-09-21T05:47:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.828+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04e0003b10 con 0x7f04e80ad7d0 2024-09-21T05:47:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.828+0000 7f04cffff640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04d4078b90 0x7f04d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.828+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f04e00ba740 con 0x7f04e80ad7d0 2024-09-21T05:47:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.829+0000 7f04e77fe640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04d4078b90 0x7f04d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:49.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.829+0000 7f04e77fe640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04d4078b90 0x7f04d407b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f04d8001010 tx=0x7f04d8001290 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:49.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:49.833+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04e0082ec0 con 0x7f04e80ad7d0 2024-09-21T05:47:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.021+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f04e80b3520 con 0x7f04e80ad7d0 2024-09-21T05:47:50.023 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.021+0000 7f04cffff640 1 -- 172.21.15.22:0/1888810391 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v105) ==== 84+0+65 (secure 0 0 0) 0x7f04e00827c0 con 0x7f04e80ad7d0 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout:[ 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: { 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "poolnum": 1, 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: "poolname": ".mgr" 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout: } 2024-09-21T05:47:50.024 INFO:teuthology.orchestra.run.smithi022.stdout:] 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.023+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04d4078b90 msgr2=0x7f04d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.023+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04d4078b90 0x7f04d407b0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f04d8001010 tx=0x7f04d8001290 comp rx=0 tx=0).stop 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.023+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 msgr2=0x7f04e80caf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.023+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 0x7f04e80caf80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f04e0036040 tx=0x7f04e0033670 comp rx=0 tx=0).stop 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 shutdown_connections 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f04e80ad7d0 0x7f04e80caf80 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:50.026 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f04e814c740 0x7f04e80caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f04d4078b90 0x7f04d407b0a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 --2- 172.21.15.22:0/1888810391 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f04e814c330 0x7f04e80ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 >> 172.21.15.22:0/1888810391 conn(0x7f04e80b3f10 msgr2=0x7f04e80ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 shutdown_connections 2024-09-21T05:47:50.027 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.024+0000 7f04ee12b640 1 -- 172.21.15.22:0/1888810391 wait complete. 2024-09-21T05:47:50.037 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T05:47:50.037 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bda870640 1 -- 172.21.15.22:0/3403002195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bbc004800 msgr2=0x7f5bbc004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bda870640 1 --2- 172.21.15.22:0/3403002195 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bbc004800 0x7f5bbc004c10 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40cfe70 tx=0x7f5bbc005830 comp rx=0 tx=0).stop 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bd8e0d640 1 -- 172.21.15.22:0/3403002195 reap_dead start 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bda870640 1 -- 172.21.15.22:0/3403002195 shutdown_connections 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bda870640 1 -- 172.21.15.22:0/3403002195 >> 172.21.15.22:0/3403002195 conn(0x7f5bd40b3ea0 msgr2=0x7f5bd40b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bda870640 1 -- 172.21.15.22:0/3403002195 shutdown_connections 2024-09-21T05:47:50.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.156+0000 7f5bda870640 1 -- 172.21.15.22:0/3403002195 wait complete. 2024-09-21T05:47:50.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.158+0000 7f5bda870640 1 Processor -- start 2024-09-21T05:47:50.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.158+0000 7f5bda870640 1 -- start start 2024-09-21T05:47:50.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bda870640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc004800 0x7f5bd40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bda870640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bd414cec0 0x7f5bd414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bda870640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 0x7f5bd41f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bda870640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd415c760 con 0x7f5bd41f2f60 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bda870640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd415c5e0 con 0x7f5bbc004800 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bda870640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd415c8e0 con 0x7f5bd414cec0 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 0x7f5bd41f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 0x7f5bd41f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33920/0 (socket says 172.21.15.22:33920) 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 -- 172.21.15.22:0/2731950322 learned_addr learned my addr 172.21.15.22:0/2731950322 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:50.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd3fff640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc004800 0x7f5bd40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:50.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd37fe640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bd414cec0 0x7f5bd414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:50.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 -- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bd414cec0 msgr2=0x7f5bd414d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:50.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bd414cec0 0x7f5bd414d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:50.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 -- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc004800 msgr2=0x7f5bd40caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:50.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc004800 0x7f5bd40caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:50.165 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 -- 172.21.15.22:0/2731950322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd414d810 con 0x7f5bd41f2f60 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd3fff640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc004800 0x7f5bd40caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd37fe640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bd414cec0 0x7f5bd414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd8e0d640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 0x7f5bd41f3370 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40ce5c0 tx=0x7f5bcc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.160+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00cb20 con 0x7f5bd41f2f60 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.161+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bcc018070 con 0x7f5bd41f2f60 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.161+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd40c93a0 con 0x7f5bd41f2f60 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.161+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc011460 con 0x7f5bd41f2f60 2024-09-21T05:47:50.166 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.161+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd40cb3f0 con 0x7f5bd41f2f60 2024-09-21T05:47:50.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.162+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd415d570 con 0x7f5bd41f2f60 2024-09-21T05:47:50.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.162+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5bcc011600 con 0x7f5bd41f2f60 2024-09-21T05:47:50.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.163+0000 7f5bd0ff9640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5bc0078b80 0x7f5bc007b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:50.167 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.163+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 4857+0+0 (secure 0 0 0) 0x7f5bcc09afe0 con 0x7f5bd41f2f60 2024-09-21T05:47:50.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.163+0000 7f5bd3fff640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5bc0078b80 0x7f5bc007b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:50.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.164+0000 7f5bd3fff640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5bc0078b80 0x7f5bc007b090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009a70 tx=0x7f5bc4005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:50.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.168+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bd414d810 con 0x7f5bd41f2f60 2024-09-21T05:47:50.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:50.358+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5bd41f3aa0 con 0x7f5bd41f2f60 2024-09-21T05:47:51.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.290+0000 7f5bd0ff9640 1 -- 172.21.15.22:0/2731950322 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v106) ==== 138+0+0 (secure 0 0 0) 0x7f5bcc063780 con 0x7f5bd41f2f60 2024-09-21T05:47:51.292 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' created 2024-09-21T05:47:51.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5bc0078b80 msgr2=0x7f5bc007b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:51.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5bc0078b80 0x7f5bc007b090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009a70 tx=0x7f5bc4005f90 comp rx=0 tx=0).stop 2024-09-21T05:47:51.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 msgr2=0x7f5bd41f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:51.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 0x7f5bd41f3370 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40ce5c0 tx=0x7f5bcc00ed10 comp rx=0 tx=0).stop 2024-09-21T05:47:51.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 shutdown_connections 2024-09-21T05:47:51.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5bd41f2f60 0x7f5bd41f3370 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5bc0078b80 0x7f5bc007b090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5bd414cec0 0x7f5bd414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 --2- 172.21.15.22:0/2731950322 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5bbc004800 0x7f5bd40caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.294+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 >> 172.21.15.22:0/2731950322 conn(0x7f5bd40b3ea0 msgr2=0x7f5bd415de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:51.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.295+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 shutdown_connections 2024-09-21T05:47:51.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.296+0000 7f5bda870640 1 -- 172.21.15.22:0/2731950322 wait complete. 2024-09-21T05:47:51.316 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 -- 172.21.15.22:0/3311605549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00d2310 msgr2=0x7f02f00d2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 --2- 172.21.15.22:0/3311605549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00d2310 0x7f02f00d2720 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f02f00d03e0 tx=0x7f02e802aa00 comp rx=0 tx=0).stop 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 -- 172.21.15.22:0/3311605549 shutdown_connections 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 --2- 172.21.15.22:0/3311605549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00d2310 0x7f02f00d2720 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 --2- 172.21.15.22:0/3311605549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00ccbe0 0x7f02f015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 -- 172.21.15.22:0/3311605549 >> 172.21.15.22:0/3311605549 conn(0x7f02f00b3f10 msgr2=0x7f02f00b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 -- 172.21.15.22:0/3311605549 shutdown_connections 2024-09-21T05:47:51.447 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.444+0000 7f02f61c6640 1 -- 172.21.15.22:0/3311605549 wait complete. 2024-09-21T05:47:51.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.446+0000 7f02f61c6640 1 Processor -- start 2024-09-21T05:47:51.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.446+0000 7f02f61c6640 1 -- start start 2024-09-21T05:47:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.447+0000 7f02f61c6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 0x7f02f00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.447+0000 7f02f61c6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f00ccbe0 0x7f02f0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.447+0000 7f02f61c6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00d2310 0x7f02f00b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.447+0000 7f02f61c6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00d18b0 con 0x7f02f00b5210 2024-09-21T05:47:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.447+0000 7f02f61c6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f02f00d1730 con 0x7f02f00ccbe0 2024-09-21T05:47:51.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02f61c6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00d1a30 con 0x7f02f00d2310 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 0x7f02f00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 0x7f02f00cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33936/0 (socket says 172.21.15.22:33936) 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 -- 172.21.15.22:0/1220176434 learned_addr learned my addr 172.21.15.22:0/1220176434 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02effff640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00d2310 0x7f02f00b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02eeffd640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f00ccbe0 0x7f02f0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 -- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00d2310 msgr2=0x7f02f00b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00d2310 0x7f02f00b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 -- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f00ccbe0 msgr2=0x7f02f0157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f00ccbe0 0x7f02f0157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 -- 172.21.15.22:0/1220176434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f0157950 con 0x7f02f00b5210 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02effff640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00d2310 0x7f02f00b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02ef7fe640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 0x7f02f00cc110 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f02e000d6d0 tx=0x7f02e000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0017070 con 0x7f02f00b5210 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02e0005330 con 0x7f02f00b5210 2024-09-21T05:47:51.452 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f0157e60 con 0x7f02f00b5210 2024-09-21T05:47:51.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0009820 con 0x7f02f00b5210 2024-09-21T05:47:51.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.448+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f0158420 con 0x7f02f00b5210 2024-09-21T05:47:51.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.450+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f00d1f50 con 0x7f02f00b5210 2024-09-21T05:47:51.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.450+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f02e0009bd0 con 0x7f02f00b5210 2024-09-21T05:47:51.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.451+0000 7f02e7fff640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f02d0078bc0 0x7f02d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:51.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.451+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5227+0+0 (secure 0 0 0) 0x7f02e009ae90 con 0x7f02f00b5210 2024-09-21T05:47:51.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.451+0000 7f02eeffd640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f02d0078bc0 0x7f02d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:51.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.452+0000 7f02eeffd640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f02d0078bc0 0x7f02d007b0d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f02d8000f60 tx=0x7f02d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:51.458 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.456+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02e0063430 con 0x7f02f00b5210 2024-09-21T05:47:51.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:51.644+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f02f00d2a20 con 0x7f02f00b5210 2024-09-21T05:47:52.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.298+0000 7f02e7fff640 1 -- 172.21.15.22:0/1220176434 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v107) ==== 170+0+0 (secure 0 0 0) 0x7f02e0062d30 con 0x7f02f00b5210 2024-09-21T05:47:52.300 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' created 2024-09-21T05:47:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.304+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f02d0078bc0 msgr2=0x7f02d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.304+0000 7f02f61c6640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f02d0078bc0 0x7f02d007b0d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f02d8000f60 tx=0x7f02d8009040 comp rx=0 tx=0).stop 2024-09-21T05:47:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.304+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 msgr2=0x7f02f00cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.304+0000 7f02f61c6640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 0x7f02f00cc110 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f02e000d6d0 tx=0x7f02e000db90 comp rx=0 tx=0).stop 2024-09-21T05:47:52.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.305+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 shutdown_connections 2024-09-21T05:47:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.305+0000 7f02f61c6640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f02d0078bc0 0x7f02d007b0d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.305+0000 7f02f61c6640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f02f00d2310 0x7f02f00b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.305+0000 7f02f61c6640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f02f00ccbe0 0x7f02f0157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.305+0000 7f02f61c6640 1 --2- 172.21.15.22:0/1220176434 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f02f00b5210 0x7f02f00cc110 secure :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f02e000d6d0 tx=0x7f02e000db90 comp rx=0 tx=0).stop 2024-09-21T05:47:52.309 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.305+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 >> 172.21.15.22:0/1220176434 conn(0x7f02f00b3f10 msgr2=0x7f02f00b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.309+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 shutdown_connections 2024-09-21T05:47:52.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.309+0000 7f02f61c6640 1 -- 172.21.15.22:0/1220176434 wait complete. 2024-09-21T05:47:52.335 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 -- 172.21.15.22:0/4292634380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 msgr2=0x7fb84c0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 --2- 172.21.15.22:0/4292634380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0b4790 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0098e0 tx=0x7fb83c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 -- 172.21.15.22:0/4292634380 shutdown_connections 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 --2- 172.21.15.22:0/4292634380 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb84c0cffd0 0x7fb84c0d4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 --2- 172.21.15.22:0/4292634380 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0b4790 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 -- 172.21.15.22:0/4292634380 >> 172.21.15.22:0/4292634380 conn(0x7fb84c08bfc0 msgr2=0x7fb84c09d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 -- 172.21.15.22:0/4292634380 shutdown_connections 2024-09-21T05:47:52.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.459+0000 7fb85307e640 1 -- 172.21.15.22:0/4292634380 wait complete. 2024-09-21T05:47:52.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.461+0000 7fb85307e640 1 Processor -- start 2024-09-21T05:47:52.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.461+0000 7fb85307e640 1 -- start start 2024-09-21T05:47:52.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85307e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:52.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85307e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c0cfbc0 0x7fb84c0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:52.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85307e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb84c0cffd0 0x7fb84c15ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85307e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0d5000 con 0x7fb84c0b3910 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85307e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0d4e80 con 0x7fb84c0cfbc0 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85307e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c0b3340 con 0x7fb84c0cffd0 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb850e1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb850e1a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0cabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33948/0 (socket says 172.21.15.22:33948) 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb850e1a640 1 -- 172.21.15.22:0/3646669548 learned_addr learned my addr 172.21.15.22:0/3646669548 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb85161b640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb84c0cffd0 0x7fb84c15ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.462+0000 7fb84bfff640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c0cfbc0 0x7fb84c0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb850e1a640 1 -- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb84c0cffd0 msgr2=0x7fb84c15ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb850e1a640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb84c0cffd0 0x7fb84c15ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb850e1a640 1 -- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c0cfbc0 msgr2=0x7fb84c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb850e1a640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c0cfbc0 0x7fb84c0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:52.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb850e1a640 1 -- 172.21.15.22:0/3646669548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c0cdec0 con 0x7fb84c0b3910 2024-09-21T05:47:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb850e1a640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0cabe0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009a00 tx=0x7fb83c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c03f070 con 0x7fb84c0b3910 2024-09-21T05:47:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb83c004360 con 0x7fb84c0b3910 2024-09-21T05:47:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c1f9540 con 0x7fb84c0b3910 2024-09-21T05:47:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c031bb0 con 0x7fb84c0b3910 2024-09-21T05:47:52.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.463+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c1f9b00 con 0x7fb84c0b3910 2024-09-21T05:47:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.464+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84c1f9330 con 0x7fb84c0b3910 2024-09-21T05:47:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.465+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb83c042070 con 0x7fb84c0b3910 2024-09-21T05:47:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.466+0000 7fb8497fa640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb82c078b90 0x7fb82c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.466+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5609+0+0 (secure 0 0 0) 0x7fb83c0bc110 con 0x7fb84c0b3910 2024-09-21T05:47:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.466+0000 7fb84bfff640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb82c078b90 0x7fb82c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:52.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.466+0000 7fb84bfff640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb82c078b90 0x7fb82c07b0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0d1ce0 tx=0x7fb83400d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:52.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.470+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb83c031770 con 0x7fb84c0b3910 2024-09-21T05:47:52.697 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:52.695+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb84c0b5bf0 con 0x7fb84c0b3910 2024-09-21T05:47:52.793 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:47:52.796 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:47:53.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.299+0000 7fb8497fa640 1 -- 172.21.15.22:0/3646669548 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v97) ==== 174+0+0 (secure 0 0 0) 0x7fb83c083fd0 con 0x7fb84c0b3910 2024-09-21T05:47:53.302 INFO:teuthology.orchestra.run.smithi022.stderr:new fs with metadata pool 16 and data pool 17 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.303+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb82c078b90 msgr2=0x7fb82c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.303+0000 7fb85307e640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb82c078b90 0x7fb82c07b0a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0d1ce0 tx=0x7fb83400d040 comp rx=0 tx=0).stop 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.303+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 msgr2=0x7fb84c0cabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.303+0000 7fb85307e640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0cabe0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009a00 tx=0x7fb83c005270 comp rx=0 tx=0).stop 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 shutdown_connections 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb82c078b90 0x7fb82c07b0a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb84c0cffd0 0x7fb84c15ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb84c0cfbc0 0x7fb84c0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 --2- 172.21.15.22:0/3646669548 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb84c0b3910 0x7fb84c0cabe0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 >> 172.21.15.22:0/3646669548 conn(0x7fb84c08bfc0 msgr2=0x7fb84c0d4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:53.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 shutdown_connections 2024-09-21T05:47:53.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.304+0000 7fb85307e640 1 -- 172.21.15.22:0/3646669548 wait complete. 2024-09-21T05:47:53.332 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 -- 172.21.15.22:0/1788704452 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 msgr2=0x7fc2340d4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 --2- 172.21.15.22:0/1788704452 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 0x7fc2340d4d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc2300098e0 tx=0x7fc23002a4a0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 -- 172.21.15.22:0/1788704452 shutdown_connections 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 --2- 172.21.15.22:0/1788704452 >> v2:172.21.15.134:3300/0 conn(0x7fc2340cf540 0x7fc2340b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 --2- 172.21.15.22:0/1788704452 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 0x7fc2340d4d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 -- 172.21.15.22:0/1788704452 >> 172.21.15.22:0/1788704452 conn(0x7fc23409d7f0 msgr2=0x7fc2340d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 -- 172.21.15.22:0/1788704452 shutdown_connections 2024-09-21T05:47:53.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.457+0000 7fc23b382640 1 -- 172.21.15.22:0/1788704452 wait complete. 2024-09-21T05:47:53.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.459+0000 7fc23b382640 1 Processor -- start 2024-09-21T05:47:53.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.459+0000 7fc23b382640 1 -- start start 2024-09-21T05:47:53.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23b382640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 0x7fc2340cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23b382640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 0x7fc234158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23b382640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2340cf540 0x7fc2340c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23b382640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340d2e40 con 0x7fc2340b64e0 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23b382640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc2340d2cc0 con 0x7fc2340cf540 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23b382640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc2340d2fc0 con 0x7fc2340b9b60 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23911e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 0x7fc2340cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23911e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 0x7fc2340cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33950/0 (socket says 172.21.15.22:33950) 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.460+0000 7fc23911e640 1 -- 172.21.15.22:0/3121349882 learned_addr learned my addr 172.21.15.22:0/3121349882 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23891d640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 0x7fc234158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23991f640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2340cf540 0x7fc2340c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23911e640 1 -- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 msgr2=0x7fc234158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23911e640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 0x7fc234158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23911e640 1 -- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2340cf540 msgr2=0x7fc2340c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23911e640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2340cf540 0x7fc2340c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23911e640 1 -- 172.21.15.22:0/3121349882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2340ce930 con 0x7fc2340b64e0 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23911e640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 0x7fc2340cb5f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc23002aa90 tx=0x7fc230004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:53.464 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230005cf0 con 0x7fc2340b64e0 2024-09-21T05:47:53.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc230004e50 con 0x7fc2340b64e0 2024-09-21T05:47:53.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23415d3f0 con 0x7fc2340b64e0 2024-09-21T05:47:53.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.461+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc230033e60 con 0x7fc2340b64e0 2024-09-21T05:47:53.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.462+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2340ce130 con 0x7fc2340b64e0 2024-09-21T05:47:53.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.463+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2340d3470 con 0x7fc2340b64e0 2024-09-21T05:47:53.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.463+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc230005040 con 0x7fc2340b64e0 2024-09-21T05:47:53.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.464+0000 7fc229ffb640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc218078b90 0x7fc21807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.464+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7fc2300bbe10 con 0x7fc2340b64e0 2024-09-21T05:47:53.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.464+0000 7fc23891d640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc218078b90 0x7fc21807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.465+0000 7fc23891d640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc218078b90 0x7fc21807b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc2340d1ce0 tx=0x7fc224005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:53.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.469+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc23003f030 con 0x7fc2340b64e0 2024-09-21T05:47:53.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.660+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2340d4770 con 0x7fc2340b64e0 2024-09-21T05:47:53.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.661+0000 7fc229ffb640 1 -- 172.21.15.22:0/3121349882 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16688 (secure 0 0 0) 0x7fc230031770 con 0x7fc2340b64e0 2024-09-21T05:47:53.663 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:53.663 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":108,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:47:52.696982+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:47:50.363316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T05:47:51.649557+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc218078b90 msgr2=0x7fc21807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc218078b90 0x7fc21807b0a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc2340d1ce0 tx=0x7fc224005f90 comp rx=0 tx=0).stop 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 msgr2=0x7fc2340cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 0x7fc2340cb5f0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc23002aa90 tx=0x7fc230004a50 comp rx=0 tx=0).stop 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 shutdown_connections 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc2340cf540 0x7fc2340c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc218078b90 0x7fc21807b0a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc2340b9b60 0x7fc234158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 --2- 172.21.15.22:0/3121349882 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc2340b64e0 0x7fc2340cb5f0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 >> 172.21.15.22:0/3121349882 conn(0x7fc23409d7f0 msgr2=0x7fc2340b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.663+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 shutdown_connections 2024-09-21T05:47:53.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.664+0000 7fc23b382640 1 -- 172.21.15.22:0/3121349882 wait complete. 2024-09-21T05:47:53.677 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 -- 172.21.15.22:0/2748860157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240cf540 msgr2=0x7f4e2406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2748860157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240cf540 0x7f4e2406eab0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005fd0 tx=0x7f4e1c02a9e0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 -- 172.21.15.22:0/2748860157 shutdown_connections 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2748860157 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240cf540 0x7f4e2406eab0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2748860157 >> v2:172.21.15.134:3300/0 conn(0x7f4e240b66b0 0x7f4e240d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 -- 172.21.15.22:0/2748860157 >> 172.21.15.22:0/2748860157 conn(0x7f4e2409d7f0 msgr2=0x7f4e240d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 -- 172.21.15.22:0/2748860157 shutdown_connections 2024-09-21T05:47:53.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.791+0000 7f4e2a484640 1 -- 172.21.15.22:0/2748860157 wait complete. 2024-09-21T05:47:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.793+0000 7f4e2a484640 1 Processor -- start 2024-09-21T05:47:53.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.793+0000 7f4e2a484640 1 -- start start 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.794+0000 7f4e2a484640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 0x7f4e240cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.794+0000 7f4e2a484640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240cf540 0x7f4e240b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e2a484640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240b55b0 0x7f4e24156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e2a484640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406f600 con 0x7f4e240b9b60 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e2a484640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2406f480 con 0x7f4e240cf540 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e2a484640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2406f780 con 0x7f4e240b55b0 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 0x7f4e240cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.798 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 0x7f4e240cb870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33966/0 (socket says 172.21.15.22:33966) 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 -- 172.21.15.22:0/2183630420 learned_addr learned my addr 172.21.15.22:0/2183630420 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e28a21640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240b55b0 0x7f4e24156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e237fe640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240cf540 0x7f4e240b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 -- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240b55b0 msgr2=0x7f4e24156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240b55b0 0x7f4e24156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 -- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240cf540 msgr2=0x7f4e240b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240cf540 0x7f4e240b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 -- 172.21.15.22:0/2183630420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e240cbdb0 con 0x7f4e240b9b60 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e237fe640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240cf540 0x7f4e240b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e23fff640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 0x7f4e240cb870 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400c390 tx=0x7f4e1400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14017070 con 0x7f4e240b9b60 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e140035f0 con 0x7f4e240b9b60 2024-09-21T05:47:53.799 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e240ce410 con 0x7f4e240b9b60 2024-09-21T05:47:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.795+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e14003a20 con 0x7f4e240b9b60 2024-09-21T05:47:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.796+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e240c8330 con 0x7f4e240b9b60 2024-09-21T05:47:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.797+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2406fca0 con 0x7f4e240b9b60 2024-09-21T05:47:53.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.797+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4e1401a070 con 0x7f4e240b9b60 2024-09-21T05:47:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.798+0000 7f4e20ff9640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e10078bc0 0x7f4e1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.798+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f4e14099e60 con 0x7f4e240b9b60 2024-09-21T05:47:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.798+0000 7f4e237fe640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e10078bc0 0x7f4e1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:53.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.799+0000 7f4e237fe640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e10078bc0 0x7f4e1007b0d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4e240b86e0 tx=0x7f4e18005f10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:53.805 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.803+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e14061ce0 con 0x7f4e240b9b60 2024-09-21T05:47:53.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.991+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e240d3fb0 con 0x7f4e240b9b60 2024-09-21T05:47:53.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.992+0000 7f4e20ff9640 1 -- 172.21.15.22:0/2183630420 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16688 (secure 0 0 0) 0x7f4e14066710 con 0x7f4e240b9b60 2024-09-21T05:47:53.995 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:53.995 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":108,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:47:52.696982+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:47:50.363316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T05:47:51.649557+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:47:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e10078bc0 msgr2=0x7f4e1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e10078bc0 0x7f4e1007b0d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4e240b86e0 tx=0x7f4e18005f10 comp rx=0 tx=0).stop 2024-09-21T05:47:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 msgr2=0x7f4e240cb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 0x7f4e240cb870 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400c390 tx=0x7f4e1400c850 comp rx=0 tx=0).stop 2024-09-21T05:47:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 shutdown_connections 2024-09-21T05:47:53.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4e10078bc0 0x7f4e1007b0d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4e240b55b0 0x7f4e24156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4e240cf540 0x7f4e240b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 --2- 172.21.15.22:0/2183630420 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4e240b9b60 0x7f4e240cb870 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 >> 172.21.15.22:0/2183630420 conn(0x7f4e2409d7f0 msgr2=0x7f4e24161b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 shutdown_connections 2024-09-21T05:47:53.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:53.995+0000 7f4e2a484640 1 -- 172.21.15.22:0/2183630420 wait complete. 2024-09-21T05:47:54.008 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T05:47:54.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 -- 172.21.15.22:0/566202461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 msgr2=0x7f5e280b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:54.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/566202461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 0x7f5e280b5fa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5e18009900 tx=0x7f5e1802a610 comp rx=0 tx=0).stop 2024-09-21T05:47:54.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 -- 172.21.15.22:0/566202461 shutdown_connections 2024-09-21T05:47:54.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/566202461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 0x7f5e280b5fa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:54.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/566202461 >> v2:172.21.15.134:3300/0 conn(0x7f5e280d4c30 0x7f5e280b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:54.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 -- 172.21.15.22:0/566202461 >> 172.21.15.22:0/566202461 conn(0x7f5e2809d830 msgr2=0x7f5e280d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:54.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 -- 172.21.15.22:0/566202461 shutdown_connections 2024-09-21T05:47:54.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.126+0000 7f5e2f00a640 1 -- 172.21.15.22:0/566202461 wait complete. 2024-09-21T05:47:54.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.128+0000 7f5e2f00a640 1 Processor -- start 2024-09-21T05:47:54.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.128+0000 7f5e2f00a640 1 -- start start 2024-09-21T05:47:54.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.129+0000 7f5e2f00a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 0x7f5e281605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.129+0000 7f5e2f00a640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e280b64e0 0x7f5e281f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.129+0000 7f5e2f00a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 0x7f5e2809e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.129+0000 7f5e2f00a640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5e280cf610 con 0x7f5e280d4c30 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.129+0000 7f5e2f00a640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5e280cf490 con 0x7f5e280b64e0 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.129+0000 7f5e2f00a640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5e280cf790 con 0x7f5e280b3910 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 0x7f5e2809e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 0x7f5e2809e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:33972/0 (socket says 172.21.15.22:33972) 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 -- 172.21.15.22:0/2466721156 learned_addr learned my addr 172.21.15.22:0/2466721156 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2cda6640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 0x7f5e281605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e1ffff640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e280b64e0 0x7f5e281f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 -- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 msgr2=0x7f5e281605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 0x7f5e281605d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 -- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e280b64e0 msgr2=0x7f5e281f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:54.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e280b64e0 0x7f5e281f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 -- 172.21.15.22:0/2466721156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e28160f40 con 0x7f5e280d4c30 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e1ffff640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e280b64e0 0x7f5e281f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2d5a7640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 0x7f5e2809e460 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000c700 tx=0x7f5e2000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20003b70 con 0x7f5e280d4c30 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e20003d10 con 0x7f5e280d4c30 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.130+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e28161220 con 0x7f5e280d4c30 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.131+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000aab0 con 0x7f5e280d4c30 2024-09-21T05:47:54.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.131+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e281617e0 con 0x7f5e280d4c30 2024-09-21T05:47:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.131+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e280d2270 con 0x7f5e280d4c30 2024-09-21T05:47:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.132+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5e2000ac50 con 0x7f5e280d4c30 2024-09-21T05:47:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.133+0000 7f5e1d7fa640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5e14078bc0 0x7f5e1407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.133+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f5e2009aa80 con 0x7f5e280d4c30 2024-09-21T05:47:54.135 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.133+0000 7f5e2cda6640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5e14078bc0 0x7f5e1407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:54.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.134+0000 7f5e2cda6640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5e14078bc0 0x7f5e1407b0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009c00 tx=0x7f5e24005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:54.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.138+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e280d2270 con 0x7f5e280d4c30 2024-09-21T05:47:54.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:54.368+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5e28161fe0 con 0x7f5e280d4c30 2024-09-21T05:47:55.285 INFO:tasks.ceph.mds.b.smithi134.stderr:2024-09-21T05:47:55.282+0000 7f8c3e190640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:47:55.327 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.325+0000 7f5e1d7fa640 1 -- 172.21.15.22:0/2466721156 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) ==== 118+0+0 (secure 0 0 0) 0x7f5e2009ce20 con 0x7f5e280d4c30 2024-09-21T05:47:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5e14078bc0 msgr2=0x7f5e1407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5e14078bc0 0x7f5e1407b0d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009c00 tx=0x7f5e24005f30 comp rx=0 tx=0).stop 2024-09-21T05:47:55.330 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 msgr2=0x7f5e2809e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 0x7f5e2809e460 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000c700 tx=0x7f5e2000cbc0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 shutdown_connections 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5e280d4c30 0x7f5e2809e460 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5e280b64e0 0x7f5e281f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5e14078bc0 0x7f5e1407b0d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 --2- 172.21.15.22:0/2466721156 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5e280b3910 0x7f5e281605d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 >> 172.21.15.22:0/2466721156 conn(0x7f5e2809d830 msgr2=0x7f5e280d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 shutdown_connections 2024-09-21T05:47:55.331 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.328+0000 7f5e2f00a640 1 -- 172.21.15.22:0/2466721156 wait complete. 2024-09-21T05:47:55.344 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T05:47:55.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.469+0000 7fe461b39640 1 -- 172.21.15.22:0/3542241202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c0705d0 msgr2=0x7fe45c0b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.469+0000 7fe461b39640 1 --2- 172.21.15.22:0/3542241202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c0705d0 0x7fe45c0b3550 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe45c071020 tx=0x7fe45002a5f0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.470+0000 7fe461b39640 1 -- 172.21.15.22:0/3542241202 shutdown_connections 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.470+0000 7fe461b39640 1 --2- 172.21.15.22:0/3542241202 >> v2:172.21.15.134:3300/0 conn(0x7fe45c0cfeb0 0x7fe45c0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.470+0000 7fe461b39640 1 --2- 172.21.15.22:0/3542241202 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c0705d0 0x7fe45c0b3550 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.470+0000 7fe461b39640 1 -- 172.21.15.22:0/3542241202 >> 172.21.15.22:0/3542241202 conn(0x7fe45c0cdcc0 msgr2=0x7fe45c0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.470+0000 7fe461b39640 1 -- 172.21.15.22:0/3542241202 shutdown_connections 2024-09-21T05:47:55.472 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.470+0000 7fe461b39640 1 -- 172.21.15.22:0/3542241202 wait complete. 2024-09-21T05:47:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.471+0000 7fe461b39640 1 Processor -- start 2024-09-21T05:47:55.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.472+0000 7fe461b39640 1 -- start start 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe461b39640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe45c0b4090 0x7fe45c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe461b39640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c0cfeb0 0x7fe45c1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe461b39640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 0x7fe45c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe461b39640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c0b4a30 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe461b39640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c0b48b0 con 0x7fe45c0cfeb0 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe461b39640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe45c0b4bb0 con 0x7fe45c0b4090 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 0x7fe45c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 0x7fe45c156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50476/0 (socket says 172.21.15.22:50476) 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe45bfff640 1 -- 172.21.15.22:0/897805433 learned_addr learned my addr 172.21.15.22:0/897805433 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe45affd640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c0cfeb0 0x7fe45c1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.473+0000 7fe45b7fe640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe45c0b4090 0x7fe45c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe45bfff640 1 -- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe45c0b4090 msgr2=0x7fe45c1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:55.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe45bfff640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe45c0b4090 0x7fe45c1594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe45bfff640 1 -- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c0cfeb0 msgr2=0x7fe45c1f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe45bfff640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c0cfeb0 0x7fe45c1f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe45bfff640 1 -- 172.21.15.22:0/897805433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c1e6400 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe45bfff640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 0x7fe45c156690 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe45400e990 tx=0x7fe45400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454004d20 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe45400caa0 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c1e5c20 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454005020 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.474+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c1f4e40 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.475+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c1e5e90 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.476+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe4540051c0 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.477+0000 7fe443fff640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe448078b90 0x7fe44807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.477+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fe45409b660 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.477+0000 7fe45b7fe640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe448078b90 0x7fe44807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:55.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.478+0000 7fe45b7fe640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe448078b90 0x7fe44807b0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe44c001720 tx=0x7fe44c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:55.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.481+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe454016030 con 0x7fe45c1e4cd0 2024-09-21T05:47:55.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:55.708+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe45c0b5060 con 0x7fe45c1e4cd0 2024-09-21T05:47:56.333 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.331+0000 7fe443fff640 1 -- 172.21.15.22:0/897805433 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) ==== 122+0+0 (secure 0 0 0) 0x7fe454063440 con 0x7fe45c1e4cd0 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe448078b90 msgr2=0x7fe44807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe448078b90 0x7fe44807b0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe44c001720 tx=0x7fe44c009040 comp rx=0 tx=0).stop 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 msgr2=0x7fe45c156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 0x7fe45c156690 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fe45400e990 tx=0x7fe45400ee50 comp rx=0 tx=0).stop 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 shutdown_connections 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe45c1e4cd0 0x7fe45c156690 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe45c0cfeb0 0x7fe45c1f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe448078b90 0x7fe44807b0a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 --2- 172.21.15.22:0/897805433 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe45c0b4090 0x7fe45c1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 >> 172.21.15.22:0/897805433 conn(0x7fe45c0cdcc0 msgr2=0x7fe45c0b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 shutdown_connections 2024-09-21T05:47:56.337 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.334+0000 7fe461b39640 1 -- 172.21.15.22:0/897805433 wait complete. 2024-09-21T05:47:56.351 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T05:47:56.351 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:47:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 -- 172.21.15.22:0/2662774232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c075170 msgr2=0x7f792c0b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2662774232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c075170 0x7f792c0b51d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f791c005fd0 tx=0x7f791c02a560 comp rx=0 tx=0).stop 2024-09-21T05:47:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 -- 172.21.15.22:0/2662774232 shutdown_connections 2024-09-21T05:47:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2662774232 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c070af0 0x7f792c070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2662774232 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c075170 0x7f792c0b51d0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 -- 172.21.15.22:0/2662774232 >> 172.21.15.22:0/2662774232 conn(0x7f792c0cdcc0 msgr2=0x7f792c0ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 -- 172.21.15.22:0/2662774232 shutdown_connections 2024-09-21T05:47:56.476 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.473+0000 7f7932f2b640 1 -- 172.21.15.22:0/2662774232 wait complete. 2024-09-21T05:47:56.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.475+0000 7f7932f2b640 1 Processor -- start 2024-09-21T05:47:56.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.475+0000 7f7932f2b640 1 -- start start 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f7932f2b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c070af0 0x7f792c1587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f7932f2b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c0b5710 0x7f792c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f7932f2b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 0x7f792c0d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f7932f2b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f792c071880 con 0x7f792c158d00 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f7932f2b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f792c071700 con 0x7f792c0b5710 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f7932f2b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f792c071a00 con 0x7f792c070af0 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f79314c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 0x7f792c0d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f79314c8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 0x7f792c0d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50496/0 (socket says 172.21.15.22:50496) 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.476+0000 7f79314c8640 1 -- 172.21.15.22:0/2654668154 learned_addr learned my addr 172.21.15.22:0/2654668154 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f792bfff640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c0b5710 0x7f792c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f7930cc7640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c070af0 0x7f792c1587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79314c8640 1 -- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c070af0 msgr2=0x7f792c1587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79314c8640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c070af0 0x7f792c1587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79314c8640 1 -- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c0b5710 msgr2=0x7f792c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79314c8640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c0b5710 0x7f792c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79314c8640 1 -- 172.21.15.22:0/2654668154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c0cee10 con 0x7f792c158d00 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f792bfff640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c0b5710 0x7f792c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79314c8640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 0x7f792c0d04b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f792400ba10 tx=0x7f792400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7924019070 con 0x7f792c158d00 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f792400cc90 con 0x7f792c158d00 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c15bf30 con 0x7f792c158d00 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79240146a0 con 0x7f792c158d00 2024-09-21T05:47:56.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.477+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c152d10 con 0x7f792c158d00 2024-09-21T05:47:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.478+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c075170 con 0x7f792c158d00 2024-09-21T05:47:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.479+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f792401d3f0 con 0x7f792c158d00 2024-09-21T05:47:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.480+0000 7f79297fa640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7918078bc0 0x7f791807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.480+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f792409a9d0 con 0x7f792c158d00 2024-09-21T05:47:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.480+0000 7f7930cc7640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7918078bc0 0x7f791807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.481+0000 7f7930cc7640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7918078bc0 0x7f791807b0d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f791c033000 tx=0x7f791c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:56.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.484+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7924062d70 con 0x7f792c158d00 2024-09-21T05:47:56.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.710+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f792c0aec20 con 0x7f792c158d00 2024-09-21T05:47:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.710+0000 7f79297fa640 1 -- 172.21.15.22:0/2654668154 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5230 (secure 0 0 0) 0x7f7924062670 con 0x7f792c158d00 2024-09-21T05:47:56.713 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:56.713 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":101,"btime":"2024-09-21T05:47:56:326350+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3772214991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3772214991},{"type":"v1","addr":"172.21.15.22:6835","nonce":3772214991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:47:55.710024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8055":{"gid":8055,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/4249862141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":4249862141},{"type":"v1","addr":"172.21.15.134:6837","nonce":4249862141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8055,"qdb_cluster":[8055]},"id":8}]} 2024-09-21T05:47:56.713 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 101 2024-09-21T05:47:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7918078bc0 msgr2=0x7f791807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7918078bc0 0x7f791807b0d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f791c033000 tx=0x7f791c004730 comp rx=0 tx=0).stop 2024-09-21T05:47:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 msgr2=0x7f792c0d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 0x7f792c0d04b0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f792400ba10 tx=0x7f792400bed0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 shutdown_connections 2024-09-21T05:47:56.715 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f792c158d00 0x7f792c0d04b0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f792c0b5710 0x7f792c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7918078bc0 0x7f791807b0d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 --2- 172.21.15.22:0/2654668154 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f792c070af0 0x7f792c1587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 >> 172.21.15.22:0/2654668154 conn(0x7f792c0cdcc0 msgr2=0x7f792c0b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 shutdown_connections 2024-09-21T05:47:56.716 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.713+0000 7f7932f2b640 1 -- 172.21.15.22:0/2654668154 wait complete. 2024-09-21T05:47:56.726 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 -- 172.21.15.22:0/1248433867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc0d0e50 msgr2=0x7f10bc0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 --2- 172.21.15.22:0/1248433867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc0d0e50 0x7f10bc0d1260 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0cebb0 tx=0x7f10ac02a440 comp rx=0 tx=0).stop 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 -- 172.21.15.22:0/1248433867 shutdown_connections 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 --2- 172.21.15.22:0/1248433867 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc0d0e50 0x7f10bc0d1260 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 --2- 172.21.15.22:0/1248433867 >> v2:172.21.15.134:3300/0 conn(0x7f10bc0b97e0 0x7f10bc06f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 -- 172.21.15.22:0/1248433867 >> 172.21.15.22:0/1248433867 conn(0x7f10bc09d840 msgr2=0x7f10bc08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 -- 172.21.15.22:0/1248433867 shutdown_connections 2024-09-21T05:47:56.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.841+0000 7f10c1107640 1 -- 172.21.15.22:0/1248433867 wait complete. 2024-09-21T05:47:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.843+0000 7f10c1107640 1 Processor -- start 2024-09-21T05:47:56.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.843+0000 7f10c1107640 1 -- start start 2024-09-21T05:47:56.847 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10c1107640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 0x7f10bc1f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10c1107640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10bc0b97e0 0x7f10bc1f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10c1107640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10bc0d0e50 0x7f10bc14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10c1107640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0cfea0 con 0x7f10bc06e0a0 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10c1107640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f10bc0cfd20 con 0x7f10bc0b97e0 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10c1107640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f10bc0d0020 con 0x7f10bc0d0e50 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 0x7f10bc1f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10bad9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 0x7f10bc1f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50510/0 (socket says 172.21.15.22:50510) 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10ba59c640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10bc0b97e0 0x7f10bc1f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.844+0000 7f10bad9d640 1 -- 172.21.15.22:0/1627090469 learned_addr learned my addr 172.21.15.22:0/1627090469 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bb59e640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10bc0d0e50 0x7f10bc14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bad9d640 1 -- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10bc0d0e50 msgr2=0x7f10bc14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bad9d640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10bc0d0e50 0x7f10bc14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bad9d640 1 -- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10bc0b97e0 msgr2=0x7f10bc1f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bad9d640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10bc0b97e0 0x7f10bc1f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:56.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bad9d640 1 -- 172.21.15.22:0/1627090469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10bc1f5530 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10bad9d640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 0x7f10bc1f5ff0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f10b400ba50 tx=0x7f10b400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4019070 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10b400cd00 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10bc1f9620 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b40146a0 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.845+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10bc1f57f0 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.846+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10bc0d03f0 con 0x7f10bc06e0a0 2024-09-21T05:47:56.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.847+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f10b4007370 con 0x7f10bc06e0a0 2024-09-21T05:47:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.848+0000 7f10b37fe640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f109c078b90 0x7f109c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.848+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f10b409aca0 con 0x7f10bc06e0a0 2024-09-21T05:47:56.850 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.848+0000 7f10ba59c640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f109c078b90 0x7f109c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:56.851 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.849+0000 7f10ba59c640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f109c078b90 0x7f109c07b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0b8b90 tx=0x7f10a400a2d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:56.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:56.853+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10b4062aa0 con 0x7f10bc06e0a0 2024-09-21T05:47:57.044 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.042+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10bc0cf3a0 con 0x7f10bc06e0a0 2024-09-21T05:47:57.045 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.043+0000 7f10b37fe640 1 -- 172.21.15.22:0/1627090469 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16675 (secure 0 0 0) 0x7f10b40674d0 con 0x7f10bc06e0a0 2024-09-21T05:47:57.045 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:57.045 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":109,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:47:54.312445+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:47:50.363316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T05:47:51.649557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:47:57.047 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f109c078b90 msgr2=0x7f109c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f109c078b90 0x7f109c07b0a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0b8b90 tx=0x7f10a400a2d0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 msgr2=0x7f10bc1f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 0x7f10bc1f5ff0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f10b400ba50 tx=0x7f10b400bf10 comp rx=0 tx=0).stop 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 shutdown_connections 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f109c078b90 0x7f109c07b0a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f10bc0d0e50 0x7f10bc14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f10bc0b97e0 0x7f10bc1f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 --2- 172.21.15.22:0/1627090469 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f10bc06e0a0 0x7f10bc1f5ff0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 >> 172.21.15.22:0/1627090469 conn(0x7f10bc09d840 msgr2=0x7f10bc0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 shutdown_connections 2024-09-21T05:47:57.048 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.045+0000 7f10c1107640 1 -- 172.21.15.22:0/1627090469 wait complete. 2024-09-21T05:47:57.059 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:47:57.060 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 -- 172.21.15.22:0/3050811698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480cf540 msgr2=0x7f514806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 --2- 172.21.15.22:0/3050811698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480cf540 0x7f514806eab0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5140005fd0 tx=0x7f514002a530 comp rx=0 tx=0).stop 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 -- 172.21.15.22:0/3050811698 shutdown_connections 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 --2- 172.21.15.22:0/3050811698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480cf540 0x7f514806eab0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 --2- 172.21.15.22:0/3050811698 >> v2:172.21.15.134:3300/0 conn(0x7f51480b66b0 0x7f51480d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 -- 172.21.15.22:0/3050811698 >> 172.21.15.22:0/3050811698 conn(0x7f514809d7f0 msgr2=0x7f51480d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 -- 172.21.15.22:0/3050811698 shutdown_connections 2024-09-21T05:47:57.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.174+0000 7f514d339640 1 -- 172.21.15.22:0/3050811698 wait complete. 2024-09-21T05:47:57.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.176+0000 7f514d339640 1 Processor -- start 2024-09-21T05:47:57.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.176+0000 7f514d339640 1 -- start start 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f514d339640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 0x7f51481549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f514d339640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51480cf540 0x7f51480ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f514d339640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51481602f0 0x7f51480b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f514d339640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f514806f600 con 0x7f51480b9b60 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f514d339640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f514806f480 con 0x7f51481602f0 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f514d339640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f514806f780 con 0x7f51480cf540 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 0x7f51481549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 0x7f51481549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50528/0 (socket says 172.21.15.22:50528) 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 -- 172.21.15.22:0/3151675318 learned_addr learned my addr 172.21.15.22:0/3151675318 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:57.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f51477fe640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51481602f0 0x7f51480b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f51467fc640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51480cf540 0x7f51480ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 -- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51480cf540 msgr2=0x7f51480ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51480cf540 0x7f51480ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 -- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51481602f0 msgr2=0x7f51480b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51481602f0 0x7f51480b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.177+0000 7f5146ffd640 1 -- 172.21.15.22:0/3151675318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5148154f00 con 0x7f51480b9b60 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.178+0000 7f5146ffd640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 0x7f51481549c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f513800c730 tx=0x7f513800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.178+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138017070 con 0x7f51480b9b60 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.178+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f513800ce40 con 0x7f51480b9b60 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.178+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514815fb30 con 0x7f51480b9b60 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.178+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380052d0 con 0x7f51480b9b60 2024-09-21T05:47:57.181 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.178+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514815d420 con 0x7f51480b9b60 2024-09-21T05:47:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.179+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514806fd00 con 0x7f51480b9b60 2024-09-21T05:47:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.180+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f513801a070 con 0x7f51480b9b60 2024-09-21T05:47:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.180+0000 7f513f7fe640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5128078b90 0x7f512807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.180+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f513809a440 con 0x7f51480b9b60 2024-09-21T05:47:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.181+0000 7f51467fc640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5128078b90 0x7f512807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.181+0000 7f51467fc640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5128078b90 0x7f512807b0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f51480c7c60 tx=0x7f5130008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:57.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.185+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5148154f00 con 0x7f51480b9b60 2024-09-21T05:47:57.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.369+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51481618b0 con 0x7f5128078b90 2024-09-21T05:47:57.376 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.375+0000 7f513f7fe640 1 -- 172.21.15.22:0/3151675318 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249123 (secure 0 0 0) 0x7f51481618b0 con 0x7f5128078b90 2024-09-21T05:47:57.379 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:57.380 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5128078b90 msgr2=0x7f512807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5128078b90 0x7f512807b0a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f51480c7c60 tx=0x7f5130008040 comp rx=0 tx=0).stop 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 msgr2=0x7f51481549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 0x7f51481549c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f513800c730 tx=0x7f513800cbf0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 shutdown_connections 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f51481602f0 0x7f51480b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.382 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5128078b90 0x7f512807b0a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f51480cf540 0x7f51480ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 --2- 172.21.15.22:0/3151675318 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f51480b9b60 0x7f51481549c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 >> 172.21.15.22:0/3151675318 conn(0x7f514809d7f0 msgr2=0x7f51480d2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 shutdown_connections 2024-09-21T05:47:57.383 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.378+0000 7f514d339640 1 -- 172.21.15.22:0/3151675318 wait complete. 2024-09-21T05:47:57.391 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-21T05:47:55.877407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282012,"kb_used_data":11660,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754692708,"statfs":{"total":773094113280,"available":772805332992,"internally_reserved":0,"allocated":11939840,"data_stored":5812063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584816"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421407+0000","last_change":"2024-09-21T05:47:53.329400+0000","last_active":"2024-09-21T05:47:54.421407+0000","last_peered":"2024-09-21T05:47:54.421407+0000","last_clean":"2024-09-21T05:47:54.421407+0000","last_became_active":"2024-09-21T05:47:53.328865+0000","last_became_peered":"2024-09-21T05:47:53.328865+0000","last_unstale":"2024-09-21T05:47:54.421407+0000","last_undegraded":"2024-09-21T05:47:54.421407+0000","last_fullsized":"2024-09-21T05:47:54.421407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:33.666746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420973+0000","last_change":"2024-09-21T05:47:53.338050+0000","last_active":"2024-09-21T05:47:54.420973+0000","last_peered":"2024-09-21T05:47:54.420973+0000","last_clean":"2024-09-21T05:47:54.420973+0000","last_became_active":"2024-09-21T05:47:53.337279+0000","last_became_peered":"2024-09-21T05:47:53.337279+0000","last_unstale":"2024-09-21T05:47:54.420973+0000","last_undegraded":"2024-09-21T05:47:54.420973+0000","last_fullsized":"2024-09-21T05:47:54.420973+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:49.343256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320073+0000","last_change":"2024-09-21T05:47:53.330191+0000","last_active":"2024-09-21T05:47:54.320073+0000","last_peered":"2024-09-21T05:47:54.320073+0000","last_clean":"2024-09-21T05:47:54.320073+0000","last_became_active":"2024-09-21T05:47:53.329578+0000","last_became_peered":"2024-09-21T05:47:53.329578+0000","last_unstale":"2024-09-21T05:47:54.320073+0000","last_undegraded":"2024-09-21T05:47:54.320073+0000","last_fullsized":"2024-09-21T05:47:54.320073+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:51.237035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420158+0000","last_change":"2024-09-21T05:47:53.314749+0000","last_active":"2024-09-21T05:47:54.420158+0000","last_peered":"2024-09-21T05:47:54.420158+0000","last_clean":"2024-09-21T05:47:54.420158+0000","last_became_active":"2024-09-21T05:47:53.314457+0000","last_became_peered":"2024-09-21T05:47:53.314457+0000","last_unstale":"2024-09-21T05:47:54.420158+0000","last_undegraded":"2024-09-21T05:47:54.420158+0000","last_fullsized":"2024-09-21T05:47:54.420158+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:39.746882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419166+0000","last_change":"2024-09-21T05:47:53.322397+0000","last_active":"2024-09-21T05:47:54.419166+0000","last_peered":"2024-09-21T05:47:54.419166+0000","last_clean":"2024-09-21T05:47:54.419166+0000","last_became_active":"2024-09-21T05:47:53.322048+0000","last_became_peered":"2024-09-21T05:47:53.322048+0000","last_unstale":"2024-09-21T05:47:54.419166+0000","last_undegraded":"2024-09-21T05:47:54.419166+0000","last_fullsized":"2024-09-21T05:47:54.419166+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:37.914628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421377+0000","last_change":"2024-09-21T05:47:53.333144+0000","last_active":"2024-09-21T05:47:54.421377+0000","last_peered":"2024-09-21T05:47:54.421377+0000","last_clean":"2024-09-21T05:47:54.421377+0000","last_became_active":"2024-09-21T05:47:53.332816+0000","last_became_peered":"2024-09-21T05:47:53.332816+0000","last_unstale":"2024-09-21T05:47:54.421377+0000","last_undegraded":"2024-09-21T05:47:54.421377+0000","last_fullsized":"2024-09-21T05:47:54.421377+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:44.276704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421287+0000","last_change":"2024-09-21T05:47:53.328426+0000","last_active":"2024-09-21T05:47:54.421287+0000","last_peered":"2024-09-21T05:47:54.421287+0000","last_clean":"2024-09-21T05:47:54.421287+0000","last_became_active":"2024-09-21T05:47:53.327992+0000","last_became_peered":"2024-09-21T05:47:53.327992+0000","last_unstale":"2024-09-21T05:47:54.421287+0000","last_undegraded":"2024-09-21T05:47:54.421287+0000","last_fullsized":"2024-09-21T05:47:54.421287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:02.621431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421042+0000","last_change":"2024-09-21T05:47:53.338328+0000","last_active":"2024-09-21T05:47:54.421042+0000","last_peered":"2024-09-21T05:47:54.421042+0000","last_clean":"2024-09-21T05:47:54.421042+0000","last_became_active":"2024-09-21T05:47:53.337637+0000","last_became_peered":"2024-09-21T05:47:53.337637+0000","last_unstale":"2024-09-21T05:47:54.421042+0000","last_undegraded":"2024-09-21T05:47:54.421042+0000","last_fullsized":"2024-09-21T05:47:54.421042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:07.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419472+0000","last_change":"2024-09-21T05:47:53.314013+0000","last_active":"2024-09-21T05:47:54.419472+0000","last_peered":"2024-09-21T05:47:54.419472+0000","last_clean":"2024-09-21T05:47:54.419472+0000","last_became_active":"2024-09-21T05:47:53.313613+0000","last_became_peered":"2024-09-21T05:47:53.313613+0000","last_unstale":"2024-09-21T05:47:54.419472+0000","last_undegraded":"2024-09-21T05:47:54.419472+0000","last_fullsized":"2024-09-21T05:47:54.419472+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:09.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420533+0000","last_change":"2024-09-21T05:47:53.340669+0000","last_active":"2024-09-21T05:47:54.420533+0000","last_peered":"2024-09-21T05:47:54.420533+0000","last_clean":"2024-09-21T05:47:54.420533+0000","last_became_active":"2024-09-21T05:47:53.340336+0000","last_became_peered":"2024-09-21T05:47:53.340336+0000","last_unstale":"2024-09-21T05:47:54.420533+0000","last_undegraded":"2024-09-21T05:47:54.420533+0000","last_fullsized":"2024-09-21T05:47:54.420533+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:28.758224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323869+0000","last_change":"2024-09-21T05:47:53.317110+0000","last_active":"2024-09-21T05:47:54.323869+0000","last_peered":"2024-09-21T05:47:54.323869+0000","last_clean":"2024-09-21T05:47:54.323869+0000","last_became_active":"2024-09-21T05:47:53.316725+0000","last_became_peered":"2024-09-21T05:47:53.316725+0000","last_unstale":"2024-09-21T05:47:54.323869+0000","last_undegraded":"2024-09-21T05:47:54.323869+0000","last_fullsized":"2024-09-21T05:47:54.323869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.043120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419862+0000","last_change":"2024-09-21T05:47:53.322155+0000","last_active":"2024-09-21T05:47:54.419862+0000","last_peered":"2024-09-21T05:47:54.419862+0000","last_clean":"2024-09-21T05:47:54.419862+0000","last_became_active":"2024-09-21T05:47:53.321833+0000","last_became_peered":"2024-09-21T05:47:53.321833+0000","last_unstale":"2024-09-21T05:47:54.419862+0000","last_undegraded":"2024-09-21T05:47:54.419862+0000","last_fullsized":"2024-09-21T05:47:54.419862+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:01.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419203+0000","last_change":"2024-09-21T05:47:53.320400+0000","last_active":"2024-09-21T05:47:54.419203+0000","last_peered":"2024-09-21T05:47:54.419203+0000","last_clean":"2024-09-21T05:47:54.419203+0000","last_became_active":"2024-09-21T05:47:53.320056+0000","last_became_peered":"2024-09-21T05:47:53.320056+0000","last_unstale":"2024-09-21T05:47:54.419203+0000","last_undegraded":"2024-09-21T05:47:54.419203+0000","last_fullsized":"2024-09-21T05:47:54.419203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:51.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421261+0000","last_change":"2024-09-21T05:47:53.335796+0000","last_active":"2024-09-21T05:47:54.421261+0000","last_peered":"2024-09-21T05:47:54.421261+0000","last_clean":"2024-09-21T05:47:54.421261+0000","last_became_active":"2024-09-21T05:47:53.335426+0000","last_became_peered":"2024-09-21T05:47:53.335426+0000","last_unstale":"2024-09-21T05:47:54.421261+0000","last_undegraded":"2024-09-21T05:47:54.421261+0000","last_fullsized":"2024-09-21T05:47:54.421261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:32.715401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419441+0000","last_change":"2024-09-21T05:47:53.328721+0000","last_active":"2024-09-21T05:47:54.419441+0000","last_peered":"2024-09-21T05:47:54.419441+0000","last_clean":"2024-09-21T05:47:54.419441+0000","last_became_active":"2024-09-21T05:47:53.328431+0000","last_became_peered":"2024-09-21T05:47:53.328431+0000","last_unstale":"2024-09-21T05:47:54.419441+0000","last_undegraded":"2024-09-21T05:47:54.419441+0000","last_fullsized":"2024-09-21T05:47:54.419441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:16.400560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420065+0000","last_change":"2024-09-21T05:47:53.339459+0000","last_active":"2024-09-21T05:47:54.420065+0000","last_peered":"2024-09-21T05:47:54.420065+0000","last_clean":"2024-09-21T05:47:54.420065+0000","last_became_active":"2024-09-21T05:47:53.339054+0000","last_became_peered":"2024-09-21T05:47:53.339054+0000","last_unstale":"2024-09-21T05:47:54.420065+0000","last_undegraded":"2024-09-21T05:47:54.420065+0000","last_fullsized":"2024-09-21T05:47:54.420065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:16.576540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419434+0000","last_change":"2024-09-21T05:47:53.323957+0000","last_active":"2024-09-21T05:47:54.419434+0000","last_peered":"2024-09-21T05:47:54.419434+0000","last_clean":"2024-09-21T05:47:54.419434+0000","last_became_active":"2024-09-21T05:47:53.323432+0000","last_became_peered":"2024-09-21T05:47:53.323432+0000","last_unstale":"2024-09-21T05:47:54.419434+0000","last_undegraded":"2024-09-21T05:47:54.419434+0000","last_fullsized":"2024-09-21T05:47:54.419434+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:57.449767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420771+0000","last_change":"2024-09-21T05:47:53.340040+0000","last_active":"2024-09-21T05:47:54.420771+0000","last_peered":"2024-09-21T05:47:54.420771+0000","last_clean":"2024-09-21T05:47:54.420771+0000","last_became_active":"2024-09-21T05:47:53.339702+0000","last_became_peered":"2024-09-21T05:47:53.339702+0000","last_unstale":"2024-09-21T05:47:54.420771+0000","last_undegraded":"2024-09-21T05:47:54.420771+0000","last_fullsized":"2024-09-21T05:47:54.420771+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:02.133659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.320519+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.320182+0000","last_became_peered":"2024-09-21T05:47:53.320182+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:16.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419700+0000","last_change":"2024-09-21T05:47:53.327229+0000","last_active":"2024-09-21T05:47:54.419700+0000","last_peered":"2024-09-21T05:47:54.419700+0000","last_clean":"2024-09-21T05:47:54.419700+0000","last_became_active":"2024-09-21T05:47:53.326881+0000","last_became_peered":"2024-09-21T05:47:53.326881+0000","last_unstale":"2024-09-21T05:47:54.419700+0000","last_undegraded":"2024-09-21T05:47:54.419700+0000","last_fullsized":"2024-09-21T05:47:54.419700+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:43.659296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":19,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419430+0000","last_change":"2024-09-21T05:47:52.315229+0000","last_active":"2024-09-21T05:47:54.419430+0000","last_peered":"2024-09-21T05:47:54.419430+0000","last_clean":"2024-09-21T05:47:54.419430+0000","last_became_active":"2024-09-21T05:47:52.314905+0000","last_became_peered":"2024-09-21T05:47:52.314905+0000","last_unstale":"2024-09-21T05:47:54.419430+0000","last_undegraded":"2024-09-21T05:47:54.419430+0000","last_fullsized":"2024-09-21T05:47:54.419430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:00.549432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323756+0000","last_change":"2024-09-21T05:47:53.325534+0000","last_active":"2024-09-21T05:47:54.323756+0000","last_peered":"2024-09-21T05:47:54.323756+0000","last_clean":"2024-09-21T05:47:54.323756+0000","last_became_active":"2024-09-21T05:47:53.325200+0000","last_became_peered":"2024-09-21T05:47:53.325200+0000","last_unstale":"2024-09-21T05:47:54.323756+0000","last_undegraded":"2024-09-21T05:47:54.323756+0000","last_fullsized":"2024-09-21T05:47:54.323756+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:49.932618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T05:47:51.300628+0000","last_change":"2024-09-21T05:47:51.300628+0000","last_active":"2024-09-21T05:47:51.289529+0000","last_peered":"2024-09-21T05:47:51.289529+0000","last_clean":"2024-09-21T05:47:51.289529+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:51.300628+0000","last_undegraded":"2024-09-21T05:47:51.300628+0000","last_fullsized":"2024-09-21T05:47:51.300628+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420098+0000","last_change":"2024-09-21T05:47:53.314734+0000","last_active":"2024-09-21T05:47:54.420098+0000","last_peered":"2024-09-21T05:47:54.420098+0000","last_clean":"2024-09-21T05:47:54.420098+0000","last_became_active":"2024-09-21T05:47:53.314373+0000","last_became_peered":"2024-09-21T05:47:53.314373+0000","last_unstale":"2024-09-21T05:47:54.420098+0000","last_undegraded":"2024-09-21T05:47:54.420098+0000","last_fullsized":"2024-09-21T05:47:54.420098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:32.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319979+0000","last_change":"2024-09-21T05:47:52.320521+0000","last_active":"2024-09-21T05:47:54.319979+0000","last_peered":"2024-09-21T05:47:54.319979+0000","last_clean":"2024-09-21T05:47:54.319979+0000","last_became_active":"2024-09-21T05:47:52.320077+0000","last_became_peered":"2024-09-21T05:47:52.320077+0000","last_unstale":"2024-09-21T05:47:54.319979+0000","last_undegraded":"2024-09-21T05:47:54.319979+0000","last_fullsized":"2024-09-21T05:47:54.319979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:11.737474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319826+0000","last_change":"2024-09-21T05:47:52.321048+0000","last_active":"2024-09-21T05:47:54.319826+0000","last_peered":"2024-09-21T05:47:54.319826+0000","last_clean":"2024-09-21T05:47:54.319826+0000","last_became_active":"2024-09-21T05:47:52.320701+0000","last_became_peered":"2024-09-21T05:47:52.320701+0000","last_unstale":"2024-09-21T05:47:54.319826+0000","last_undegraded":"2024-09-21T05:47:54.319826+0000","last_fullsized":"2024-09-21T05:47:54.319826+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:30.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420672+0000","last_change":"2024-09-21T05:47:53.336978+0000","last_active":"2024-09-21T05:47:54.420672+0000","last_peered":"2024-09-21T05:47:54.420672+0000","last_clean":"2024-09-21T05:47:54.420672+0000","last_became_active":"2024-09-21T05:47:53.336304+0000","last_became_peered":"2024-09-21T05:47:53.336304+0000","last_unstale":"2024-09-21T05:47:54.420672+0000","last_undegraded":"2024-09-21T05:47:54.420672+0000","last_fullsized":"2024-09-21T05:47:54.420672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:23.474145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419148+0000","last_change":"2024-09-21T05:47:52.316047+0000","last_active":"2024-09-21T05:47:54.419148+0000","last_peered":"2024-09-21T05:47:54.419148+0000","last_clean":"2024-09-21T05:47:54.419148+0000","last_became_active":"2024-09-21T05:47:52.315653+0000","last_became_peered":"2024-09-21T05:47:52.315653+0000","last_unstale":"2024-09-21T05:47:54.419148+0000","last_undegraded":"2024-09-21T05:47:54.419148+0000","last_fullsized":"2024-09-21T05:47:54.419148+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:29.936206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420062+0000","last_change":"2024-09-21T05:47:52.315884+0000","last_active":"2024-09-21T05:47:54.420062+0000","last_peered":"2024-09-21T05:47:54.420062+0000","last_clean":"2024-09-21T05:47:54.420062+0000","last_became_active":"2024-09-21T05:47:52.315492+0000","last_became_peered":"2024-09-21T05:47:52.315492+0000","last_unstale":"2024-09-21T05:47:54.420062+0000","last_undegraded":"2024-09-21T05:47:54.420062+0000","last_fullsized":"2024-09-21T05:47:54.420062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:47.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420539+0000","last_change":"2024-09-21T05:47:53.339451+0000","last_active":"2024-09-21T05:47:54.420539+0000","last_peered":"2024-09-21T05:47:54.420539+0000","last_clean":"2024-09-21T05:47:54.420539+0000","last_became_active":"2024-09-21T05:47:53.339042+0000","last_became_peered":"2024-09-21T05:47:53.339042+0000","last_unstale":"2024-09-21T05:47:54.420539+0000","last_undegraded":"2024-09-21T05:47:54.420539+0000","last_fullsized":"2024-09-21T05:47:54.420539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:24.227545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T05:47:51.294598+0000","last_change":"2024-09-21T05:47:51.294598+0000","last_active":"2024-09-21T05:47:51.289529+0000","last_peered":"2024-09-21T05:47:51.289529+0000","last_clean":"2024-09-21T05:47:51.289529+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:51.294598+0000","last_undegraded":"2024-09-21T05:47:51.294598+0000","last_fullsized":"2024-09-21T05:47:51.294598+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419785+0000","last_change":"2024-09-21T05:47:53.324736+0000","last_active":"2024-09-21T05:47:54.419785+0000","last_peered":"2024-09-21T05:47:54.419785+0000","last_clean":"2024-09-21T05:47:54.419785+0000","last_became_active":"2024-09-21T05:47:53.324398+0000","last_became_peered":"2024-09-21T05:47:53.324398+0000","last_unstale":"2024-09-21T05:47:54.419785+0000","last_undegraded":"2024-09-21T05:47:54.419785+0000","last_fullsized":"2024-09-21T05:47:54.419785+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:14.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420656+0000","last_change":"2024-09-21T05:47:52.315887+0000","last_active":"2024-09-21T05:47:54.420656+0000","last_peered":"2024-09-21T05:47:54.420656+0000","last_clean":"2024-09-21T05:47:54.420656+0000","last_became_active":"2024-09-21T05:47:52.315504+0000","last_became_peered":"2024-09-21T05:47:52.315504+0000","last_unstale":"2024-09-21T05:47:54.420656+0000","last_undegraded":"2024-09-21T05:47:54.420656+0000","last_fullsized":"2024-09-21T05:47:54.420656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:57.957523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419923+0000","last_change":"2024-09-21T05:47:53.329242+0000","last_active":"2024-09-21T05:47:54.419923+0000","last_peered":"2024-09-21T05:47:54.419923+0000","last_clean":"2024-09-21T05:47:54.419923+0000","last_became_active":"2024-09-21T05:47:53.327266+0000","last_became_peered":"2024-09-21T05:47:53.327266+0000","last_unstale":"2024-09-21T05:47:54.419923+0000","last_undegraded":"2024-09-21T05:47:54.419923+0000","last_fullsized":"2024-09-21T05:47:54.419923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:53.808702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320068+0000","last_change":"2024-09-21T05:47:52.318716+0000","last_active":"2024-09-21T05:47:54.320068+0000","last_peered":"2024-09-21T05:47:54.320068+0000","last_clean":"2024-09-21T05:47:54.320068+0000","last_became_active":"2024-09-21T05:47:52.318376+0000","last_became_peered":"2024-09-21T05:47:52.318376+0000","last_unstale":"2024-09-21T05:47:54.320068+0000","last_undegraded":"2024-09-21T05:47:54.320068+0000","last_fullsized":"2024-09-21T05:47:54.320068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:36.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323810+0000","last_change":"2024-09-21T05:47:53.330894+0000","last_active":"2024-09-21T05:47:54.323810+0000","last_peered":"2024-09-21T05:47:54.323810+0000","last_clean":"2024-09-21T05:47:54.323810+0000","last_became_active":"2024-09-21T05:47:53.330525+0000","last_became_peered":"2024-09-21T05:47:53.330525+0000","last_unstale":"2024-09-21T05:47:54.323810+0000","last_undegraded":"2024-09-21T05:47:54.323810+0000","last_fullsized":"2024-09-21T05:47:54.323810+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:53.569567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419648+0000","last_change":"2024-09-21T05:47:52.315933+0000","last_active":"2024-09-21T05:47:54.419648+0000","last_peered":"2024-09-21T05:47:54.419648+0000","last_clean":"2024-09-21T05:47:54.419648+0000","last_became_active":"2024-09-21T05:47:52.315578+0000","last_became_peered":"2024-09-21T05:47:52.315578+0000","last_unstale":"2024-09-21T05:47:54.419648+0000","last_undegraded":"2024-09-21T05:47:54.419648+0000","last_fullsized":"2024-09-21T05:47:54.419648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:07.116156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420303+0000","last_change":"2024-09-21T05:47:53.320523+0000","last_active":"2024-09-21T05:47:54.420303+0000","last_peered":"2024-09-21T05:47:54.420303+0000","last_clean":"2024-09-21T05:47:54.420303+0000","last_became_active":"2024-09-21T05:47:53.320183+0000","last_became_peered":"2024-09-21T05:47:53.320183+0000","last_unstale":"2024-09-21T05:47:54.420303+0000","last_undegraded":"2024-09-21T05:47:54.420303+0000","last_fullsized":"2024-09-21T05:47:54.420303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:58.959425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421035+0000","last_change":"2024-09-21T05:47:52.322650+0000","last_active":"2024-09-21T05:47:54.421035+0000","last_peered":"2024-09-21T05:47:54.421035+0000","last_clean":"2024-09-21T05:47:54.421035+0000","last_became_active":"2024-09-21T05:47:52.322310+0000","last_became_peered":"2024-09-21T05:47:52.322310+0000","last_unstale":"2024-09-21T05:47:54.421035+0000","last_undegraded":"2024-09-21T05:47:54.421035+0000","last_fullsized":"2024-09-21T05:47:54.421035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:23.430441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419420+0000","last_change":"2024-09-21T05:47:53.326552+0000","last_active":"2024-09-21T05:47:54.419420+0000","last_peered":"2024-09-21T05:47:54.419420+0000","last_clean":"2024-09-21T05:47:54.419420+0000","last_became_active":"2024-09-21T05:47:53.326200+0000","last_became_peered":"2024-09-21T05:47:53.326200+0000","last_unstale":"2024-09-21T05:47:54.419420+0000","last_undegraded":"2024-09-21T05:47:54.419420+0000","last_fullsized":"2024-09-21T05:47:54.419420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:13.011382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":21,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421178+0000","last_change":"2024-09-21T05:47:52.324477+0000","last_active":"2024-09-21T05:47:54.421178+0000","last_peered":"2024-09-21T05:47:54.421178+0000","last_clean":"2024-09-21T05:47:54.421178+0000","last_became_active":"2024-09-21T05:47:52.324166+0000","last_became_peered":"2024-09-21T05:47:52.324166+0000","last_unstale":"2024-09-21T05:47:54.421178+0000","last_undegraded":"2024-09-21T05:47:54.421178+0000","last_fullsized":"2024-09-21T05:47:54.421178+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:00.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419994+0000","last_change":"2024-09-21T05:47:53.326873+0000","last_active":"2024-09-21T05:47:54.419994+0000","last_peered":"2024-09-21T05:47:54.419994+0000","last_clean":"2024-09-21T05:47:54.419994+0000","last_became_active":"2024-09-21T05:47:53.326548+0000","last_became_peered":"2024-09-21T05:47:53.326548+0000","last_unstale":"2024-09-21T05:47:54.419994+0000","last_undegraded":"2024-09-21T05:47:54.419994+0000","last_fullsized":"2024-09-21T05:47:54.419994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:34.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419775+0000","last_change":"2024-09-21T05:47:52.316414+0000","last_active":"2024-09-21T05:47:54.419775+0000","last_peered":"2024-09-21T05:47:54.419775+0000","last_clean":"2024-09-21T05:47:54.419775+0000","last_became_active":"2024-09-21T05:47:52.316075+0000","last_became_peered":"2024-09-21T05:47:52.316075+0000","last_unstale":"2024-09-21T05:47:54.419775+0000","last_undegraded":"2024-09-21T05:47:54.419775+0000","last_fullsized":"2024-09-21T05:47:54.419775+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:58.423946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419724+0000","last_change":"2024-09-21T05:47:53.331201+0000","last_active":"2024-09-21T05:47:54.419724+0000","last_peered":"2024-09-21T05:47:54.419724+0000","last_clean":"2024-09-21T05:47:54.419724+0000","last_became_active":"2024-09-21T05:47:53.330540+0000","last_became_peered":"2024-09-21T05:47:53.330540+0000","last_unstale":"2024-09-21T05:47:54.419724+0000","last_undegraded":"2024-09-21T05:47:54.419724+0000","last_fullsized":"2024-09-21T05:47:54.419724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:19.803831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323431+0000","last_change":"2024-09-21T05:47:52.327286+0000","last_active":"2024-09-21T05:47:54.323431+0000","last_peered":"2024-09-21T05:47:54.323431+0000","last_clean":"2024-09-21T05:47:54.323431+0000","last_became_active":"2024-09-21T05:47:52.326918+0000","last_became_peered":"2024-09-21T05:47:52.326918+0000","last_unstale":"2024-09-21T05:47:54.323431+0000","last_undegraded":"2024-09-21T05:47:54.323431+0000","last_fullsized":"2024-09-21T05:47:54.323431+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:21.234490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419301+0000","last_change":"2024-09-21T05:47:53.319222+0000","last_active":"2024-09-21T05:47:54.419301+0000","last_peered":"2024-09-21T05:47:54.419301+0000","last_clean":"2024-09-21T05:47:54.419301+0000","last_became_active":"2024-09-21T05:47:53.318912+0000","last_became_peered":"2024-09-21T05:47:53.318912+0000","last_unstale":"2024-09-21T05:47:54.419301+0000","last_undegraded":"2024-09-21T05:47:54.419301+0000","last_fullsized":"2024-09-21T05:47:54.419301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:55.408820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":23,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319663+0000","last_change":"2024-09-21T05:47:52.318238+0000","last_active":"2024-09-21T05:47:54.319663+0000","last_peered":"2024-09-21T05:47:54.319663+0000","last_clean":"2024-09-21T05:47:54.319663+0000","last_became_active":"2024-09-21T05:47:52.317230+0000","last_became_peered":"2024-09-21T05:47:52.317230+0000","last_unstale":"2024-09-21T05:47:54.319663+0000","last_undegraded":"2024-09-21T05:47:54.319663+0000","last_fullsized":"2024-09-21T05:47:54.319663+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:37.840490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320255+0000","last_change":"2024-09-21T05:47:53.321414+0000","last_active":"2024-09-21T05:47:54.320255+0000","last_peered":"2024-09-21T05:47:54.320255+0000","last_clean":"2024-09-21T05:47:54.320255+0000","last_became_active":"2024-09-21T05:47:53.321109+0000","last_became_peered":"2024-09-21T05:47:53.321109+0000","last_unstale":"2024-09-21T05:47:54.320255+0000","last_undegraded":"2024-09-21T05:47:54.320255+0000","last_fullsized":"2024-09-21T05:47:54.320255+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:49.957606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421568+0000","last_change":"2024-09-21T05:47:52.326635+0000","last_active":"2024-09-21T05:47:54.421568+0000","last_peered":"2024-09-21T05:47:54.421568+0000","last_clean":"2024-09-21T05:47:54.421568+0000","last_became_active":"2024-09-21T05:47:52.326303+0000","last_became_peered":"2024-09-21T05:47:52.326303+0000","last_unstale":"2024-09-21T05:47:54.421568+0000","last_undegraded":"2024-09-21T05:47:54.421568+0000","last_fullsized":"2024-09-21T05:47:54.421568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.319086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419800+0000","last_change":"2024-09-21T05:47:53.334483+0000","last_active":"2024-09-21T05:47:54.419800+0000","last_peered":"2024-09-21T05:47:54.419800+0000","last_clean":"2024-09-21T05:47:54.419800+0000","last_became_active":"2024-09-21T05:47:53.334127+0000","last_became_peered":"2024-09-21T05:47:53.334127+0000","last_unstale":"2024-09-21T05:47:54.419800+0000","last_undegraded":"2024-09-21T05:47:54.419800+0000","last_fullsized":"2024-09-21T05:47:54.419800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.795868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320154+0000","last_change":"2024-09-21T05:47:52.314440+0000","last_active":"2024-09-21T05:47:54.320154+0000","last_peered":"2024-09-21T05:47:54.320154+0000","last_clean":"2024-09-21T05:47:54.320154+0000","last_became_active":"2024-09-21T05:47:52.314083+0000","last_became_peered":"2024-09-21T05:47:52.314083+0000","last_unstale":"2024-09-21T05:47:54.320154+0000","last_undegraded":"2024-09-21T05:47:54.320154+0000","last_fullsized":"2024-09-21T05:47:54.320154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:05.210599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323537+0000","last_change":"2024-09-21T05:47:52.318503+0000","last_active":"2024-09-21T05:47:54.323537+0000","last_peered":"2024-09-21T05:47:54.323537+0000","last_clean":"2024-09-21T05:47:54.323537+0000","last_became_active":"2024-09-21T05:47:52.318174+0000","last_became_peered":"2024-09-21T05:47:52.318174+0000","last_unstale":"2024-09-21T05:47:54.323537+0000","last_undegraded":"2024-09-21T05:47:54.323537+0000","last_fullsized":"2024-09-21T05:47:54.323537+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:30.834390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419066+0000","last_change":"2024-09-21T05:47:52.316818+0000","last_active":"2024-09-21T05:47:54.419066+0000","last_peered":"2024-09-21T05:47:54.419066+0000","last_clean":"2024-09-21T05:47:54.419066+0000","last_became_active":"2024-09-21T05:47:52.316512+0000","last_became_peered":"2024-09-21T05:47:52.316512+0000","last_unstale":"2024-09-21T05:47:54.419066+0000","last_undegraded":"2024-09-21T05:47:54.419066+0000","last_fullsized":"2024-09-21T05:47:54.419066+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:56.759531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419891+0000","last_change":"2024-09-21T05:47:52.321742+0000","last_active":"2024-09-21T05:47:54.419891+0000","last_peered":"2024-09-21T05:47:54.419891+0000","last_clean":"2024-09-21T05:47:54.419891+0000","last_became_active":"2024-09-21T05:47:52.321019+0000","last_became_peered":"2024-09-21T05:47:52.321019+0000","last_unstale":"2024-09-21T05:47:54.419891+0000","last_undegraded":"2024-09-21T05:47:54.419891+0000","last_fullsized":"2024-09-21T05:47:54.419891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:34.946405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420780+0000","last_change":"2024-09-21T05:47:53.335546+0000","last_active":"2024-09-21T05:47:54.420780+0000","last_peered":"2024-09-21T05:47:54.420780+0000","last_clean":"2024-09-21T05:47:54.420780+0000","last_became_active":"2024-09-21T05:47:53.335215+0000","last_became_peered":"2024-09-21T05:47:53.335215+0000","last_unstale":"2024-09-21T05:47:54.420780+0000","last_undegraded":"2024-09-21T05:47:54.420780+0000","last_fullsized":"2024-09-21T05:47:54.420780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:18.844619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319672+0000","last_change":"2024-09-21T05:47:52.319751+0000","last_active":"2024-09-21T05:47:54.319672+0000","last_peered":"2024-09-21T05:47:54.319672+0000","last_clean":"2024-09-21T05:47:54.319672+0000","last_became_active":"2024-09-21T05:47:52.319375+0000","last_became_peered":"2024-09-21T05:47:52.319375+0000","last_unstale":"2024-09-21T05:47:54.319672+0000","last_undegraded":"2024-09-21T05:47:54.319672+0000","last_fullsized":"2024-09-21T05:47:54.319672+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:30.482915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420237+0000","last_change":"2024-09-21T05:47:52.321620+0000","last_active":"2024-09-21T05:47:54.420237+0000","last_peered":"2024-09-21T05:47:54.420237+0000","last_clean":"2024-09-21T05:47:54.420237+0000","last_became_active":"2024-09-21T05:47:52.321166+0000","last_became_peered":"2024-09-21T05:47:52.321166+0000","last_unstale":"2024-09-21T05:47:54.420237+0000","last_undegraded":"2024-09-21T05:47:54.420237+0000","last_fullsized":"2024-09-21T05:47:54.420237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.967667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419550+0000","last_change":"2024-09-21T05:47:53.340973+0000","last_active":"2024-09-21T05:47:54.419550+0000","last_peered":"2024-09-21T05:47:54.419550+0000","last_clean":"2024-09-21T05:47:54.419550+0000","last_became_active":"2024-09-21T05:47:53.340603+0000","last_became_peered":"2024-09-21T05:47:53.340603+0000","last_unstale":"2024-09-21T05:47:54.419550+0000","last_undegraded":"2024-09-21T05:47:54.419550+0000","last_fullsized":"2024-09-21T05:47:54.419550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.320987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420772+0000","last_change":"2024-09-21T05:47:52.326447+0000","last_active":"2024-09-21T05:47:54.420772+0000","last_peered":"2024-09-21T05:47:54.420772+0000","last_clean":"2024-09-21T05:47:54.420772+0000","last_became_active":"2024-09-21T05:47:52.326064+0000","last_became_peered":"2024-09-21T05:47:52.326064+0000","last_unstale":"2024-09-21T05:47:54.420772+0000","last_undegraded":"2024-09-21T05:47:54.420772+0000","last_fullsized":"2024-09-21T05:47:54.420772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:27.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323686+0000","last_change":"2024-09-21T05:47:53.322645+0000","last_active":"2024-09-21T05:47:54.323686+0000","last_peered":"2024-09-21T05:47:54.323686+0000","last_clean":"2024-09-21T05:47:54.323686+0000","last_became_active":"2024-09-21T05:47:53.322384+0000","last_became_peered":"2024-09-21T05:47:53.322384+0000","last_unstale":"2024-09-21T05:47:54.323686+0000","last_undegraded":"2024-09-21T05:47:54.323686+0000","last_fullsized":"2024-09-21T05:47:54.323686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:06.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419780+0000","last_change":"2024-09-21T05:47:52.316775+0000","last_active":"2024-09-21T05:47:54.419780+0000","last_peered":"2024-09-21T05:47:54.419780+0000","last_clean":"2024-09-21T05:47:54.419780+0000","last_became_active":"2024-09-21T05:47:52.315552+0000","last_became_peered":"2024-09-21T05:47:52.315552+0000","last_unstale":"2024-09-21T05:47:54.419780+0000","last_undegraded":"2024-09-21T05:47:54.419780+0000","last_fullsized":"2024-09-21T05:47:54.419780+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:56.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421499+0000","last_change":"2024-09-21T05:47:53.329320+0000","last_active":"2024-09-21T05:47:54.421499+0000","last_peered":"2024-09-21T05:47:54.421499+0000","last_clean":"2024-09-21T05:47:54.421499+0000","last_became_active":"2024-09-21T05:47:53.328785+0000","last_became_peered":"2024-09-21T05:47:53.328785+0000","last_unstale":"2024-09-21T05:47:54.421499+0000","last_undegraded":"2024-09-21T05:47:54.421499+0000","last_fullsized":"2024-09-21T05:47:54.421499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:11.061652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:55.306521+0000","last_change":"2024-09-21T05:47:52.317943+0000","last_active":"2024-09-21T05:47:55.306521+0000","last_peered":"2024-09-21T05:47:55.306521+0000","last_clean":"2024-09-21T05:47:55.306521+0000","last_became_active":"2024-09-21T05:47:52.317606+0000","last_became_peered":"2024-09-21T05:47:52.317606+0000","last_unstale":"2024-09-21T05:47:55.306521+0000","last_undegraded":"2024-09-21T05:47:55.306521+0000","last_fullsized":"2024-09-21T05:47:55.306521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:53.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421251+0000","last_change":"2024-09-21T05:47:53.337568+0000","last_active":"2024-09-21T05:47:54.421251+0000","last_peered":"2024-09-21T05:47:54.421251+0000","last_clean":"2024-09-21T05:47:54.421251+0000","last_became_active":"2024-09-21T05:47:53.337068+0000","last_became_peered":"2024-09-21T05:47:53.337068+0000","last_unstale":"2024-09-21T05:47:54.421251+0000","last_undegraded":"2024-09-21T05:47:54.421251+0000","last_fullsized":"2024-09-21T05:47:54.421251+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:19.441038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421108+0000","last_change":"2024-09-21T05:47:52.325423+0000","last_active":"2024-09-21T05:47:54.421108+0000","last_peered":"2024-09-21T05:47:54.421108+0000","last_clean":"2024-09-21T05:47:54.421108+0000","last_became_active":"2024-09-21T05:47:52.325118+0000","last_became_peered":"2024-09-21T05:47:52.325118+0000","last_unstale":"2024-09-21T05:47:54.421108+0000","last_undegraded":"2024-09-21T05:47:54.421108+0000","last_fullsized":"2024-09-21T05:47:54.421108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:54.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323673+0000","last_change":"2024-09-21T05:47:53.329564+0000","last_active":"2024-09-21T05:47:54.323673+0000","last_peered":"2024-09-21T05:47:54.323673+0000","last_clean":"2024-09-21T05:47:54.323673+0000","last_became_active":"2024-09-21T05:47:53.329189+0000","last_became_peered":"2024-09-21T05:47:53.329189+0000","last_unstale":"2024-09-21T05:47:54.323673+0000","last_undegraded":"2024-09-21T05:47:54.323673+0000","last_fullsized":"2024-09-21T05:47:54.323673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:30.665406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420349+0000","last_change":"2024-09-21T05:47:52.317088+0000","last_active":"2024-09-21T05:47:54.420349+0000","last_peered":"2024-09-21T05:47:54.420349+0000","last_clean":"2024-09-21T05:47:54.420349+0000","last_became_active":"2024-09-21T05:47:52.316526+0000","last_became_peered":"2024-09-21T05:47:52.316526+0000","last_unstale":"2024-09-21T05:47:54.420349+0000","last_undegraded":"2024-09-21T05:47:54.420349+0000","last_fullsized":"2024-09-21T05:47:54.420349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:20.821196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419272+0000","last_change":"2024-09-21T05:47:52.314266+0000","last_active":"2024-09-21T05:47:54.419272+0000","last_peered":"2024-09-21T05:47:54.419272+0000","last_clean":"2024-09-21T05:47:54.419272+0000","last_became_active":"2024-09-21T05:47:52.313950+0000","last_became_peered":"2024-09-21T05:47:52.313950+0000","last_unstale":"2024-09-21T05:47:54.419272+0000","last_undegraded":"2024-09-21T05:47:54.419272+0000","last_fullsized":"2024-09-21T05:47:54.419272+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:54.199451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T05:47:51.294534+0000","last_change":"2024-09-21T05:47:51.294534+0000","last_active":"2024-09-21T05:47:51.289529+0000","last_peered":"2024-09-21T05:47:51.289529+0000","last_clean":"2024-09-21T05:47:51.289529+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:51.294534+0000","last_undegraded":"2024-09-21T05:47:51.294534+0000","last_fullsized":"2024-09-21T05:47:51.294534+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319732+0000","last_change":"2024-09-21T05:47:52.319216+0000","last_active":"2024-09-21T05:47:54.319732+0000","last_peered":"2024-09-21T05:47:54.319732+0000","last_clean":"2024-09-21T05:47:54.319732+0000","last_became_active":"2024-09-21T05:47:52.318756+0000","last_became_peered":"2024-09-21T05:47:52.318756+0000","last_unstale":"2024-09-21T05:47:54.319732+0000","last_undegraded":"2024-09-21T05:47:54.319732+0000","last_fullsized":"2024-09-21T05:47:54.319732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:12.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420865+0000","last_change":"2024-09-21T05:47:53.337719+0000","last_active":"2024-09-21T05:47:54.420865+0000","last_peered":"2024-09-21T05:47:54.420865+0000","last_clean":"2024-09-21T05:47:54.420865+0000","last_became_active":"2024-09-21T05:47:53.329968+0000","last_became_peered":"2024-09-21T05:47:53.329968+0000","last_unstale":"2024-09-21T05:47:54.420865+0000","last_undegraded":"2024-09-21T05:47:54.420865+0000","last_fullsized":"2024-09-21T05:47:54.420865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:11.856321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420792+0000","last_change":"2024-09-21T05:47:52.324337+0000","last_active":"2024-09-21T05:47:54.420792+0000","last_peered":"2024-09-21T05:47:54.420792+0000","last_clean":"2024-09-21T05:47:54.420792+0000","last_became_active":"2024-09-21T05:47:52.324032+0000","last_became_peered":"2024-09-21T05:47:52.324032+0000","last_unstale":"2024-09-21T05:47:54.420792+0000","last_undegraded":"2024-09-21T05:47:54.420792+0000","last_fullsized":"2024-09-21T05:47:54.420792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:03.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419235+0000","last_change":"2024-09-21T05:47:53.319715+0000","last_active":"2024-09-21T05:47:54.419235+0000","last_peered":"2024-09-21T05:47:54.419235+0000","last_clean":"2024-09-21T05:47:54.419235+0000","last_became_active":"2024-09-21T05:47:53.319394+0000","last_became_peered":"2024-09-21T05:47:53.319394+0000","last_unstale":"2024-09-21T05:47:54.419235+0000","last_undegraded":"2024-09-21T05:47:54.419235+0000","last_fullsized":"2024-09-21T05:47:54.419235+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:24.432342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419478+0000","last_change":"2024-09-21T05:47:52.316016+0000","last_active":"2024-09-21T05:47:54.419478+0000","last_peered":"2024-09-21T05:47:54.419478+0000","last_clean":"2024-09-21T05:47:54.419478+0000","last_became_active":"2024-09-21T05:47:52.315661+0000","last_became_peered":"2024-09-21T05:47:52.315661+0000","last_unstale":"2024-09-21T05:47:54.419478+0000","last_undegraded":"2024-09-21T05:47:54.419478+0000","last_fullsized":"2024-09-21T05:47:54.419478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:23.229664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":265,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T05:47:51.300954+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:47:51.300954+0000","last_peered":"2024-09-21T05:47:51.300954+0000","last_clean":"2024-09-21T05:47:51.300954+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:47:51.300954+0000","last_undegraded":"2024-09-21T05:47:51.300954+0000","last_fullsized":"2024-09-21T05:47:51.300954+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1904640,"data_stored":1154973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1298432,"data_stored":558788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1351680,"data_stored":584025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1355776,"data_stored":584559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1363968,"data_stored":585637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40840,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331000,"statfs":{"total":96636764160,"available":96594944000,"internally_reserved":0,"allocated":1355776,"data_stored":584079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542252,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":1176047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:47:57.395 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:47:57.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 -- 172.21.15.22:0/22642259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a8005350 msgr2=0x7fe1a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/22642259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a8005350 0x7fe1a800b9c0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0b9b10 tx=0x7fe1a8005830 comp rx=0 tx=0).stop 2024-09-21T05:47:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 -- 172.21.15.22:0/22642259 shutdown_connections 2024-09-21T05:47:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/22642259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1a8005350 0x7fe1a800b9c0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 -- 172.21.15.22:0/22642259 >> 172.21.15.22:0/22642259 conn(0x7fe1bc09d8f0 msgr2=0x7fe1bc0d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 -- 172.21.15.22:0/22642259 shutdown_connections 2024-09-21T05:47:57.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.547+0000 7fe1c18c5640 1 -- 172.21.15.22:0/22642259 wait complete. 2024-09-21T05:47:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.549+0000 7fe1c18c5640 1 Processor -- start 2024-09-21T05:47:57.551 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.549+0000 7fe1c18c5640 1 -- start start 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.550+0000 7fe1c18c5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a8005350 0x7fe1bc0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.550+0000 7fe1c18c5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1bc15a510 0x7fe1bc1f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.550+0000 7fe1c18c5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 0x7fe1bc1613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.550+0000 7fe1c18c5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc0d1340 con 0x7fe1bc160fa0 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1c18c5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fe1bc0d11c0 con 0x7fe1a8005350 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1c18c5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fe1bc0d14c0 con 0x7fe1bc15a510 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 0x7fe1bc1613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 0x7fe1bc1613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50550/0 (socket says 172.21.15.22:50550) 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 -- 172.21.15.22:0/1875368394 learned_addr learned my addr 172.21.15.22:0/1875368394 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:57.554 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1baffd640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a8005350 0x7fe1bc0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1b27fc640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1bc15a510 0x7fe1bc1f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 -- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1bc15a510 msgr2=0x7fe1bc1f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1bc15a510 0x7fe1bc1f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 -- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a8005350 msgr2=0x7fe1bc0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a8005350 0x7fe1bc0c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 -- 172.21.15.22:0/1875368394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc15f0a0 con 0x7fe1bc160fa0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1bb7fe640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 0x7fe1bc1613b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400c6d0 tx=0x7fe1b400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4039070 con 0x7fe1bc160fa0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b402bc50 con 0x7fe1bc160fa0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1bc1f6660 con 0x7fe1bc160fa0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4006290 con 0x7fe1bc160fa0 2024-09-21T05:47:57.555 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.551+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1bc15edc0 con 0x7fe1bc160fa0 2024-09-21T05:47:57.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.552+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1bc162430 con 0x7fe1bc160fa0 2024-09-21T05:47:57.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.553+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1b4006430 con 0x7fe1bc160fa0 2024-09-21T05:47:57.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.554+0000 7fe1b8ff9640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe19c078bc0 0x7fe19c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.554+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b40baaf0 con 0x7fe1bc160fa0 2024-09-21T05:47:57.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.554+0000 7fe1baffd640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe19c078bc0 0x7fe19c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.556 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.555+0000 7fe1baffd640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe19c078bc0 0x7fe19c07b0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0066b0 tx=0x7fe1ac006b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:57.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.558+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1bc1f6660 con 0x7fe1bc160fa0 2024-09-21T05:47:57.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.743+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1bc0ba0d0 con 0x7fe19c078bc0 2024-09-21T05:47:57.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.747+0000 7fe1b8ff9640 1 -- 172.21.15.22:0/1875368394 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249123 (secure 0 0 0) 0x7fe1bc0ba0d0 con 0x7fe19c078bc0 2024-09-21T05:47:57.752 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:57.753 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:47:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe19c078bc0 msgr2=0x7fe19c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe19c078bc0 0x7fe19c07b0d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac0066b0 tx=0x7fe1ac006b70 comp rx=0 tx=0).stop 2024-09-21T05:47:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 msgr2=0x7fe1bc1613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 0x7fe1bc1613b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe1b400c6d0 tx=0x7fe1b400c2d0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 shutdown_connections 2024-09-21T05:47:57.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fe1bc160fa0 0x7fe1bc1613b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fe19c078bc0 0x7fe19c07b0d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fe1bc15a510 0x7fe1bc1f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 --2- 172.21.15.22:0/1875368394 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fe1a8005350 0x7fe1bc0c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 >> 172.21.15.22:0/1875368394 conn(0x7fe1bc09d8f0 msgr2=0x7fe1bc0d2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 shutdown_connections 2024-09-21T05:47:57.755 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.751+0000 7fe1c18c5640 1 -- 172.21.15.22:0/1875368394 wait complete. 2024-09-21T05:47:57.764 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-21T05:47:55.877407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":150,"acting":150,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282012,"kb_used_data":11660,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754692708,"statfs":{"total":773094113280,"available":772805332992,"internally_reserved":0,"allocated":11939840,"data_stored":5812063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.584816"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421407+0000","last_change":"2024-09-21T05:47:53.329400+0000","last_active":"2024-09-21T05:47:54.421407+0000","last_peered":"2024-09-21T05:47:54.421407+0000","last_clean":"2024-09-21T05:47:54.421407+0000","last_became_active":"2024-09-21T05:47:53.328865+0000","last_became_peered":"2024-09-21T05:47:53.328865+0000","last_unstale":"2024-09-21T05:47:54.421407+0000","last_undegraded":"2024-09-21T05:47:54.421407+0000","last_fullsized":"2024-09-21T05:47:54.421407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:33.666746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420973+0000","last_change":"2024-09-21T05:47:53.338050+0000","last_active":"2024-09-21T05:47:54.420973+0000","last_peered":"2024-09-21T05:47:54.420973+0000","last_clean":"2024-09-21T05:47:54.420973+0000","last_became_active":"2024-09-21T05:47:53.337279+0000","last_became_peered":"2024-09-21T05:47:53.337279+0000","last_unstale":"2024-09-21T05:47:54.420973+0000","last_undegraded":"2024-09-21T05:47:54.420973+0000","last_fullsized":"2024-09-21T05:47:54.420973+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:49.343256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320073+0000","last_change":"2024-09-21T05:47:53.330191+0000","last_active":"2024-09-21T05:47:54.320073+0000","last_peered":"2024-09-21T05:47:54.320073+0000","last_clean":"2024-09-21T05:47:54.320073+0000","last_became_active":"2024-09-21T05:47:53.329578+0000","last_became_peered":"2024-09-21T05:47:53.329578+0000","last_unstale":"2024-09-21T05:47:54.320073+0000","last_undegraded":"2024-09-21T05:47:54.320073+0000","last_fullsized":"2024-09-21T05:47:54.320073+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:51.237035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420158+0000","last_change":"2024-09-21T05:47:53.314749+0000","last_active":"2024-09-21T05:47:54.420158+0000","last_peered":"2024-09-21T05:47:54.420158+0000","last_clean":"2024-09-21T05:47:54.420158+0000","last_became_active":"2024-09-21T05:47:53.314457+0000","last_became_peered":"2024-09-21T05:47:53.314457+0000","last_unstale":"2024-09-21T05:47:54.420158+0000","last_undegraded":"2024-09-21T05:47:54.420158+0000","last_fullsized":"2024-09-21T05:47:54.420158+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:39.746882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419166+0000","last_change":"2024-09-21T05:47:53.322397+0000","last_active":"2024-09-21T05:47:54.419166+0000","last_peered":"2024-09-21T05:47:54.419166+0000","last_clean":"2024-09-21T05:47:54.419166+0000","last_became_active":"2024-09-21T05:47:53.322048+0000","last_became_peered":"2024-09-21T05:47:53.322048+0000","last_unstale":"2024-09-21T05:47:54.419166+0000","last_undegraded":"2024-09-21T05:47:54.419166+0000","last_fullsized":"2024-09-21T05:47:54.419166+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:37.914628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421377+0000","last_change":"2024-09-21T05:47:53.333144+0000","last_active":"2024-09-21T05:47:54.421377+0000","last_peered":"2024-09-21T05:47:54.421377+0000","last_clean":"2024-09-21T05:47:54.421377+0000","last_became_active":"2024-09-21T05:47:53.332816+0000","last_became_peered":"2024-09-21T05:47:53.332816+0000","last_unstale":"2024-09-21T05:47:54.421377+0000","last_undegraded":"2024-09-21T05:47:54.421377+0000","last_fullsized":"2024-09-21T05:47:54.421377+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:44.276704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421287+0000","last_change":"2024-09-21T05:47:53.328426+0000","last_active":"2024-09-21T05:47:54.421287+0000","last_peered":"2024-09-21T05:47:54.421287+0000","last_clean":"2024-09-21T05:47:54.421287+0000","last_became_active":"2024-09-21T05:47:53.327992+0000","last_became_peered":"2024-09-21T05:47:53.327992+0000","last_unstale":"2024-09-21T05:47:54.421287+0000","last_undegraded":"2024-09-21T05:47:54.421287+0000","last_fullsized":"2024-09-21T05:47:54.421287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:02.621431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421042+0000","last_change":"2024-09-21T05:47:53.338328+0000","last_active":"2024-09-21T05:47:54.421042+0000","last_peered":"2024-09-21T05:47:54.421042+0000","last_clean":"2024-09-21T05:47:54.421042+0000","last_became_active":"2024-09-21T05:47:53.337637+0000","last_became_peered":"2024-09-21T05:47:53.337637+0000","last_unstale":"2024-09-21T05:47:54.421042+0000","last_undegraded":"2024-09-21T05:47:54.421042+0000","last_fullsized":"2024-09-21T05:47:54.421042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:07.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419472+0000","last_change":"2024-09-21T05:47:53.314013+0000","last_active":"2024-09-21T05:47:54.419472+0000","last_peered":"2024-09-21T05:47:54.419472+0000","last_clean":"2024-09-21T05:47:54.419472+0000","last_became_active":"2024-09-21T05:47:53.313613+0000","last_became_peered":"2024-09-21T05:47:53.313613+0000","last_unstale":"2024-09-21T05:47:54.419472+0000","last_undegraded":"2024-09-21T05:47:54.419472+0000","last_fullsized":"2024-09-21T05:47:54.419472+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:09.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420533+0000","last_change":"2024-09-21T05:47:53.340669+0000","last_active":"2024-09-21T05:47:54.420533+0000","last_peered":"2024-09-21T05:47:54.420533+0000","last_clean":"2024-09-21T05:47:54.420533+0000","last_became_active":"2024-09-21T05:47:53.340336+0000","last_became_peered":"2024-09-21T05:47:53.340336+0000","last_unstale":"2024-09-21T05:47:54.420533+0000","last_undegraded":"2024-09-21T05:47:54.420533+0000","last_fullsized":"2024-09-21T05:47:54.420533+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:28.758224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323869+0000","last_change":"2024-09-21T05:47:53.317110+0000","last_active":"2024-09-21T05:47:54.323869+0000","last_peered":"2024-09-21T05:47:54.323869+0000","last_clean":"2024-09-21T05:47:54.323869+0000","last_became_active":"2024-09-21T05:47:53.316725+0000","last_became_peered":"2024-09-21T05:47:53.316725+0000","last_unstale":"2024-09-21T05:47:54.323869+0000","last_undegraded":"2024-09-21T05:47:54.323869+0000","last_fullsized":"2024-09-21T05:47:54.323869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.043120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419862+0000","last_change":"2024-09-21T05:47:53.322155+0000","last_active":"2024-09-21T05:47:54.419862+0000","last_peered":"2024-09-21T05:47:54.419862+0000","last_clean":"2024-09-21T05:47:54.419862+0000","last_became_active":"2024-09-21T05:47:53.321833+0000","last_became_peered":"2024-09-21T05:47:53.321833+0000","last_unstale":"2024-09-21T05:47:54.419862+0000","last_undegraded":"2024-09-21T05:47:54.419862+0000","last_fullsized":"2024-09-21T05:47:54.419862+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:01.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419203+0000","last_change":"2024-09-21T05:47:53.320400+0000","last_active":"2024-09-21T05:47:54.419203+0000","last_peered":"2024-09-21T05:47:54.419203+0000","last_clean":"2024-09-21T05:47:54.419203+0000","last_became_active":"2024-09-21T05:47:53.320056+0000","last_became_peered":"2024-09-21T05:47:53.320056+0000","last_unstale":"2024-09-21T05:47:54.419203+0000","last_undegraded":"2024-09-21T05:47:54.419203+0000","last_fullsized":"2024-09-21T05:47:54.419203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:51.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421261+0000","last_change":"2024-09-21T05:47:53.335796+0000","last_active":"2024-09-21T05:47:54.421261+0000","last_peered":"2024-09-21T05:47:54.421261+0000","last_clean":"2024-09-21T05:47:54.421261+0000","last_became_active":"2024-09-21T05:47:53.335426+0000","last_became_peered":"2024-09-21T05:47:53.335426+0000","last_unstale":"2024-09-21T05:47:54.421261+0000","last_undegraded":"2024-09-21T05:47:54.421261+0000","last_fullsized":"2024-09-21T05:47:54.421261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:32.715401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419441+0000","last_change":"2024-09-21T05:47:53.328721+0000","last_active":"2024-09-21T05:47:54.419441+0000","last_peered":"2024-09-21T05:47:54.419441+0000","last_clean":"2024-09-21T05:47:54.419441+0000","last_became_active":"2024-09-21T05:47:53.328431+0000","last_became_peered":"2024-09-21T05:47:53.328431+0000","last_unstale":"2024-09-21T05:47:54.419441+0000","last_undegraded":"2024-09-21T05:47:54.419441+0000","last_fullsized":"2024-09-21T05:47:54.419441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:16.400560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420065+0000","last_change":"2024-09-21T05:47:53.339459+0000","last_active":"2024-09-21T05:47:54.420065+0000","last_peered":"2024-09-21T05:47:54.420065+0000","last_clean":"2024-09-21T05:47:54.420065+0000","last_became_active":"2024-09-21T05:47:53.339054+0000","last_became_peered":"2024-09-21T05:47:53.339054+0000","last_unstale":"2024-09-21T05:47:54.420065+0000","last_undegraded":"2024-09-21T05:47:54.420065+0000","last_fullsized":"2024-09-21T05:47:54.420065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:16.576540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419434+0000","last_change":"2024-09-21T05:47:53.323957+0000","last_active":"2024-09-21T05:47:54.419434+0000","last_peered":"2024-09-21T05:47:54.419434+0000","last_clean":"2024-09-21T05:47:54.419434+0000","last_became_active":"2024-09-21T05:47:53.323432+0000","last_became_peered":"2024-09-21T05:47:53.323432+0000","last_unstale":"2024-09-21T05:47:54.419434+0000","last_undegraded":"2024-09-21T05:47:54.419434+0000","last_fullsized":"2024-09-21T05:47:54.419434+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:57.449767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420771+0000","last_change":"2024-09-21T05:47:53.340040+0000","last_active":"2024-09-21T05:47:54.420771+0000","last_peered":"2024-09-21T05:47:54.420771+0000","last_clean":"2024-09-21T05:47:54.420771+0000","last_became_active":"2024-09-21T05:47:53.339702+0000","last_became_peered":"2024-09-21T05:47:53.339702+0000","last_unstale":"2024-09-21T05:47:54.420771+0000","last_undegraded":"2024-09-21T05:47:54.420771+0000","last_fullsized":"2024-09-21T05:47:54.420771+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:02.133659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.320519+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.320182+0000","last_became_peered":"2024-09-21T05:47:53.320182+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:16.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419700+0000","last_change":"2024-09-21T05:47:53.327229+0000","last_active":"2024-09-21T05:47:54.419700+0000","last_peered":"2024-09-21T05:47:54.419700+0000","last_clean":"2024-09-21T05:47:54.419700+0000","last_became_active":"2024-09-21T05:47:53.326881+0000","last_became_peered":"2024-09-21T05:47:53.326881+0000","last_unstale":"2024-09-21T05:47:54.419700+0000","last_undegraded":"2024-09-21T05:47:54.419700+0000","last_fullsized":"2024-09-21T05:47:54.419700+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:43.659296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":19,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419430+0000","last_change":"2024-09-21T05:47:52.315229+0000","last_active":"2024-09-21T05:47:54.419430+0000","last_peered":"2024-09-21T05:47:54.419430+0000","last_clean":"2024-09-21T05:47:54.419430+0000","last_became_active":"2024-09-21T05:47:52.314905+0000","last_became_peered":"2024-09-21T05:47:52.314905+0000","last_unstale":"2024-09-21T05:47:54.419430+0000","last_undegraded":"2024-09-21T05:47:54.419430+0000","last_fullsized":"2024-09-21T05:47:54.419430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:00.549432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323756+0000","last_change":"2024-09-21T05:47:53.325534+0000","last_active":"2024-09-21T05:47:54.323756+0000","last_peered":"2024-09-21T05:47:54.323756+0000","last_clean":"2024-09-21T05:47:54.323756+0000","last_became_active":"2024-09-21T05:47:53.325200+0000","last_became_peered":"2024-09-21T05:47:53.325200+0000","last_unstale":"2024-09-21T05:47:54.323756+0000","last_undegraded":"2024-09-21T05:47:54.323756+0000","last_fullsized":"2024-09-21T05:47:54.323756+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:49.932618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T05:47:51.300628+0000","last_change":"2024-09-21T05:47:51.300628+0000","last_active":"2024-09-21T05:47:51.289529+0000","last_peered":"2024-09-21T05:47:51.289529+0000","last_clean":"2024-09-21T05:47:51.289529+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:51.300628+0000","last_undegraded":"2024-09-21T05:47:51.300628+0000","last_fullsized":"2024-09-21T05:47:51.300628+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420098+0000","last_change":"2024-09-21T05:47:53.314734+0000","last_active":"2024-09-21T05:47:54.420098+0000","last_peered":"2024-09-21T05:47:54.420098+0000","last_clean":"2024-09-21T05:47:54.420098+0000","last_became_active":"2024-09-21T05:47:53.314373+0000","last_became_peered":"2024-09-21T05:47:53.314373+0000","last_unstale":"2024-09-21T05:47:54.420098+0000","last_undegraded":"2024-09-21T05:47:54.420098+0000","last_fullsized":"2024-09-21T05:47:54.420098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:32.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319979+0000","last_change":"2024-09-21T05:47:52.320521+0000","last_active":"2024-09-21T05:47:54.319979+0000","last_peered":"2024-09-21T05:47:54.319979+0000","last_clean":"2024-09-21T05:47:54.319979+0000","last_became_active":"2024-09-21T05:47:52.320077+0000","last_became_peered":"2024-09-21T05:47:52.320077+0000","last_unstale":"2024-09-21T05:47:54.319979+0000","last_undegraded":"2024-09-21T05:47:54.319979+0000","last_fullsized":"2024-09-21T05:47:54.319979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:11.737474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319826+0000","last_change":"2024-09-21T05:47:52.321048+0000","last_active":"2024-09-21T05:47:54.319826+0000","last_peered":"2024-09-21T05:47:54.319826+0000","last_clean":"2024-09-21T05:47:54.319826+0000","last_became_active":"2024-09-21T05:47:52.320701+0000","last_became_peered":"2024-09-21T05:47:52.320701+0000","last_unstale":"2024-09-21T05:47:54.319826+0000","last_undegraded":"2024-09-21T05:47:54.319826+0000","last_fullsized":"2024-09-21T05:47:54.319826+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:30.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420672+0000","last_change":"2024-09-21T05:47:53.336978+0000","last_active":"2024-09-21T05:47:54.420672+0000","last_peered":"2024-09-21T05:47:54.420672+0000","last_clean":"2024-09-21T05:47:54.420672+0000","last_became_active":"2024-09-21T05:47:53.336304+0000","last_became_peered":"2024-09-21T05:47:53.336304+0000","last_unstale":"2024-09-21T05:47:54.420672+0000","last_undegraded":"2024-09-21T05:47:54.420672+0000","last_fullsized":"2024-09-21T05:47:54.420672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:23.474145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419148+0000","last_change":"2024-09-21T05:47:52.316047+0000","last_active":"2024-09-21T05:47:54.419148+0000","last_peered":"2024-09-21T05:47:54.419148+0000","last_clean":"2024-09-21T05:47:54.419148+0000","last_became_active":"2024-09-21T05:47:52.315653+0000","last_became_peered":"2024-09-21T05:47:52.315653+0000","last_unstale":"2024-09-21T05:47:54.419148+0000","last_undegraded":"2024-09-21T05:47:54.419148+0000","last_fullsized":"2024-09-21T05:47:54.419148+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:29.936206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420062+0000","last_change":"2024-09-21T05:47:52.315884+0000","last_active":"2024-09-21T05:47:54.420062+0000","last_peered":"2024-09-21T05:47:54.420062+0000","last_clean":"2024-09-21T05:47:54.420062+0000","last_became_active":"2024-09-21T05:47:52.315492+0000","last_became_peered":"2024-09-21T05:47:52.315492+0000","last_unstale":"2024-09-21T05:47:54.420062+0000","last_undegraded":"2024-09-21T05:47:54.420062+0000","last_fullsized":"2024-09-21T05:47:54.420062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:47.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420539+0000","last_change":"2024-09-21T05:47:53.339451+0000","last_active":"2024-09-21T05:47:54.420539+0000","last_peered":"2024-09-21T05:47:54.420539+0000","last_clean":"2024-09-21T05:47:54.420539+0000","last_became_active":"2024-09-21T05:47:53.339042+0000","last_became_peered":"2024-09-21T05:47:53.339042+0000","last_unstale":"2024-09-21T05:47:54.420539+0000","last_undegraded":"2024-09-21T05:47:54.420539+0000","last_fullsized":"2024-09-21T05:47:54.420539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:24.227545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T05:47:51.294598+0000","last_change":"2024-09-21T05:47:51.294598+0000","last_active":"2024-09-21T05:47:51.289529+0000","last_peered":"2024-09-21T05:47:51.289529+0000","last_clean":"2024-09-21T05:47:51.289529+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:51.294598+0000","last_undegraded":"2024-09-21T05:47:51.294598+0000","last_fullsized":"2024-09-21T05:47:51.294598+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419785+0000","last_change":"2024-09-21T05:47:53.324736+0000","last_active":"2024-09-21T05:47:54.419785+0000","last_peered":"2024-09-21T05:47:54.419785+0000","last_clean":"2024-09-21T05:47:54.419785+0000","last_became_active":"2024-09-21T05:47:53.324398+0000","last_became_peered":"2024-09-21T05:47:53.324398+0000","last_unstale":"2024-09-21T05:47:54.419785+0000","last_undegraded":"2024-09-21T05:47:54.419785+0000","last_fullsized":"2024-09-21T05:47:54.419785+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:14.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420656+0000","last_change":"2024-09-21T05:47:52.315887+0000","last_active":"2024-09-21T05:47:54.420656+0000","last_peered":"2024-09-21T05:47:54.420656+0000","last_clean":"2024-09-21T05:47:54.420656+0000","last_became_active":"2024-09-21T05:47:52.315504+0000","last_became_peered":"2024-09-21T05:47:52.315504+0000","last_unstale":"2024-09-21T05:47:54.420656+0000","last_undegraded":"2024-09-21T05:47:54.420656+0000","last_fullsized":"2024-09-21T05:47:54.420656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:57.957523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419923+0000","last_change":"2024-09-21T05:47:53.329242+0000","last_active":"2024-09-21T05:47:54.419923+0000","last_peered":"2024-09-21T05:47:54.419923+0000","last_clean":"2024-09-21T05:47:54.419923+0000","last_became_active":"2024-09-21T05:47:53.327266+0000","last_became_peered":"2024-09-21T05:47:53.327266+0000","last_unstale":"2024-09-21T05:47:54.419923+0000","last_undegraded":"2024-09-21T05:47:54.419923+0000","last_fullsized":"2024-09-21T05:47:54.419923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:53.808702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320068+0000","last_change":"2024-09-21T05:47:52.318716+0000","last_active":"2024-09-21T05:47:54.320068+0000","last_peered":"2024-09-21T05:47:54.320068+0000","last_clean":"2024-09-21T05:47:54.320068+0000","last_became_active":"2024-09-21T05:47:52.318376+0000","last_became_peered":"2024-09-21T05:47:52.318376+0000","last_unstale":"2024-09-21T05:47:54.320068+0000","last_undegraded":"2024-09-21T05:47:54.320068+0000","last_fullsized":"2024-09-21T05:47:54.320068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:36.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323810+0000","last_change":"2024-09-21T05:47:53.330894+0000","last_active":"2024-09-21T05:47:54.323810+0000","last_peered":"2024-09-21T05:47:54.323810+0000","last_clean":"2024-09-21T05:47:54.323810+0000","last_became_active":"2024-09-21T05:47:53.330525+0000","last_became_peered":"2024-09-21T05:47:53.330525+0000","last_unstale":"2024-09-21T05:47:54.323810+0000","last_undegraded":"2024-09-21T05:47:54.323810+0000","last_fullsized":"2024-09-21T05:47:54.323810+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:53.569567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419648+0000","last_change":"2024-09-21T05:47:52.315933+0000","last_active":"2024-09-21T05:47:54.419648+0000","last_peered":"2024-09-21T05:47:54.419648+0000","last_clean":"2024-09-21T05:47:54.419648+0000","last_became_active":"2024-09-21T05:47:52.315578+0000","last_became_peered":"2024-09-21T05:47:52.315578+0000","last_unstale":"2024-09-21T05:47:54.419648+0000","last_undegraded":"2024-09-21T05:47:54.419648+0000","last_fullsized":"2024-09-21T05:47:54.419648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:07.116156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420303+0000","last_change":"2024-09-21T05:47:53.320523+0000","last_active":"2024-09-21T05:47:54.420303+0000","last_peered":"2024-09-21T05:47:54.420303+0000","last_clean":"2024-09-21T05:47:54.420303+0000","last_became_active":"2024-09-21T05:47:53.320183+0000","last_became_peered":"2024-09-21T05:47:53.320183+0000","last_unstale":"2024-09-21T05:47:54.420303+0000","last_undegraded":"2024-09-21T05:47:54.420303+0000","last_fullsized":"2024-09-21T05:47:54.420303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:58.959425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421035+0000","last_change":"2024-09-21T05:47:52.322650+0000","last_active":"2024-09-21T05:47:54.421035+0000","last_peered":"2024-09-21T05:47:54.421035+0000","last_clean":"2024-09-21T05:47:54.421035+0000","last_became_active":"2024-09-21T05:47:52.322310+0000","last_became_peered":"2024-09-21T05:47:52.322310+0000","last_unstale":"2024-09-21T05:47:54.421035+0000","last_undegraded":"2024-09-21T05:47:54.421035+0000","last_fullsized":"2024-09-21T05:47:54.421035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:23.430441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419420+0000","last_change":"2024-09-21T05:47:53.326552+0000","last_active":"2024-09-21T05:47:54.419420+0000","last_peered":"2024-09-21T05:47:54.419420+0000","last_clean":"2024-09-21T05:47:54.419420+0000","last_became_active":"2024-09-21T05:47:53.326200+0000","last_became_peered":"2024-09-21T05:47:53.326200+0000","last_unstale":"2024-09-21T05:47:54.419420+0000","last_undegraded":"2024-09-21T05:47:54.419420+0000","last_fullsized":"2024-09-21T05:47:54.419420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:13.011382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":21,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421178+0000","last_change":"2024-09-21T05:47:52.324477+0000","last_active":"2024-09-21T05:47:54.421178+0000","last_peered":"2024-09-21T05:47:54.421178+0000","last_clean":"2024-09-21T05:47:54.421178+0000","last_became_active":"2024-09-21T05:47:52.324166+0000","last_became_peered":"2024-09-21T05:47:52.324166+0000","last_unstale":"2024-09-21T05:47:54.421178+0000","last_undegraded":"2024-09-21T05:47:54.421178+0000","last_fullsized":"2024-09-21T05:47:54.421178+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:00.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419994+0000","last_change":"2024-09-21T05:47:53.326873+0000","last_active":"2024-09-21T05:47:54.419994+0000","last_peered":"2024-09-21T05:47:54.419994+0000","last_clean":"2024-09-21T05:47:54.419994+0000","last_became_active":"2024-09-21T05:47:53.326548+0000","last_became_peered":"2024-09-21T05:47:53.326548+0000","last_unstale":"2024-09-21T05:47:54.419994+0000","last_undegraded":"2024-09-21T05:47:54.419994+0000","last_fullsized":"2024-09-21T05:47:54.419994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:34.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419775+0000","last_change":"2024-09-21T05:47:52.316414+0000","last_active":"2024-09-21T05:47:54.419775+0000","last_peered":"2024-09-21T05:47:54.419775+0000","last_clean":"2024-09-21T05:47:54.419775+0000","last_became_active":"2024-09-21T05:47:52.316075+0000","last_became_peered":"2024-09-21T05:47:52.316075+0000","last_unstale":"2024-09-21T05:47:54.419775+0000","last_undegraded":"2024-09-21T05:47:54.419775+0000","last_fullsized":"2024-09-21T05:47:54.419775+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:58.423946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419724+0000","last_change":"2024-09-21T05:47:53.331201+0000","last_active":"2024-09-21T05:47:54.419724+0000","last_peered":"2024-09-21T05:47:54.419724+0000","last_clean":"2024-09-21T05:47:54.419724+0000","last_became_active":"2024-09-21T05:47:53.330540+0000","last_became_peered":"2024-09-21T05:47:53.330540+0000","last_unstale":"2024-09-21T05:47:54.419724+0000","last_undegraded":"2024-09-21T05:47:54.419724+0000","last_fullsized":"2024-09-21T05:47:54.419724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:19.803831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323431+0000","last_change":"2024-09-21T05:47:52.327286+0000","last_active":"2024-09-21T05:47:54.323431+0000","last_peered":"2024-09-21T05:47:54.323431+0000","last_clean":"2024-09-21T05:47:54.323431+0000","last_became_active":"2024-09-21T05:47:52.326918+0000","last_became_peered":"2024-09-21T05:47:52.326918+0000","last_unstale":"2024-09-21T05:47:54.323431+0000","last_undegraded":"2024-09-21T05:47:54.323431+0000","last_fullsized":"2024-09-21T05:47:54.323431+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:21.234490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419301+0000","last_change":"2024-09-21T05:47:53.319222+0000","last_active":"2024-09-21T05:47:54.419301+0000","last_peered":"2024-09-21T05:47:54.419301+0000","last_clean":"2024-09-21T05:47:54.419301+0000","last_became_active":"2024-09-21T05:47:53.318912+0000","last_became_peered":"2024-09-21T05:47:53.318912+0000","last_unstale":"2024-09-21T05:47:54.419301+0000","last_undegraded":"2024-09-21T05:47:54.419301+0000","last_fullsized":"2024-09-21T05:47:54.419301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:55.408820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":23,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319663+0000","last_change":"2024-09-21T05:47:52.318238+0000","last_active":"2024-09-21T05:47:54.319663+0000","last_peered":"2024-09-21T05:47:54.319663+0000","last_clean":"2024-09-21T05:47:54.319663+0000","last_became_active":"2024-09-21T05:47:52.317230+0000","last_became_peered":"2024-09-21T05:47:52.317230+0000","last_unstale":"2024-09-21T05:47:54.319663+0000","last_undegraded":"2024-09-21T05:47:54.319663+0000","last_fullsized":"2024-09-21T05:47:54.319663+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:37.840490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320255+0000","last_change":"2024-09-21T05:47:53.321414+0000","last_active":"2024-09-21T05:47:54.320255+0000","last_peered":"2024-09-21T05:47:54.320255+0000","last_clean":"2024-09-21T05:47:54.320255+0000","last_became_active":"2024-09-21T05:47:53.321109+0000","last_became_peered":"2024-09-21T05:47:53.321109+0000","last_unstale":"2024-09-21T05:47:54.320255+0000","last_undegraded":"2024-09-21T05:47:54.320255+0000","last_fullsized":"2024-09-21T05:47:54.320255+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:49.957606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421568+0000","last_change":"2024-09-21T05:47:52.326635+0000","last_active":"2024-09-21T05:47:54.421568+0000","last_peered":"2024-09-21T05:47:54.421568+0000","last_clean":"2024-09-21T05:47:54.421568+0000","last_became_active":"2024-09-21T05:47:52.326303+0000","last_became_peered":"2024-09-21T05:47:52.326303+0000","last_unstale":"2024-09-21T05:47:54.421568+0000","last_undegraded":"2024-09-21T05:47:54.421568+0000","last_fullsized":"2024-09-21T05:47:54.421568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.319086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419800+0000","last_change":"2024-09-21T05:47:53.334483+0000","last_active":"2024-09-21T05:47:54.419800+0000","last_peered":"2024-09-21T05:47:54.419800+0000","last_clean":"2024-09-21T05:47:54.419800+0000","last_became_active":"2024-09-21T05:47:53.334127+0000","last_became_peered":"2024-09-21T05:47:53.334127+0000","last_unstale":"2024-09-21T05:47:54.419800+0000","last_undegraded":"2024-09-21T05:47:54.419800+0000","last_fullsized":"2024-09-21T05:47:54.419800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.795868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320154+0000","last_change":"2024-09-21T05:47:52.314440+0000","last_active":"2024-09-21T05:47:54.320154+0000","last_peered":"2024-09-21T05:47:54.320154+0000","last_clean":"2024-09-21T05:47:54.320154+0000","last_became_active":"2024-09-21T05:47:52.314083+0000","last_became_peered":"2024-09-21T05:47:52.314083+0000","last_unstale":"2024-09-21T05:47:54.320154+0000","last_undegraded":"2024-09-21T05:47:54.320154+0000","last_fullsized":"2024-09-21T05:47:54.320154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:05.210599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323537+0000","last_change":"2024-09-21T05:47:52.318503+0000","last_active":"2024-09-21T05:47:54.323537+0000","last_peered":"2024-09-21T05:47:54.323537+0000","last_clean":"2024-09-21T05:47:54.323537+0000","last_became_active":"2024-09-21T05:47:52.318174+0000","last_became_peered":"2024-09-21T05:47:52.318174+0000","last_unstale":"2024-09-21T05:47:54.323537+0000","last_undegraded":"2024-09-21T05:47:54.323537+0000","last_fullsized":"2024-09-21T05:47:54.323537+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:30.834390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419066+0000","last_change":"2024-09-21T05:47:52.316818+0000","last_active":"2024-09-21T05:47:54.419066+0000","last_peered":"2024-09-21T05:47:54.419066+0000","last_clean":"2024-09-21T05:47:54.419066+0000","last_became_active":"2024-09-21T05:47:52.316512+0000","last_became_peered":"2024-09-21T05:47:52.316512+0000","last_unstale":"2024-09-21T05:47:54.419066+0000","last_undegraded":"2024-09-21T05:47:54.419066+0000","last_fullsized":"2024-09-21T05:47:54.419066+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:56.759531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419891+0000","last_change":"2024-09-21T05:47:52.321742+0000","last_active":"2024-09-21T05:47:54.419891+0000","last_peered":"2024-09-21T05:47:54.419891+0000","last_clean":"2024-09-21T05:47:54.419891+0000","last_became_active":"2024-09-21T05:47:52.321019+0000","last_became_peered":"2024-09-21T05:47:52.321019+0000","last_unstale":"2024-09-21T05:47:54.419891+0000","last_undegraded":"2024-09-21T05:47:54.419891+0000","last_fullsized":"2024-09-21T05:47:54.419891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:34.946405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420780+0000","last_change":"2024-09-21T05:47:53.335546+0000","last_active":"2024-09-21T05:47:54.420780+0000","last_peered":"2024-09-21T05:47:54.420780+0000","last_clean":"2024-09-21T05:47:54.420780+0000","last_became_active":"2024-09-21T05:47:53.335215+0000","last_became_peered":"2024-09-21T05:47:53.335215+0000","last_unstale":"2024-09-21T05:47:54.420780+0000","last_undegraded":"2024-09-21T05:47:54.420780+0000","last_fullsized":"2024-09-21T05:47:54.420780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:18.844619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319672+0000","last_change":"2024-09-21T05:47:52.319751+0000","last_active":"2024-09-21T05:47:54.319672+0000","last_peered":"2024-09-21T05:47:54.319672+0000","last_clean":"2024-09-21T05:47:54.319672+0000","last_became_active":"2024-09-21T05:47:52.319375+0000","last_became_peered":"2024-09-21T05:47:52.319375+0000","last_unstale":"2024-09-21T05:47:54.319672+0000","last_undegraded":"2024-09-21T05:47:54.319672+0000","last_fullsized":"2024-09-21T05:47:54.319672+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:30.482915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420237+0000","last_change":"2024-09-21T05:47:52.321620+0000","last_active":"2024-09-21T05:47:54.420237+0000","last_peered":"2024-09-21T05:47:54.420237+0000","last_clean":"2024-09-21T05:47:54.420237+0000","last_became_active":"2024-09-21T05:47:52.321166+0000","last_became_peered":"2024-09-21T05:47:52.321166+0000","last_unstale":"2024-09-21T05:47:54.420237+0000","last_undegraded":"2024-09-21T05:47:54.420237+0000","last_fullsized":"2024-09-21T05:47:54.420237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.967667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419550+0000","last_change":"2024-09-21T05:47:53.340973+0000","last_active":"2024-09-21T05:47:54.419550+0000","last_peered":"2024-09-21T05:47:54.419550+0000","last_clean":"2024-09-21T05:47:54.419550+0000","last_became_active":"2024-09-21T05:47:53.340603+0000","last_became_peered":"2024-09-21T05:47:53.340603+0000","last_unstale":"2024-09-21T05:47:54.419550+0000","last_undegraded":"2024-09-21T05:47:54.419550+0000","last_fullsized":"2024-09-21T05:47:54.419550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.320987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420772+0000","last_change":"2024-09-21T05:47:52.326447+0000","last_active":"2024-09-21T05:47:54.420772+0000","last_peered":"2024-09-21T05:47:54.420772+0000","last_clean":"2024-09-21T05:47:54.420772+0000","last_became_active":"2024-09-21T05:47:52.326064+0000","last_became_peered":"2024-09-21T05:47:52.326064+0000","last_unstale":"2024-09-21T05:47:54.420772+0000","last_undegraded":"2024-09-21T05:47:54.420772+0000","last_fullsized":"2024-09-21T05:47:54.420772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:27.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323686+0000","last_change":"2024-09-21T05:47:53.322645+0000","last_active":"2024-09-21T05:47:54.323686+0000","last_peered":"2024-09-21T05:47:54.323686+0000","last_clean":"2024-09-21T05:47:54.323686+0000","last_became_active":"2024-09-21T05:47:53.322384+0000","last_became_peered":"2024-09-21T05:47:53.322384+0000","last_unstale":"2024-09-21T05:47:54.323686+0000","last_undegraded":"2024-09-21T05:47:54.323686+0000","last_fullsized":"2024-09-21T05:47:54.323686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:06.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419780+0000","last_change":"2024-09-21T05:47:52.316775+0000","last_active":"2024-09-21T05:47:54.419780+0000","last_peered":"2024-09-21T05:47:54.419780+0000","last_clean":"2024-09-21T05:47:54.419780+0000","last_became_active":"2024-09-21T05:47:52.315552+0000","last_became_peered":"2024-09-21T05:47:52.315552+0000","last_unstale":"2024-09-21T05:47:54.419780+0000","last_undegraded":"2024-09-21T05:47:54.419780+0000","last_fullsized":"2024-09-21T05:47:54.419780+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:56.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421499+0000","last_change":"2024-09-21T05:47:53.329320+0000","last_active":"2024-09-21T05:47:54.421499+0000","last_peered":"2024-09-21T05:47:54.421499+0000","last_clean":"2024-09-21T05:47:54.421499+0000","last_became_active":"2024-09-21T05:47:53.328785+0000","last_became_peered":"2024-09-21T05:47:53.328785+0000","last_unstale":"2024-09-21T05:47:54.421499+0000","last_undegraded":"2024-09-21T05:47:54.421499+0000","last_fullsized":"2024-09-21T05:47:54.421499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:11.061652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:55.306521+0000","last_change":"2024-09-21T05:47:52.317943+0000","last_active":"2024-09-21T05:47:55.306521+0000","last_peered":"2024-09-21T05:47:55.306521+0000","last_clean":"2024-09-21T05:47:55.306521+0000","last_became_active":"2024-09-21T05:47:52.317606+0000","last_became_peered":"2024-09-21T05:47:52.317606+0000","last_unstale":"2024-09-21T05:47:55.306521+0000","last_undegraded":"2024-09-21T05:47:55.306521+0000","last_fullsized":"2024-09-21T05:47:55.306521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:53.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421251+0000","last_change":"2024-09-21T05:47:53.337568+0000","last_active":"2024-09-21T05:47:54.421251+0000","last_peered":"2024-09-21T05:47:54.421251+0000","last_clean":"2024-09-21T05:47:54.421251+0000","last_became_active":"2024-09-21T05:47:53.337068+0000","last_became_peered":"2024-09-21T05:47:53.337068+0000","last_unstale":"2024-09-21T05:47:54.421251+0000","last_undegraded":"2024-09-21T05:47:54.421251+0000","last_fullsized":"2024-09-21T05:47:54.421251+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:19.441038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421108+0000","last_change":"2024-09-21T05:47:52.325423+0000","last_active":"2024-09-21T05:47:54.421108+0000","last_peered":"2024-09-21T05:47:54.421108+0000","last_clean":"2024-09-21T05:47:54.421108+0000","last_became_active":"2024-09-21T05:47:52.325118+0000","last_became_peered":"2024-09-21T05:47:52.325118+0000","last_unstale":"2024-09-21T05:47:54.421108+0000","last_undegraded":"2024-09-21T05:47:54.421108+0000","last_fullsized":"2024-09-21T05:47:54.421108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:54.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323673+0000","last_change":"2024-09-21T05:47:53.329564+0000","last_active":"2024-09-21T05:47:54.323673+0000","last_peered":"2024-09-21T05:47:54.323673+0000","last_clean":"2024-09-21T05:47:54.323673+0000","last_became_active":"2024-09-21T05:47:53.329189+0000","last_became_peered":"2024-09-21T05:47:53.329189+0000","last_unstale":"2024-09-21T05:47:54.323673+0000","last_undegraded":"2024-09-21T05:47:54.323673+0000","last_fullsized":"2024-09-21T05:47:54.323673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:30.665406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420349+0000","last_change":"2024-09-21T05:47:52.317088+0000","last_active":"2024-09-21T05:47:54.420349+0000","last_peered":"2024-09-21T05:47:54.420349+0000","last_clean":"2024-09-21T05:47:54.420349+0000","last_became_active":"2024-09-21T05:47:52.316526+0000","last_became_peered":"2024-09-21T05:47:52.316526+0000","last_unstale":"2024-09-21T05:47:54.420349+0000","last_undegraded":"2024-09-21T05:47:54.420349+0000","last_fullsized":"2024-09-21T05:47:54.420349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:20.821196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419272+0000","last_change":"2024-09-21T05:47:52.314266+0000","last_active":"2024-09-21T05:47:54.419272+0000","last_peered":"2024-09-21T05:47:54.419272+0000","last_clean":"2024-09-21T05:47:54.419272+0000","last_became_active":"2024-09-21T05:47:52.313950+0000","last_became_peered":"2024-09-21T05:47:52.313950+0000","last_unstale":"2024-09-21T05:47:54.419272+0000","last_undegraded":"2024-09-21T05:47:54.419272+0000","last_fullsized":"2024-09-21T05:47:54.419272+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:54.199451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-09-21T05:47:51.294534+0000","last_change":"2024-09-21T05:47:51.294534+0000","last_active":"2024-09-21T05:47:51.289529+0000","last_peered":"2024-09-21T05:47:51.289529+0000","last_clean":"2024-09-21T05:47:51.289529+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:51.294534+0000","last_undegraded":"2024-09-21T05:47:51.294534+0000","last_fullsized":"2024-09-21T05:47:51.294534+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T05:47:52.291430+0000","last_change":"2024-09-21T05:47:52.291430+0000","last_active":"2024-09-21T05:47:52.291430+0000","last_peered":"2024-09-21T05:47:52.291430+0000","last_clean":"2024-09-21T05:47:52.291430+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T05:47:52.291430+0000","last_undegraded":"2024-09-21T05:47:52.291430+0000","last_fullsized":"2024-09-21T05:47:52.291430+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319732+0000","last_change":"2024-09-21T05:47:52.319216+0000","last_active":"2024-09-21T05:47:54.319732+0000","last_peered":"2024-09-21T05:47:54.319732+0000","last_clean":"2024-09-21T05:47:54.319732+0000","last_became_active":"2024-09-21T05:47:52.318756+0000","last_became_peered":"2024-09-21T05:47:52.318756+0000","last_unstale":"2024-09-21T05:47:54.319732+0000","last_undegraded":"2024-09-21T05:47:54.319732+0000","last_fullsized":"2024-09-21T05:47:54.319732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:12.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420865+0000","last_change":"2024-09-21T05:47:53.337719+0000","last_active":"2024-09-21T05:47:54.420865+0000","last_peered":"2024-09-21T05:47:54.420865+0000","last_clean":"2024-09-21T05:47:54.420865+0000","last_became_active":"2024-09-21T05:47:53.329968+0000","last_became_peered":"2024-09-21T05:47:53.329968+0000","last_unstale":"2024-09-21T05:47:54.420865+0000","last_undegraded":"2024-09-21T05:47:54.420865+0000","last_fullsized":"2024-09-21T05:47:54.420865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:11.856321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420792+0000","last_change":"2024-09-21T05:47:52.324337+0000","last_active":"2024-09-21T05:47:54.420792+0000","last_peered":"2024-09-21T05:47:54.420792+0000","last_clean":"2024-09-21T05:47:54.420792+0000","last_became_active":"2024-09-21T05:47:52.324032+0000","last_became_peered":"2024-09-21T05:47:52.324032+0000","last_unstale":"2024-09-21T05:47:54.420792+0000","last_undegraded":"2024-09-21T05:47:54.420792+0000","last_fullsized":"2024-09-21T05:47:54.420792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:03.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419235+0000","last_change":"2024-09-21T05:47:53.319715+0000","last_active":"2024-09-21T05:47:54.419235+0000","last_peered":"2024-09-21T05:47:54.419235+0000","last_clean":"2024-09-21T05:47:54.419235+0000","last_became_active":"2024-09-21T05:47:53.319394+0000","last_became_peered":"2024-09-21T05:47:53.319394+0000","last_unstale":"2024-09-21T05:47:54.419235+0000","last_undegraded":"2024-09-21T05:47:54.419235+0000","last_fullsized":"2024-09-21T05:47:54.419235+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:24.432342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419478+0000","last_change":"2024-09-21T05:47:52.316016+0000","last_active":"2024-09-21T05:47:54.419478+0000","last_peered":"2024-09-21T05:47:54.419478+0000","last_clean":"2024-09-21T05:47:54.419478+0000","last_became_active":"2024-09-21T05:47:52.315661+0000","last_became_peered":"2024-09-21T05:47:52.315661+0000","last_unstale":"2024-09-21T05:47:54.419478+0000","last_undegraded":"2024-09-21T05:47:54.419478+0000","last_fullsized":"2024-09-21T05:47:54.419478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:23.229664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":265,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-21T05:47:51.300954+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:47:51.300954+0000","last_peered":"2024-09-21T05:47:51.300954+0000","last_clean":"2024-09-21T05:47:51.300954+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:47:51.300954+0000","last_undegraded":"2024-09-21T05:47:51.300954+0000","last_fullsized":"2024-09-21T05:47:51.300954+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":84,"acting":84,"num_store_stats":6},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509548,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33048,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338792,"statfs":{"total":96636764160,"available":96602923008,"internally_reserved":0,"allocated":1904640,"data_stored":1154973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1298432,"data_stored":558788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1351680,"data_stored":584025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1355776,"data_stored":584559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1363968,"data_stored":585637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40840,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331000,"statfs":{"total":96636764160,"available":96594944000,"internally_reserved":0,"allocated":1355776,"data_stored":584079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542252,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":1176047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:47:57.768 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:47:57.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 -- 172.21.15.22:0/2228956358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070030 msgr2=0x7fb214074640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 --2- 172.21.15.22:0/2228956358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070030 0x7fb214074640 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb208005fd0 tx=0x7fb20802a4a0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 -- 172.21.15.22:0/2228956358 shutdown_connections 2024-09-21T05:47:57.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 --2- 172.21.15.22:0/2228956358 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070030 0x7fb214074640 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.921 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 --2- 172.21.15.22:0/2228956358 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb214073170 0x7fb21415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 -- 172.21.15.22:0/2228956358 >> 172.21.15.22:0/2228956358 conn(0x7fb2140b1cb0 msgr2=0x7fb2140b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 -- 172.21.15.22:0/2228956358 shutdown_connections 2024-09-21T05:47:57.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.919+0000 7fb2193f1640 1 -- 172.21.15.22:0/2228956358 wait complete. 2024-09-21T05:47:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.921+0000 7fb2193f1640 1 Processor -- start 2024-09-21T05:47:57.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.921+0000 7fb2193f1640 1 -- start start 2024-09-21T05:47:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2193f1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb214070030 0x7fb21415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2193f1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 0x7fb21409c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2193f1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb214073170 0x7fb2141e9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2193f1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb21406f5a0 con 0x7fb214070440 2024-09-21T05:47:57.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2193f1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb214074f80 con 0x7fb214073170 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2193f1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb21406f720 con 0x7fb214070030 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 0x7fb21409c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 0x7fb21409c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50576/0 (socket says 172.21.15.22:50576) 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 -- 172.21.15.22:0/3722876116 learned_addr learned my addr 172.21.15.22:0/3722876116 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2137fe640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb214073170 0x7fb2141e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb212ffd640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb214070030 0x7fb21415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 -- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb214070030 msgr2=0x7fb21415a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb214070030 0x7fb21415a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 -- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb214073170 msgr2=0x7fb2141e9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb214073170 0x7fb2141e9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.922+0000 7fb2127fc640 1 -- 172.21.15.22:0/3722876116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb21415b950 con 0x7fb214070440 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.923+0000 7fb2127fc640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 0x7fb21409c4f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb208034040 tx=0x7fb2080039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.923+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20803e030 con 0x7fb214070440 2024-09-21T05:47:57.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.923+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2080047a0 con 0x7fb214070440 2024-09-21T05:47:57.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.923+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2140ca8a0 con 0x7fb214070440 2024-09-21T05:47:57.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.923+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208035040 con 0x7fb214070440 2024-09-21T05:47:57.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.923+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2140cae60 con 0x7fb214070440 2024-09-21T05:47:57.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.924+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb21409ce40 con 0x7fb214070440 2024-09-21T05:47:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.925+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb208041030 con 0x7fb214070440 2024-09-21T05:47:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.925+0000 7fb1fb7fe640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb200078b90 0x7fb20007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.925+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fb2080bb510 con 0x7fb214070440 2024-09-21T05:47:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.925+0000 7fb212ffd640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb200078b90 0x7fb20007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:57.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.926+0000 7fb212ffd640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb200078b90 0x7fb20007b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb204005c40 tx=0x7fb20400a350 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:57.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:57.930+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb21415b950 con 0x7fb214070440 2024-09-21T05:47:58.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.117+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb21409d0d0 con 0x7fb200078b90 2024-09-21T05:47:58.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.122+0000 7fb1fb7fe640 1 -- 172.21.15.22:0/3722876116 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7fb21409d0d0 con 0x7fb200078b90 2024-09-21T05:47:58.127 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:58.128 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:47:58.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.125+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb200078b90 msgr2=0x7fb20007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.125+0000 7fb2193f1640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb200078b90 0x7fb20007b0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb204005c40 tx=0x7fb20400a350 comp rx=0 tx=0).stop 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.125+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 msgr2=0x7fb21409c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.125+0000 7fb2193f1640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 0x7fb21409c4f0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fb208034040 tx=0x7fb2080039e0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 shutdown_connections 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb214073170 0x7fb2141e9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb214070440 0x7fb21409c4f0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb200078b90 0x7fb20007b0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 --2- 172.21.15.22:0/3722876116 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb214070030 0x7fb21415a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 >> 172.21.15.22:0/3722876116 conn(0x7fb2140b1cb0 msgr2=0x7fb21415ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 shutdown_connections 2024-09-21T05:47:58.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.126+0000 7fb2193f1640 1 -- 172.21.15.22:0/3722876116 wait complete. 2024-09-21T05:47:58.139 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-21T05:47:57.878028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282100,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754692620,"statfs":{"total":773094113280,"available":772805242880,"internally_reserved":0,"allocated":12034048,"data_stored":5857196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001519"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421407+0000","last_change":"2024-09-21T05:47:53.329400+0000","last_active":"2024-09-21T05:47:54.421407+0000","last_peered":"2024-09-21T05:47:54.421407+0000","last_clean":"2024-09-21T05:47:54.421407+0000","last_became_active":"2024-09-21T05:47:53.328865+0000","last_became_peered":"2024-09-21T05:47:53.328865+0000","last_unstale":"2024-09-21T05:47:54.421407+0000","last_undegraded":"2024-09-21T05:47:54.421407+0000","last_fullsized":"2024-09-21T05:47:54.421407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:33.666746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322207+0000","last_change":"2024-09-21T05:47:53.315211+0000","last_active":"2024-09-21T05:47:54.322207+0000","last_peered":"2024-09-21T05:47:54.322207+0000","last_clean":"2024-09-21T05:47:54.322207+0000","last_became_active":"2024-09-21T05:47:53.314268+0000","last_became_peered":"2024-09-21T05:47:53.314268+0000","last_unstale":"2024-09-21T05:47:54.322207+0000","last_undegraded":"2024-09-21T05:47:54.322207+0000","last_fullsized":"2024-09-21T05:47:54.322207+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:15.893666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420190+0000","last_change":"2024-09-21T05:47:53.322252+0000","last_active":"2024-09-21T05:47:54.420190+0000","last_peered":"2024-09-21T05:47:54.420190+0000","last_clean":"2024-09-21T05:47:54.420190+0000","last_became_active":"2024-09-21T05:47:53.321925+0000","last_became_peered":"2024-09-21T05:47:53.321925+0000","last_unstale":"2024-09-21T05:47:54.420190+0000","last_undegraded":"2024-09-21T05:47:54.420190+0000","last_fullsized":"2024-09-21T05:47:54.420190+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:11.917317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420973+0000","last_change":"2024-09-21T05:47:53.338050+0000","last_active":"2024-09-21T05:47:54.420973+0000","last_peered":"2024-09-21T05:47:54.420973+0000","last_clean":"2024-09-21T05:47:54.420973+0000","last_became_active":"2024-09-21T05:47:53.337279+0000","last_became_peered":"2024-09-21T05:47:53.337279+0000","last_unstale":"2024-09-21T05:47:54.420973+0000","last_undegraded":"2024-09-21T05:47:54.420973+0000","last_fullsized":"2024-09-21T05:47:54.420973+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:49.343256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320073+0000","last_change":"2024-09-21T05:47:53.330191+0000","last_active":"2024-09-21T05:47:54.320073+0000","last_peered":"2024-09-21T05:47:54.320073+0000","last_clean":"2024-09-21T05:47:54.320073+0000","last_became_active":"2024-09-21T05:47:53.329578+0000","last_became_peered":"2024-09-21T05:47:53.329578+0000","last_unstale":"2024-09-21T05:47:54.320073+0000","last_undegraded":"2024-09-21T05:47:54.320073+0000","last_fullsized":"2024-09-21T05:47:54.320073+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:51.237035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420158+0000","last_change":"2024-09-21T05:47:53.314749+0000","last_active":"2024-09-21T05:47:54.420158+0000","last_peered":"2024-09-21T05:47:54.420158+0000","last_clean":"2024-09-21T05:47:54.420158+0000","last_became_active":"2024-09-21T05:47:53.314457+0000","last_became_peered":"2024-09-21T05:47:53.314457+0000","last_unstale":"2024-09-21T05:47:54.420158+0000","last_undegraded":"2024-09-21T05:47:54.420158+0000","last_fullsized":"2024-09-21T05:47:54.420158+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:39.746882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419166+0000","last_change":"2024-09-21T05:47:53.322397+0000","last_active":"2024-09-21T05:47:54.419166+0000","last_peered":"2024-09-21T05:47:54.419166+0000","last_clean":"2024-09-21T05:47:54.419166+0000","last_became_active":"2024-09-21T05:47:53.322048+0000","last_became_peered":"2024-09-21T05:47:53.322048+0000","last_unstale":"2024-09-21T05:47:54.419166+0000","last_undegraded":"2024-09-21T05:47:54.419166+0000","last_fullsized":"2024-09-21T05:47:54.419166+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:37.914628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.323792+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.323271+0000","last_became_peered":"2024-09-21T05:47:53.323271+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:22.199819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421377+0000","last_change":"2024-09-21T05:47:53.333144+0000","last_active":"2024-09-21T05:47:54.421377+0000","last_peered":"2024-09-21T05:47:54.421377+0000","last_clean":"2024-09-21T05:47:54.421377+0000","last_became_active":"2024-09-21T05:47:53.332816+0000","last_became_peered":"2024-09-21T05:47:53.332816+0000","last_unstale":"2024-09-21T05:47:54.421377+0000","last_undegraded":"2024-09-21T05:47:54.421377+0000","last_fullsized":"2024-09-21T05:47:54.421377+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:44.276704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421287+0000","last_change":"2024-09-21T05:47:53.328426+0000","last_active":"2024-09-21T05:47:54.421287+0000","last_peered":"2024-09-21T05:47:54.421287+0000","last_clean":"2024-09-21T05:47:54.421287+0000","last_became_active":"2024-09-21T05:47:53.327992+0000","last_became_peered":"2024-09-21T05:47:53.327992+0000","last_unstale":"2024-09-21T05:47:54.421287+0000","last_undegraded":"2024-09-21T05:47:54.421287+0000","last_fullsized":"2024-09-21T05:47:54.421287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:02.621431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321989+0000","last_change":"2024-09-21T05:47:53.318301+0000","last_active":"2024-09-21T05:47:54.321989+0000","last_peered":"2024-09-21T05:47:54.321989+0000","last_clean":"2024-09-21T05:47:54.321989+0000","last_became_active":"2024-09-21T05:47:53.317941+0000","last_became_peered":"2024-09-21T05:47:53.317941+0000","last_unstale":"2024-09-21T05:47:54.321989+0000","last_undegraded":"2024-09-21T05:47:54.321989+0000","last_fullsized":"2024-09-21T05:47:54.321989+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421042+0000","last_change":"2024-09-21T05:47:53.338328+0000","last_active":"2024-09-21T05:47:54.421042+0000","last_peered":"2024-09-21T05:47:54.421042+0000","last_clean":"2024-09-21T05:47:54.421042+0000","last_became_active":"2024-09-21T05:47:53.337637+0000","last_became_peered":"2024-09-21T05:47:53.337637+0000","last_unstale":"2024-09-21T05:47:54.421042+0000","last_undegraded":"2024-09-21T05:47:54.421042+0000","last_fullsized":"2024-09-21T05:47:54.421042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:07.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420323+0000","last_change":"2024-09-21T05:47:53.329762+0000","last_active":"2024-09-21T05:47:54.420323+0000","last_peered":"2024-09-21T05:47:54.420323+0000","last_clean":"2024-09-21T05:47:54.420323+0000","last_became_active":"2024-09-21T05:47:53.329426+0000","last_became_peered":"2024-09-21T05:47:53.329426+0000","last_unstale":"2024-09-21T05:47:54.420323+0000","last_undegraded":"2024-09-21T05:47:54.420323+0000","last_fullsized":"2024-09-21T05:47:54.420323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:44.894421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419422+0000","last_change":"2024-09-21T05:47:53.328646+0000","last_active":"2024-09-21T05:47:54.419422+0000","last_peered":"2024-09-21T05:47:54.419422+0000","last_clean":"2024-09-21T05:47:54.419422+0000","last_became_active":"2024-09-21T05:47:53.328313+0000","last_became_peered":"2024-09-21T05:47:53.328313+0000","last_unstale":"2024-09-21T05:47:54.419422+0000","last_undegraded":"2024-09-21T05:47:54.419422+0000","last_fullsized":"2024-09-21T05:47:54.419422+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:14.951874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419472+0000","last_change":"2024-09-21T05:47:53.314013+0000","last_active":"2024-09-21T05:47:54.419472+0000","last_peered":"2024-09-21T05:47:54.419472+0000","last_clean":"2024-09-21T05:47:54.419472+0000","last_became_active":"2024-09-21T05:47:53.313613+0000","last_became_peered":"2024-09-21T05:47:53.313613+0000","last_unstale":"2024-09-21T05:47:54.419472+0000","last_undegraded":"2024-09-21T05:47:54.419472+0000","last_fullsized":"2024-09-21T05:47:54.419472+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:09.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420533+0000","last_change":"2024-09-21T05:47:53.340669+0000","last_active":"2024-09-21T05:47:54.420533+0000","last_peered":"2024-09-21T05:47:54.420533+0000","last_clean":"2024-09-21T05:47:54.420533+0000","last_became_active":"2024-09-21T05:47:53.340336+0000","last_became_peered":"2024-09-21T05:47:53.340336+0000","last_unstale":"2024-09-21T05:47:54.420533+0000","last_undegraded":"2024-09-21T05:47:54.420533+0000","last_fullsized":"2024-09-21T05:47:54.420533+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:28.758224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322315+0000","last_change":"2024-09-21T05:47:53.320885+0000","last_active":"2024-09-21T05:47:54.322315+0000","last_peered":"2024-09-21T05:47:54.322315+0000","last_clean":"2024-09-21T05:47:54.322315+0000","last_became_active":"2024-09-21T05:47:53.320175+0000","last_became_peered":"2024-09-21T05:47:53.320175+0000","last_unstale":"2024-09-21T05:47:54.322315+0000","last_undegraded":"2024-09-21T05:47:54.322315+0000","last_fullsized":"2024-09-21T05:47:54.322315+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:57.310115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323869+0000","last_change":"2024-09-21T05:47:53.317110+0000","last_active":"2024-09-21T05:47:54.323869+0000","last_peered":"2024-09-21T05:47:54.323869+0000","last_clean":"2024-09-21T05:47:54.323869+0000","last_became_active":"2024-09-21T05:47:53.316725+0000","last_became_peered":"2024-09-21T05:47:53.316725+0000","last_unstale":"2024-09-21T05:47:54.323869+0000","last_undegraded":"2024-09-21T05:47:54.323869+0000","last_fullsized":"2024-09-21T05:47:54.323869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.043120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420287+0000","last_change":"2024-09-21T05:47:53.332561+0000","last_active":"2024-09-21T05:47:54.420287+0000","last_peered":"2024-09-21T05:47:54.420287+0000","last_clean":"2024-09-21T05:47:54.420287+0000","last_became_active":"2024-09-21T05:47:53.331867+0000","last_became_peered":"2024-09-21T05:47:53.331867+0000","last_unstale":"2024-09-21T05:47:54.420287+0000","last_undegraded":"2024-09-21T05:47:54.420287+0000","last_fullsized":"2024-09-21T05:47:54.420287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:16.303376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322107+0000","last_change":"2024-09-21T05:47:53.314765+0000","last_active":"2024-09-21T05:47:54.322107+0000","last_peered":"2024-09-21T05:47:54.322107+0000","last_clean":"2024-09-21T05:47:54.322107+0000","last_became_active":"2024-09-21T05:47:53.314417+0000","last_became_peered":"2024-09-21T05:47:53.314417+0000","last_unstale":"2024-09-21T05:47:54.322107+0000","last_undegraded":"2024-09-21T05:47:54.322107+0000","last_fullsized":"2024-09-21T05:47:54.322107+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:44.328561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322342+0000","last_change":"2024-09-21T05:47:53.323870+0000","last_active":"2024-09-21T05:47:54.322342+0000","last_peered":"2024-09-21T05:47:54.322342+0000","last_clean":"2024-09-21T05:47:54.322342+0000","last_became_active":"2024-09-21T05:47:53.323552+0000","last_became_peered":"2024-09-21T05:47:53.323552+0000","last_unstale":"2024-09-21T05:47:54.322342+0000","last_undegraded":"2024-09-21T05:47:54.322342+0000","last_fullsized":"2024-09-21T05:47:54.322342+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:33.010490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419862+0000","last_change":"2024-09-21T05:47:53.322155+0000","last_active":"2024-09-21T05:47:54.419862+0000","last_peered":"2024-09-21T05:47:54.419862+0000","last_clean":"2024-09-21T05:47:54.419862+0000","last_became_active":"2024-09-21T05:47:53.321833+0000","last_became_peered":"2024-09-21T05:47:53.321833+0000","last_unstale":"2024-09-21T05:47:54.419862+0000","last_undegraded":"2024-09-21T05:47:54.419862+0000","last_fullsized":"2024-09-21T05:47:54.419862+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:01.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321454+0000","last_change":"2024-09-21T05:47:53.325437+0000","last_active":"2024-09-21T05:47:54.321454+0000","last_peered":"2024-09-21T05:47:54.321454+0000","last_clean":"2024-09-21T05:47:54.321454+0000","last_became_active":"2024-09-21T05:47:53.318729+0000","last_became_peered":"2024-09-21T05:47:53.318729+0000","last_unstale":"2024-09-21T05:47:54.321454+0000","last_undegraded":"2024-09-21T05:47:54.321454+0000","last_fullsized":"2024-09-21T05:47:54.321454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:23.462572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419203+0000","last_change":"2024-09-21T05:47:53.320400+0000","last_active":"2024-09-21T05:47:54.419203+0000","last_peered":"2024-09-21T05:47:54.419203+0000","last_clean":"2024-09-21T05:47:54.419203+0000","last_became_active":"2024-09-21T05:47:53.320056+0000","last_became_peered":"2024-09-21T05:47:53.320056+0000","last_unstale":"2024-09-21T05:47:54.419203+0000","last_undegraded":"2024-09-21T05:47:54.419203+0000","last_fullsized":"2024-09-21T05:47:54.419203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:51.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322541+0000","last_change":"2024-09-21T05:47:53.321337+0000","last_active":"2024-09-21T05:47:54.322541+0000","last_peered":"2024-09-21T05:47:54.322541+0000","last_clean":"2024-09-21T05:47:54.322541+0000","last_became_active":"2024-09-21T05:47:53.320583+0000","last_became_peered":"2024-09-21T05:47:53.320583+0000","last_unstale":"2024-09-21T05:47:54.322541+0000","last_undegraded":"2024-09-21T05:47:54.322541+0000","last_fullsized":"2024-09-21T05:47:54.322541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:57.315042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421261+0000","last_change":"2024-09-21T05:47:53.335796+0000","last_active":"2024-09-21T05:47:54.421261+0000","last_peered":"2024-09-21T05:47:54.421261+0000","last_clean":"2024-09-21T05:47:54.421261+0000","last_became_active":"2024-09-21T05:47:53.335426+0000","last_became_peered":"2024-09-21T05:47:53.335426+0000","last_unstale":"2024-09-21T05:47:54.421261+0000","last_undegraded":"2024-09-21T05:47:54.421261+0000","last_fullsized":"2024-09-21T05:47:54.421261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:32.715401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419441+0000","last_change":"2024-09-21T05:47:53.328721+0000","last_active":"2024-09-21T05:47:54.419441+0000","last_peered":"2024-09-21T05:47:54.419441+0000","last_clean":"2024-09-21T05:47:54.419441+0000","last_became_active":"2024-09-21T05:47:53.328431+0000","last_became_peered":"2024-09-21T05:47:53.328431+0000","last_unstale":"2024-09-21T05:47:54.419441+0000","last_undegraded":"2024-09-21T05:47:54.419441+0000","last_fullsized":"2024-09-21T05:47:54.419441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:16.400560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420065+0000","last_change":"2024-09-21T05:47:53.339459+0000","last_active":"2024-09-21T05:47:54.420065+0000","last_peered":"2024-09-21T05:47:54.420065+0000","last_clean":"2024-09-21T05:47:54.420065+0000","last_became_active":"2024-09-21T05:47:53.339054+0000","last_became_peered":"2024-09-21T05:47:53.339054+0000","last_unstale":"2024-09-21T05:47:54.420065+0000","last_undegraded":"2024-09-21T05:47:54.420065+0000","last_fullsized":"2024-09-21T05:47:54.420065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:16.576540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321766+0000","last_change":"2024-09-21T05:47:53.319552+0000","last_active":"2024-09-21T05:47:54.321766+0000","last_peered":"2024-09-21T05:47:54.321766+0000","last_clean":"2024-09-21T05:47:54.321766+0000","last_became_active":"2024-09-21T05:47:53.319101+0000","last_became_peered":"2024-09-21T05:47:53.319101+0000","last_unstale":"2024-09-21T05:47:54.321766+0000","last_undegraded":"2024-09-21T05:47:54.321766+0000","last_fullsized":"2024-09-21T05:47:54.321766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:44.232318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419434+0000","last_change":"2024-09-21T05:47:53.323957+0000","last_active":"2024-09-21T05:47:54.419434+0000","last_peered":"2024-09-21T05:47:54.419434+0000","last_clean":"2024-09-21T05:47:54.419434+0000","last_became_active":"2024-09-21T05:47:53.323432+0000","last_became_peered":"2024-09-21T05:47:53.323432+0000","last_unstale":"2024-09-21T05:47:54.419434+0000","last_undegraded":"2024-09-21T05:47:54.419434+0000","last_fullsized":"2024-09-21T05:47:54.419434+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:57.449767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420771+0000","last_change":"2024-09-21T05:47:53.340040+0000","last_active":"2024-09-21T05:47:54.420771+0000","last_peered":"2024-09-21T05:47:54.420771+0000","last_clean":"2024-09-21T05:47:54.420771+0000","last_became_active":"2024-09-21T05:47:53.339702+0000","last_became_peered":"2024-09-21T05:47:53.339702+0000","last_unstale":"2024-09-21T05:47:54.420771+0000","last_undegraded":"2024-09-21T05:47:54.420771+0000","last_fullsized":"2024-09-21T05:47:54.420771+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:02.133659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.320519+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.320182+0000","last_became_peered":"2024-09-21T05:47:53.320182+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:16.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419700+0000","last_change":"2024-09-21T05:47:53.327229+0000","last_active":"2024-09-21T05:47:54.419700+0000","last_peered":"2024-09-21T05:47:54.419700+0000","last_clean":"2024-09-21T05:47:54.419700+0000","last_became_active":"2024-09-21T05:47:53.326881+0000","last_became_peered":"2024-09-21T05:47:53.326881+0000","last_unstale":"2024-09-21T05:47:54.419700+0000","last_undegraded":"2024-09-21T05:47:54.419700+0000","last_fullsized":"2024-09-21T05:47:54.419700+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:43.659296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":19,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419430+0000","last_change":"2024-09-21T05:47:52.315229+0000","last_active":"2024-09-21T05:47:54.419430+0000","last_peered":"2024-09-21T05:47:54.419430+0000","last_clean":"2024-09-21T05:47:54.419430+0000","last_became_active":"2024-09-21T05:47:52.314905+0000","last_became_peered":"2024-09-21T05:47:52.314905+0000","last_unstale":"2024-09-21T05:47:54.419430+0000","last_undegraded":"2024-09-21T05:47:54.419430+0000","last_fullsized":"2024-09-21T05:47:54.419430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:00.549432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323756+0000","last_change":"2024-09-21T05:47:53.325534+0000","last_active":"2024-09-21T05:47:54.323756+0000","last_peered":"2024-09-21T05:47:54.323756+0000","last_clean":"2024-09-21T05:47:54.323756+0000","last_became_active":"2024-09-21T05:47:53.325200+0000","last_became_peered":"2024-09-21T05:47:53.325200+0000","last_unstale":"2024-09-21T05:47:54.323756+0000","last_undegraded":"2024-09-21T05:47:54.323756+0000","last_fullsized":"2024-09-21T05:47:54.323756+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:49.932618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420018+0000","last_change":"2024-09-21T05:47:52.317856+0000","last_active":"2024-09-21T05:47:54.420018+0000","last_peered":"2024-09-21T05:47:54.420018+0000","last_clean":"2024-09-21T05:47:54.420018+0000","last_became_active":"2024-09-21T05:47:52.317462+0000","last_became_peered":"2024-09-21T05:47:52.317462+0000","last_unstale":"2024-09-21T05:47:54.420018+0000","last_undegraded":"2024-09-21T05:47:54.420018+0000","last_fullsized":"2024-09-21T05:47:54.420018+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:58.602683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420098+0000","last_change":"2024-09-21T05:47:53.314734+0000","last_active":"2024-09-21T05:47:54.420098+0000","last_peered":"2024-09-21T05:47:54.420098+0000","last_clean":"2024-09-21T05:47:54.420098+0000","last_became_active":"2024-09-21T05:47:53.314373+0000","last_became_peered":"2024-09-21T05:47:53.314373+0000","last_unstale":"2024-09-21T05:47:54.420098+0000","last_undegraded":"2024-09-21T05:47:54.420098+0000","last_fullsized":"2024-09-21T05:47:54.420098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:32.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319979+0000","last_change":"2024-09-21T05:47:52.320521+0000","last_active":"2024-09-21T05:47:54.319979+0000","last_peered":"2024-09-21T05:47:54.319979+0000","last_clean":"2024-09-21T05:47:54.319979+0000","last_became_active":"2024-09-21T05:47:52.320077+0000","last_became_peered":"2024-09-21T05:47:52.320077+0000","last_unstale":"2024-09-21T05:47:54.319979+0000","last_undegraded":"2024-09-21T05:47:54.319979+0000","last_fullsized":"2024-09-21T05:47:54.319979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:11.737474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420363+0000","last_change":"2024-09-21T05:47:53.329736+0000","last_active":"2024-09-21T05:47:54.420363+0000","last_peered":"2024-09-21T05:47:54.420363+0000","last_clean":"2024-09-21T05:47:54.420363+0000","last_became_active":"2024-09-21T05:47:53.329390+0000","last_became_peered":"2024-09-21T05:47:53.329390+0000","last_unstale":"2024-09-21T05:47:54.420363+0000","last_undegraded":"2024-09-21T05:47:54.420363+0000","last_fullsized":"2024-09-21T05:47:54.420363+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:27.003776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319826+0000","last_change":"2024-09-21T05:47:52.321048+0000","last_active":"2024-09-21T05:47:54.319826+0000","last_peered":"2024-09-21T05:47:54.319826+0000","last_clean":"2024-09-21T05:47:54.319826+0000","last_became_active":"2024-09-21T05:47:52.320701+0000","last_became_peered":"2024-09-21T05:47:52.320701+0000","last_unstale":"2024-09-21T05:47:54.319826+0000","last_undegraded":"2024-09-21T05:47:54.319826+0000","last_fullsized":"2024-09-21T05:47:54.319826+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:30.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420672+0000","last_change":"2024-09-21T05:47:53.336978+0000","last_active":"2024-09-21T05:47:54.420672+0000","last_peered":"2024-09-21T05:47:54.420672+0000","last_clean":"2024-09-21T05:47:54.420672+0000","last_became_active":"2024-09-21T05:47:53.336304+0000","last_became_peered":"2024-09-21T05:47:53.336304+0000","last_unstale":"2024-09-21T05:47:54.420672+0000","last_undegraded":"2024-09-21T05:47:54.420672+0000","last_fullsized":"2024-09-21T05:47:54.420672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:23.474145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419148+0000","last_change":"2024-09-21T05:47:52.316047+0000","last_active":"2024-09-21T05:47:54.419148+0000","last_peered":"2024-09-21T05:47:54.419148+0000","last_clean":"2024-09-21T05:47:54.419148+0000","last_became_active":"2024-09-21T05:47:52.315653+0000","last_became_peered":"2024-09-21T05:47:52.315653+0000","last_unstale":"2024-09-21T05:47:54.419148+0000","last_undegraded":"2024-09-21T05:47:54.419148+0000","last_fullsized":"2024-09-21T05:47:54.419148+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:29.936206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321828+0000","last_change":"2024-09-21T05:47:53.321821+0000","last_active":"2024-09-21T05:47:54.321828+0000","last_peered":"2024-09-21T05:47:54.321828+0000","last_clean":"2024-09-21T05:47:54.321828+0000","last_became_active":"2024-09-21T05:47:53.321524+0000","last_became_peered":"2024-09-21T05:47:53.321524+0000","last_unstale":"2024-09-21T05:47:54.321828+0000","last_undegraded":"2024-09-21T05:47:54.321828+0000","last_fullsized":"2024-09-21T05:47:54.321828+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:49.030281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420062+0000","last_change":"2024-09-21T05:47:52.315884+0000","last_active":"2024-09-21T05:47:54.420062+0000","last_peered":"2024-09-21T05:47:54.420062+0000","last_clean":"2024-09-21T05:47:54.420062+0000","last_became_active":"2024-09-21T05:47:52.315492+0000","last_became_peered":"2024-09-21T05:47:52.315492+0000","last_unstale":"2024-09-21T05:47:54.420062+0000","last_undegraded":"2024-09-21T05:47:54.420062+0000","last_fullsized":"2024-09-21T05:47:54.420062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:47.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420539+0000","last_change":"2024-09-21T05:47:53.339451+0000","last_active":"2024-09-21T05:47:54.420539+0000","last_peered":"2024-09-21T05:47:54.420539+0000","last_clean":"2024-09-21T05:47:54.420539+0000","last_became_active":"2024-09-21T05:47:53.339042+0000","last_became_peered":"2024-09-21T05:47:53.339042+0000","last_unstale":"2024-09-21T05:47:54.420539+0000","last_undegraded":"2024-09-21T05:47:54.420539+0000","last_fullsized":"2024-09-21T05:47:54.420539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:24.227545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321401+0000","last_change":"2024-09-21T05:47:52.322566+0000","last_active":"2024-09-21T05:47:54.321401+0000","last_peered":"2024-09-21T05:47:54.321401+0000","last_clean":"2024-09-21T05:47:54.321401+0000","last_became_active":"2024-09-21T05:47:52.322294+0000","last_became_peered":"2024-09-21T05:47:52.322294+0000","last_unstale":"2024-09-21T05:47:54.321401+0000","last_undegraded":"2024-09-21T05:47:54.321401+0000","last_fullsized":"2024-09-21T05:47:54.321401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:33.895532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419785+0000","last_change":"2024-09-21T05:47:53.324736+0000","last_active":"2024-09-21T05:47:54.419785+0000","last_peered":"2024-09-21T05:47:54.419785+0000","last_clean":"2024-09-21T05:47:54.419785+0000","last_became_active":"2024-09-21T05:47:53.324398+0000","last_became_peered":"2024-09-21T05:47:53.324398+0000","last_unstale":"2024-09-21T05:47:54.419785+0000","last_undegraded":"2024-09-21T05:47:54.419785+0000","last_fullsized":"2024-09-21T05:47:54.419785+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:14.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420656+0000","last_change":"2024-09-21T05:47:52.315887+0000","last_active":"2024-09-21T05:47:54.420656+0000","last_peered":"2024-09-21T05:47:54.420656+0000","last_clean":"2024-09-21T05:47:54.420656+0000","last_became_active":"2024-09-21T05:47:52.315504+0000","last_became_peered":"2024-09-21T05:47:52.315504+0000","last_unstale":"2024-09-21T05:47:54.420656+0000","last_undegraded":"2024-09-21T05:47:54.420656+0000","last_fullsized":"2024-09-21T05:47:54.420656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:57.957523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419923+0000","last_change":"2024-09-21T05:47:53.329242+0000","last_active":"2024-09-21T05:47:54.419923+0000","last_peered":"2024-09-21T05:47:54.419923+0000","last_clean":"2024-09-21T05:47:54.419923+0000","last_became_active":"2024-09-21T05:47:53.327266+0000","last_became_peered":"2024-09-21T05:47:53.327266+0000","last_unstale":"2024-09-21T05:47:54.419923+0000","last_undegraded":"2024-09-21T05:47:54.419923+0000","last_fullsized":"2024-09-21T05:47:54.419923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:53.808702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320068+0000","last_change":"2024-09-21T05:47:52.318716+0000","last_active":"2024-09-21T05:47:54.320068+0000","last_peered":"2024-09-21T05:47:54.320068+0000","last_clean":"2024-09-21T05:47:54.320068+0000","last_became_active":"2024-09-21T05:47:52.318376+0000","last_became_peered":"2024-09-21T05:47:52.318376+0000","last_unstale":"2024-09-21T05:47:54.320068+0000","last_undegraded":"2024-09-21T05:47:54.320068+0000","last_fullsized":"2024-09-21T05:47:54.320068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:36.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323810+0000","last_change":"2024-09-21T05:47:53.330894+0000","last_active":"2024-09-21T05:47:54.323810+0000","last_peered":"2024-09-21T05:47:54.323810+0000","last_clean":"2024-09-21T05:47:54.323810+0000","last_became_active":"2024-09-21T05:47:53.330525+0000","last_became_peered":"2024-09-21T05:47:53.330525+0000","last_unstale":"2024-09-21T05:47:54.323810+0000","last_undegraded":"2024-09-21T05:47:54.323810+0000","last_fullsized":"2024-09-21T05:47:54.323810+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:53.569567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419648+0000","last_change":"2024-09-21T05:47:52.315933+0000","last_active":"2024-09-21T05:47:54.419648+0000","last_peered":"2024-09-21T05:47:54.419648+0000","last_clean":"2024-09-21T05:47:54.419648+0000","last_became_active":"2024-09-21T05:47:52.315578+0000","last_became_peered":"2024-09-21T05:47:52.315578+0000","last_unstale":"2024-09-21T05:47:54.419648+0000","last_undegraded":"2024-09-21T05:47:54.419648+0000","last_fullsized":"2024-09-21T05:47:54.419648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:07.116156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420303+0000","last_change":"2024-09-21T05:47:53.320523+0000","last_active":"2024-09-21T05:47:54.420303+0000","last_peered":"2024-09-21T05:47:54.420303+0000","last_clean":"2024-09-21T05:47:54.420303+0000","last_became_active":"2024-09-21T05:47:53.320183+0000","last_became_peered":"2024-09-21T05:47:53.320183+0000","last_unstale":"2024-09-21T05:47:54.420303+0000","last_undegraded":"2024-09-21T05:47:54.420303+0000","last_fullsized":"2024-09-21T05:47:54.420303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:58.959425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421035+0000","last_change":"2024-09-21T05:47:52.322650+0000","last_active":"2024-09-21T05:47:54.421035+0000","last_peered":"2024-09-21T05:47:54.421035+0000","last_clean":"2024-09-21T05:47:54.421035+0000","last_became_active":"2024-09-21T05:47:52.322310+0000","last_became_peered":"2024-09-21T05:47:52.322310+0000","last_unstale":"2024-09-21T05:47:54.421035+0000","last_undegraded":"2024-09-21T05:47:54.421035+0000","last_fullsized":"2024-09-21T05:47:54.421035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:23.430441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419420+0000","last_change":"2024-09-21T05:47:53.326552+0000","last_active":"2024-09-21T05:47:54.419420+0000","last_peered":"2024-09-21T05:47:54.419420+0000","last_clean":"2024-09-21T05:47:54.419420+0000","last_became_active":"2024-09-21T05:47:53.326200+0000","last_became_peered":"2024-09-21T05:47:53.326200+0000","last_unstale":"2024-09-21T05:47:54.419420+0000","last_undegraded":"2024-09-21T05:47:54.419420+0000","last_fullsized":"2024-09-21T05:47:54.419420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:13.011382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":21,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421178+0000","last_change":"2024-09-21T05:47:52.324477+0000","last_active":"2024-09-21T05:47:54.421178+0000","last_peered":"2024-09-21T05:47:54.421178+0000","last_clean":"2024-09-21T05:47:54.421178+0000","last_became_active":"2024-09-21T05:47:52.324166+0000","last_became_peered":"2024-09-21T05:47:52.324166+0000","last_unstale":"2024-09-21T05:47:54.421178+0000","last_undegraded":"2024-09-21T05:47:54.421178+0000","last_fullsized":"2024-09-21T05:47:54.421178+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:00.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419994+0000","last_change":"2024-09-21T05:47:53.326873+0000","last_active":"2024-09-21T05:47:54.419994+0000","last_peered":"2024-09-21T05:47:54.419994+0000","last_clean":"2024-09-21T05:47:54.419994+0000","last_became_active":"2024-09-21T05:47:53.326548+0000","last_became_peered":"2024-09-21T05:47:53.326548+0000","last_unstale":"2024-09-21T05:47:54.419994+0000","last_undegraded":"2024-09-21T05:47:54.419994+0000","last_fullsized":"2024-09-21T05:47:54.419994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:34.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419775+0000","last_change":"2024-09-21T05:47:52.316414+0000","last_active":"2024-09-21T05:47:54.419775+0000","last_peered":"2024-09-21T05:47:54.419775+0000","last_clean":"2024-09-21T05:47:54.419775+0000","last_became_active":"2024-09-21T05:47:52.316075+0000","last_became_peered":"2024-09-21T05:47:52.316075+0000","last_unstale":"2024-09-21T05:47:54.419775+0000","last_undegraded":"2024-09-21T05:47:54.419775+0000","last_fullsized":"2024-09-21T05:47:54.419775+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:58.423946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419724+0000","last_change":"2024-09-21T05:47:53.331201+0000","last_active":"2024-09-21T05:47:54.419724+0000","last_peered":"2024-09-21T05:47:54.419724+0000","last_clean":"2024-09-21T05:47:54.419724+0000","last_became_active":"2024-09-21T05:47:53.330540+0000","last_became_peered":"2024-09-21T05:47:53.330540+0000","last_unstale":"2024-09-21T05:47:54.419724+0000","last_undegraded":"2024-09-21T05:47:54.419724+0000","last_fullsized":"2024-09-21T05:47:54.419724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:19.803831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323431+0000","last_change":"2024-09-21T05:47:52.327286+0000","last_active":"2024-09-21T05:47:54.323431+0000","last_peered":"2024-09-21T05:47:54.323431+0000","last_clean":"2024-09-21T05:47:54.323431+0000","last_became_active":"2024-09-21T05:47:52.326918+0000","last_became_peered":"2024-09-21T05:47:52.326918+0000","last_unstale":"2024-09-21T05:47:54.323431+0000","last_undegraded":"2024-09-21T05:47:54.323431+0000","last_fullsized":"2024-09-21T05:47:54.323431+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:21.234490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419301+0000","last_change":"2024-09-21T05:47:53.319222+0000","last_active":"2024-09-21T05:47:54.419301+0000","last_peered":"2024-09-21T05:47:54.419301+0000","last_clean":"2024-09-21T05:47:54.419301+0000","last_became_active":"2024-09-21T05:47:53.318912+0000","last_became_peered":"2024-09-21T05:47:53.318912+0000","last_unstale":"2024-09-21T05:47:54.419301+0000","last_undegraded":"2024-09-21T05:47:54.419301+0000","last_fullsized":"2024-09-21T05:47:54.419301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:55.408820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":23,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319663+0000","last_change":"2024-09-21T05:47:52.318238+0000","last_active":"2024-09-21T05:47:54.319663+0000","last_peered":"2024-09-21T05:47:54.319663+0000","last_clean":"2024-09-21T05:47:54.319663+0000","last_became_active":"2024-09-21T05:47:52.317230+0000","last_became_peered":"2024-09-21T05:47:52.317230+0000","last_unstale":"2024-09-21T05:47:54.319663+0000","last_undegraded":"2024-09-21T05:47:54.319663+0000","last_fullsized":"2024-09-21T05:47:54.319663+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:37.840490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320255+0000","last_change":"2024-09-21T05:47:53.321414+0000","last_active":"2024-09-21T05:47:54.320255+0000","last_peered":"2024-09-21T05:47:54.320255+0000","last_clean":"2024-09-21T05:47:54.320255+0000","last_became_active":"2024-09-21T05:47:53.321109+0000","last_became_peered":"2024-09-21T05:47:53.321109+0000","last_unstale":"2024-09-21T05:47:54.320255+0000","last_undegraded":"2024-09-21T05:47:54.320255+0000","last_fullsized":"2024-09-21T05:47:54.320255+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:49.957606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421568+0000","last_change":"2024-09-21T05:47:52.326635+0000","last_active":"2024-09-21T05:47:54.421568+0000","last_peered":"2024-09-21T05:47:54.421568+0000","last_clean":"2024-09-21T05:47:54.421568+0000","last_became_active":"2024-09-21T05:47:52.326303+0000","last_became_peered":"2024-09-21T05:47:52.326303+0000","last_unstale":"2024-09-21T05:47:54.421568+0000","last_undegraded":"2024-09-21T05:47:54.421568+0000","last_fullsized":"2024-09-21T05:47:54.421568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.319086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419800+0000","last_change":"2024-09-21T05:47:53.334483+0000","last_active":"2024-09-21T05:47:54.419800+0000","last_peered":"2024-09-21T05:47:54.419800+0000","last_clean":"2024-09-21T05:47:54.419800+0000","last_became_active":"2024-09-21T05:47:53.334127+0000","last_became_peered":"2024-09-21T05:47:53.334127+0000","last_unstale":"2024-09-21T05:47:54.419800+0000","last_undegraded":"2024-09-21T05:47:54.419800+0000","last_fullsized":"2024-09-21T05:47:54.419800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.795868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320154+0000","last_change":"2024-09-21T05:47:52.314440+0000","last_active":"2024-09-21T05:47:54.320154+0000","last_peered":"2024-09-21T05:47:54.320154+0000","last_clean":"2024-09-21T05:47:54.320154+0000","last_became_active":"2024-09-21T05:47:52.314083+0000","last_became_peered":"2024-09-21T05:47:52.314083+0000","last_unstale":"2024-09-21T05:47:54.320154+0000","last_undegraded":"2024-09-21T05:47:54.320154+0000","last_fullsized":"2024-09-21T05:47:54.320154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:05.210599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420081+0000","last_change":"2024-09-21T05:47:53.339114+0000","last_active":"2024-09-21T05:47:54.420081+0000","last_peered":"2024-09-21T05:47:54.420081+0000","last_clean":"2024-09-21T05:47:54.420081+0000","last_became_active":"2024-09-21T05:47:53.338663+0000","last_became_peered":"2024-09-21T05:47:53.338663+0000","last_unstale":"2024-09-21T05:47:54.420081+0000","last_undegraded":"2024-09-21T05:47:54.420081+0000","last_fullsized":"2024-09-21T05:47:54.420081+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:50.046862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323537+0000","last_change":"2024-09-21T05:47:52.318503+0000","last_active":"2024-09-21T05:47:54.323537+0000","last_peered":"2024-09-21T05:47:54.323537+0000","last_clean":"2024-09-21T05:47:54.323537+0000","last_became_active":"2024-09-21T05:47:52.318174+0000","last_became_peered":"2024-09-21T05:47:52.318174+0000","last_unstale":"2024-09-21T05:47:54.323537+0000","last_undegraded":"2024-09-21T05:47:54.323537+0000","last_fullsized":"2024-09-21T05:47:54.323537+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:30.834390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420460+0000","last_change":"2024-09-21T05:47:53.330917+0000","last_active":"2024-09-21T05:47:54.420460+0000","last_peered":"2024-09-21T05:47:54.420460+0000","last_clean":"2024-09-21T05:47:54.420460+0000","last_became_active":"2024-09-21T05:47:53.330486+0000","last_became_peered":"2024-09-21T05:47:53.330486+0000","last_unstale":"2024-09-21T05:47:54.420460+0000","last_undegraded":"2024-09-21T05:47:54.420460+0000","last_fullsized":"2024-09-21T05:47:54.420460+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:00.420835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419066+0000","last_change":"2024-09-21T05:47:52.316818+0000","last_active":"2024-09-21T05:47:54.419066+0000","last_peered":"2024-09-21T05:47:54.419066+0000","last_clean":"2024-09-21T05:47:54.419066+0000","last_became_active":"2024-09-21T05:47:52.316512+0000","last_became_peered":"2024-09-21T05:47:52.316512+0000","last_unstale":"2024-09-21T05:47:54.419066+0000","last_undegraded":"2024-09-21T05:47:54.419066+0000","last_fullsized":"2024-09-21T05:47:54.419066+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:56.759531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419677+0000","last_change":"2024-09-21T05:47:53.322629+0000","last_active":"2024-09-21T05:47:54.419677+0000","last_peered":"2024-09-21T05:47:54.419677+0000","last_clean":"2024-09-21T05:47:54.419677+0000","last_became_active":"2024-09-21T05:47:53.322243+0000","last_became_peered":"2024-09-21T05:47:53.322243+0000","last_unstale":"2024-09-21T05:47:54.419677+0000","last_undegraded":"2024-09-21T05:47:54.419677+0000","last_fullsized":"2024-09-21T05:47:54.419677+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.624866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419891+0000","last_change":"2024-09-21T05:47:52.321742+0000","last_active":"2024-09-21T05:47:54.419891+0000","last_peered":"2024-09-21T05:47:54.419891+0000","last_clean":"2024-09-21T05:47:54.419891+0000","last_became_active":"2024-09-21T05:47:52.321019+0000","last_became_peered":"2024-09-21T05:47:52.321019+0000","last_unstale":"2024-09-21T05:47:54.419891+0000","last_undegraded":"2024-09-21T05:47:54.419891+0000","last_fullsized":"2024-09-21T05:47:54.419891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:34.946405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420780+0000","last_change":"2024-09-21T05:47:53.335546+0000","last_active":"2024-09-21T05:47:54.420780+0000","last_peered":"2024-09-21T05:47:54.420780+0000","last_clean":"2024-09-21T05:47:54.420780+0000","last_became_active":"2024-09-21T05:47:53.335215+0000","last_became_peered":"2024-09-21T05:47:53.335215+0000","last_unstale":"2024-09-21T05:47:54.420780+0000","last_undegraded":"2024-09-21T05:47:54.420780+0000","last_fullsized":"2024-09-21T05:47:54.420780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:18.844619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319672+0000","last_change":"2024-09-21T05:47:52.319751+0000","last_active":"2024-09-21T05:47:54.319672+0000","last_peered":"2024-09-21T05:47:54.319672+0000","last_clean":"2024-09-21T05:47:54.319672+0000","last_became_active":"2024-09-21T05:47:52.319375+0000","last_became_peered":"2024-09-21T05:47:52.319375+0000","last_unstale":"2024-09-21T05:47:54.319672+0000","last_undegraded":"2024-09-21T05:47:54.319672+0000","last_fullsized":"2024-09-21T05:47:54.319672+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:30.482915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419571+0000","last_change":"2024-09-21T05:47:53.332300+0000","last_active":"2024-09-21T05:47:54.419571+0000","last_peered":"2024-09-21T05:47:54.419571+0000","last_clean":"2024-09-21T05:47:54.419571+0000","last_became_active":"2024-09-21T05:47:53.329284+0000","last_became_peered":"2024-09-21T05:47:53.329284+0000","last_unstale":"2024-09-21T05:47:54.419571+0000","last_undegraded":"2024-09-21T05:47:54.419571+0000","last_fullsized":"2024-09-21T05:47:54.419571+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:26.838302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420237+0000","last_change":"2024-09-21T05:47:52.321620+0000","last_active":"2024-09-21T05:47:54.420237+0000","last_peered":"2024-09-21T05:47:54.420237+0000","last_clean":"2024-09-21T05:47:54.420237+0000","last_became_active":"2024-09-21T05:47:52.321166+0000","last_became_peered":"2024-09-21T05:47:52.321166+0000","last_unstale":"2024-09-21T05:47:54.420237+0000","last_undegraded":"2024-09-21T05:47:54.420237+0000","last_fullsized":"2024-09-21T05:47:54.420237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.967667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419550+0000","last_change":"2024-09-21T05:47:53.340973+0000","last_active":"2024-09-21T05:47:54.419550+0000","last_peered":"2024-09-21T05:47:54.419550+0000","last_clean":"2024-09-21T05:47:54.419550+0000","last_became_active":"2024-09-21T05:47:53.340603+0000","last_became_peered":"2024-09-21T05:47:53.340603+0000","last_unstale":"2024-09-21T05:47:54.419550+0000","last_undegraded":"2024-09-21T05:47:54.419550+0000","last_fullsized":"2024-09-21T05:47:54.419550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.320987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420772+0000","last_change":"2024-09-21T05:47:52.326447+0000","last_active":"2024-09-21T05:47:54.420772+0000","last_peered":"2024-09-21T05:47:54.420772+0000","last_clean":"2024-09-21T05:47:54.420772+0000","last_became_active":"2024-09-21T05:47:52.326064+0000","last_became_peered":"2024-09-21T05:47:52.326064+0000","last_unstale":"2024-09-21T05:47:54.420772+0000","last_undegraded":"2024-09-21T05:47:54.420772+0000","last_fullsized":"2024-09-21T05:47:54.420772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:27.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323686+0000","last_change":"2024-09-21T05:47:53.322645+0000","last_active":"2024-09-21T05:47:54.323686+0000","last_peered":"2024-09-21T05:47:54.323686+0000","last_clean":"2024-09-21T05:47:54.323686+0000","last_became_active":"2024-09-21T05:47:53.322384+0000","last_became_peered":"2024-09-21T05:47:53.322384+0000","last_unstale":"2024-09-21T05:47:54.323686+0000","last_undegraded":"2024-09-21T05:47:54.323686+0000","last_fullsized":"2024-09-21T05:47:54.323686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:06.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419780+0000","last_change":"2024-09-21T05:47:52.316775+0000","last_active":"2024-09-21T05:47:54.419780+0000","last_peered":"2024-09-21T05:47:54.419780+0000","last_clean":"2024-09-21T05:47:54.419780+0000","last_became_active":"2024-09-21T05:47:52.315552+0000","last_became_peered":"2024-09-21T05:47:52.315552+0000","last_unstale":"2024-09-21T05:47:54.419780+0000","last_undegraded":"2024-09-21T05:47:54.419780+0000","last_fullsized":"2024-09-21T05:47:54.419780+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:56.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421499+0000","last_change":"2024-09-21T05:47:53.329320+0000","last_active":"2024-09-21T05:47:54.421499+0000","last_peered":"2024-09-21T05:47:54.421499+0000","last_clean":"2024-09-21T05:47:54.421499+0000","last_became_active":"2024-09-21T05:47:53.328785+0000","last_became_peered":"2024-09-21T05:47:53.328785+0000","last_unstale":"2024-09-21T05:47:54.421499+0000","last_undegraded":"2024-09-21T05:47:54.421499+0000","last_fullsized":"2024-09-21T05:47:54.421499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:11.061652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:55.306521+0000","last_change":"2024-09-21T05:47:52.317943+0000","last_active":"2024-09-21T05:47:55.306521+0000","last_peered":"2024-09-21T05:47:55.306521+0000","last_clean":"2024-09-21T05:47:55.306521+0000","last_became_active":"2024-09-21T05:47:52.317606+0000","last_became_peered":"2024-09-21T05:47:52.317606+0000","last_unstale":"2024-09-21T05:47:55.306521+0000","last_undegraded":"2024-09-21T05:47:55.306521+0000","last_fullsized":"2024-09-21T05:47:55.306521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:53.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421251+0000","last_change":"2024-09-21T05:47:53.337568+0000","last_active":"2024-09-21T05:47:54.421251+0000","last_peered":"2024-09-21T05:47:54.421251+0000","last_clean":"2024-09-21T05:47:54.421251+0000","last_became_active":"2024-09-21T05:47:53.337068+0000","last_became_peered":"2024-09-21T05:47:53.337068+0000","last_unstale":"2024-09-21T05:47:54.421251+0000","last_undegraded":"2024-09-21T05:47:54.421251+0000","last_fullsized":"2024-09-21T05:47:54.421251+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:19.441038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421108+0000","last_change":"2024-09-21T05:47:52.325423+0000","last_active":"2024-09-21T05:47:54.421108+0000","last_peered":"2024-09-21T05:47:54.421108+0000","last_clean":"2024-09-21T05:47:54.421108+0000","last_became_active":"2024-09-21T05:47:52.325118+0000","last_became_peered":"2024-09-21T05:47:52.325118+0000","last_unstale":"2024-09-21T05:47:54.421108+0000","last_undegraded":"2024-09-21T05:47:54.421108+0000","last_fullsized":"2024-09-21T05:47:54.421108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:54.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323673+0000","last_change":"2024-09-21T05:47:53.329564+0000","last_active":"2024-09-21T05:47:54.323673+0000","last_peered":"2024-09-21T05:47:54.323673+0000","last_clean":"2024-09-21T05:47:54.323673+0000","last_became_active":"2024-09-21T05:47:53.329189+0000","last_became_peered":"2024-09-21T05:47:53.329189+0000","last_unstale":"2024-09-21T05:47:54.323673+0000","last_undegraded":"2024-09-21T05:47:54.323673+0000","last_fullsized":"2024-09-21T05:47:54.323673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:30.665406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420349+0000","last_change":"2024-09-21T05:47:52.317088+0000","last_active":"2024-09-21T05:47:54.420349+0000","last_peered":"2024-09-21T05:47:54.420349+0000","last_clean":"2024-09-21T05:47:54.420349+0000","last_became_active":"2024-09-21T05:47:52.316526+0000","last_became_peered":"2024-09-21T05:47:52.316526+0000","last_unstale":"2024-09-21T05:47:54.420349+0000","last_undegraded":"2024-09-21T05:47:54.420349+0000","last_fullsized":"2024-09-21T05:47:54.420349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:20.821196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322095+0000","last_change":"2024-09-21T05:47:53.327666+0000","last_active":"2024-09-21T05:47:54.322095+0000","last_peered":"2024-09-21T05:47:54.322095+0000","last_clean":"2024-09-21T05:47:54.322095+0000","last_became_active":"2024-09-21T05:47:53.327242+0000","last_became_peered":"2024-09-21T05:47:53.327242+0000","last_unstale":"2024-09-21T05:47:54.322095+0000","last_undegraded":"2024-09-21T05:47:54.322095+0000","last_fullsized":"2024-09-21T05:47:54.322095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:43.684698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419272+0000","last_change":"2024-09-21T05:47:52.314266+0000","last_active":"2024-09-21T05:47:54.419272+0000","last_peered":"2024-09-21T05:47:54.419272+0000","last_clean":"2024-09-21T05:47:54.419272+0000","last_became_active":"2024-09-21T05:47:52.313950+0000","last_became_peered":"2024-09-21T05:47:52.313950+0000","last_unstale":"2024-09-21T05:47:54.419272+0000","last_undegraded":"2024-09-21T05:47:54.419272+0000","last_fullsized":"2024-09-21T05:47:54.419272+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:54.199451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322027+0000","last_change":"2024-09-21T05:47:53.328445+0000","last_active":"2024-09-21T05:47:54.322027+0000","last_peered":"2024-09-21T05:47:54.322027+0000","last_clean":"2024-09-21T05:47:54.322027+0000","last_became_active":"2024-09-21T05:47:53.328071+0000","last_became_peered":"2024-09-21T05:47:53.328071+0000","last_unstale":"2024-09-21T05:47:54.322027+0000","last_undegraded":"2024-09-21T05:47:54.322027+0000","last_fullsized":"2024-09-21T05:47:54.322027+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:34.151456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321715+0000","last_change":"2024-09-21T05:47:52.312743+0000","last_active":"2024-09-21T05:47:54.321715+0000","last_peered":"2024-09-21T05:47:54.321715+0000","last_clean":"2024-09-21T05:47:54.321715+0000","last_became_active":"2024-09-21T05:47:52.312378+0000","last_became_peered":"2024-09-21T05:47:52.312378+0000","last_unstale":"2024-09-21T05:47:54.321715+0000","last_undegraded":"2024-09-21T05:47:54.321715+0000","last_fullsized":"2024-09-21T05:47:54.321715+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.556704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419782+0000","last_change":"2024-09-21T05:47:53.332432+0000","last_active":"2024-09-21T05:47:54.419782+0000","last_peered":"2024-09-21T05:47:54.419782+0000","last_clean":"2024-09-21T05:47:54.419782+0000","last_became_active":"2024-09-21T05:47:53.331881+0000","last_became_peered":"2024-09-21T05:47:53.331881+0000","last_unstale":"2024-09-21T05:47:54.419782+0000","last_undegraded":"2024-09-21T05:47:54.419782+0000","last_fullsized":"2024-09-21T05:47:54.419782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:21.407811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319732+0000","last_change":"2024-09-21T05:47:52.319216+0000","last_active":"2024-09-21T05:47:54.319732+0000","last_peered":"2024-09-21T05:47:54.319732+0000","last_clean":"2024-09-21T05:47:54.319732+0000","last_became_active":"2024-09-21T05:47:52.318756+0000","last_became_peered":"2024-09-21T05:47:52.318756+0000","last_unstale":"2024-09-21T05:47:54.319732+0000","last_undegraded":"2024-09-21T05:47:54.319732+0000","last_fullsized":"2024-09-21T05:47:54.319732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:12.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420865+0000","last_change":"2024-09-21T05:47:53.337719+0000","last_active":"2024-09-21T05:47:54.420865+0000","last_peered":"2024-09-21T05:47:54.420865+0000","last_clean":"2024-09-21T05:47:54.420865+0000","last_became_active":"2024-09-21T05:47:53.329968+0000","last_became_peered":"2024-09-21T05:47:53.329968+0000","last_unstale":"2024-09-21T05:47:54.420865+0000","last_undegraded":"2024-09-21T05:47:54.420865+0000","last_fullsized":"2024-09-21T05:47:54.420865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:11.856321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420792+0000","last_change":"2024-09-21T05:47:52.324337+0000","last_active":"2024-09-21T05:47:54.420792+0000","last_peered":"2024-09-21T05:47:54.420792+0000","last_clean":"2024-09-21T05:47:54.420792+0000","last_became_active":"2024-09-21T05:47:52.324032+0000","last_became_peered":"2024-09-21T05:47:52.324032+0000","last_unstale":"2024-09-21T05:47:54.420792+0000","last_undegraded":"2024-09-21T05:47:54.420792+0000","last_fullsized":"2024-09-21T05:47:54.420792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:03.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419235+0000","last_change":"2024-09-21T05:47:53.319715+0000","last_active":"2024-09-21T05:47:54.419235+0000","last_peered":"2024-09-21T05:47:54.419235+0000","last_clean":"2024-09-21T05:47:54.419235+0000","last_became_active":"2024-09-21T05:47:53.319394+0000","last_became_peered":"2024-09-21T05:47:53.319394+0000","last_unstale":"2024-09-21T05:47:54.419235+0000","last_undegraded":"2024-09-21T05:47:54.419235+0000","last_fullsized":"2024-09-21T05:47:54.419235+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:24.432342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419478+0000","last_change":"2024-09-21T05:47:52.316016+0000","last_active":"2024-09-21T05:47:54.419478+0000","last_peered":"2024-09-21T05:47:54.419478+0000","last_clean":"2024-09-21T05:47:54.419478+0000","last_became_active":"2024-09-21T05:47:52.315661+0000","last_became_peered":"2024-09-21T05:47:52.315661+0000","last_unstale":"2024-09-21T05:47:54.419478+0000","last_undegraded":"2024-09-21T05:47:54.419478+0000","last_fullsized":"2024-09-21T05:47:54.419478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:23.229664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":271,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419782+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:47:54.419782+0000","last_peered":"2024-09-21T05:47:54.419782+0000","last_clean":"2024-09-21T05:47:54.419782+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:47:54.419782+0000","last_undegraded":"2024-09-21T05:47:54.419782+0000","last_fullsized":"2024-09-21T05:47:54.419782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509549,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1174939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1351680,"data_stored":584025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1355776,"data_stored":584559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1363968,"data_stored":585637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40840,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331000,"statfs":{"total":96636764160,"available":96594944000,"internally_reserved":0,"allocated":1355776,"data_stored":584079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542252,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":1176047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:47:58.143 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T05:47:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.292+0000 7f566ce4d640 1 -- 172.21.15.22:0/3623491036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56681600a0 msgr2=0x7f5668071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.292+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3623491036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56681600a0 0x7f5668071250 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f5668070f40 tx=0x7f565802a5f0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 -- 172.21.15.22:0/3623491036 shutdown_connections 2024-09-21T05:47:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3623491036 >> v2:172.21.15.134:3300/0 conn(0x7f56680ce7d0 0x7f566806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3623491036 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 0x7f566806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3623491036 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56681600a0 0x7f5668071250 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 -- 172.21.15.22:0/3623491036 >> 172.21.15.22:0/3623491036 conn(0x7f56680cdcb0 msgr2=0x7f56680ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 -- 172.21.15.22:0/3623491036 shutdown_connections 2024-09-21T05:47:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.293+0000 7f566ce4d640 1 -- 172.21.15.22:0/3623491036 wait complete. 2024-09-21T05:47:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.295+0000 7f566ce4d640 1 Processor -- start 2024-09-21T05:47:58.296 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.295+0000 7f566ce4d640 1 -- start start 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566ce4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 0x7f5668154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566ce4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 0x7f566809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566ce4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56681600a0 0x7f56681f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566ce4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f56680b3040 con 0x7f56680ce7d0 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566ce4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f56680b2ec0 con 0x7f56681600a0 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566ce4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f56680b31c0 con 0x7f5668070360 2024-09-21T05:47:58.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 0x7f566809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 0x7f566809aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50590/0 (socket says 172.21.15.22:50590) 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 -- 172.21.15.22:0/3088849190 learned_addr learned my addr 172.21.15.22:0/3088849190 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5666d9d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56681600a0 0x7f56681f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566659c640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 0x7f5668154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 -- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 msgr2=0x7f5668154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 0x7f5668154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 -- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56681600a0 msgr2=0x7f56681f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56681600a0 0x7f56681f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f5665d9b640 1 -- 172.21.15.22:0/3088849190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56680cb880 con 0x7f56680ce7d0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.296+0000 7f566659c640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 0x7f5668154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.297+0000 7f5665d9b640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 0x7f566809aa30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b470 tx=0x7f565c00b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.297+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c015020 con 0x7f56680ce7d0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.297+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f565c0119a0 con 0x7f56680ce7d0 2024-09-21T05:47:58.300 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.297+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5668154fe0 con 0x7f56680ce7d0 2024-09-21T05:47:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.297+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c011df0 con 0x7f56680ce7d0 2024-09-21T05:47:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.297+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566815be80 con 0x7f56680ce7d0 2024-09-21T05:47:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.298+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566815bbf0 con 0x7f56680ce7d0 2024-09-21T05:47:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.299+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f565c018020 con 0x7f56680ce7d0 2024-09-21T05:47:58.301 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.299+0000 7f5656ffd640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5648078b90 0x7f564807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.300+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f565c09adc0 con 0x7f56680ce7d0 2024-09-21T05:47:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.300+0000 7f566659c640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5648078b90 0x7f564807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:58.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.300+0000 7f566659c640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5648078b90 0x7f564807b0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5658033000 tx=0x7f5658004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:58.306 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.304+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f565c01f020 con 0x7f56680ce7d0 2024-09-21T05:47:58.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.569+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5668071250 con 0x7f56680ce7d0 2024-09-21T05:47:58.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.570+0000 7f5656ffd640 1 -- 172.21.15.22:0/3088849190 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1058 (secure 0 0 0) 0x7f565c00d040 con 0x7f56680ce7d0 2024-09-21T05:47:58.572 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:58.572 INFO:teuthology.orchestra.run.smithi022.stdout:{"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":541,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":109,"num_osds":8,"num_up_osds":8,"osd_up_since":1726897150,"num_in_osds":8,"osd_in_since":1726897140,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":288870400,"bytes_avail":772805242880,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":101,"btime":"2024-09-21T05:47:56:326350+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8055}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-09-21T05:47:37.873184+0000","services":{}},"progress_events":{}} 2024-09-21T05:47:58.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5648078b90 msgr2=0x7f564807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5648078b90 0x7f564807b0a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5658033000 tx=0x7f5658004730 comp rx=0 tx=0).stop 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 msgr2=0x7f566809aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 0x7f566809aa30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b470 tx=0x7f565c00b930 comp rx=0 tx=0).stop 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 shutdown_connections 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f56681600a0 0x7f56681f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f56680ce7d0 0x7f566809aa30 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5648078b90 0x7f564807b0a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 --2- 172.21.15.22:0/3088849190 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5668070360 0x7f5668154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 >> 172.21.15.22:0/3088849190 conn(0x7f56680cdcb0 msgr2=0x7f56680cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 shutdown_connections 2024-09-21T05:47:58.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.572+0000 7f566ce4d640 1 -- 172.21.15.22:0/3088849190 wait complete. 2024-09-21T05:47:58.586 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T05:47:58.586 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:47:58.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149f077640 1 -- 172.21.15.22:0/3855355478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480004d40 msgr2=0x7f1480005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149f077640 1 --2- 172.21.15.22:0/3855355478 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f1480004d40 0x7f1480005150 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1490005fd0 tx=0x7f1480005830 comp rx=0 tx=0).stop 2024-09-21T05:47:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149d614640 1 -- 172.21.15.22:0/3855355478 reap_dead start 2024-09-21T05:47:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149f077640 1 -- 172.21.15.22:0/3855355478 shutdown_connections 2024-09-21T05:47:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149f077640 1 -- 172.21.15.22:0/3855355478 >> 172.21.15.22:0/3855355478 conn(0x7f149809d840 msgr2=0x7f149808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149f077640 1 -- 172.21.15.22:0/3855355478 shutdown_connections 2024-09-21T05:47:58.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.701+0000 7f149f077640 1 -- 172.21.15.22:0/3855355478 wait complete. 2024-09-21T05:47:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.703+0000 7f149f077640 1 Processor -- start 2024-09-21T05:47:58.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.703+0000 7f149f077640 1 -- start start 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149f077640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 0x7f14980b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149f077640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f149815fa50 0x7f149815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149f077640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f149815fe60 0x7f14980b7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149f077640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f14980d15f0 con 0x7f14981f9760 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149f077640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f14980d1470 con 0x7f149815fa50 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149f077640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f14980ba040 con 0x7f149815fe60 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.704+0000 7f149ce13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 0x7f14980b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 0x7f14980b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50604/0 (socket says 172.21.15.22:50604) 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 -- 172.21.15.22:0/1758612578 learned_addr learned my addr 172.21.15.22:0/1758612578 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 -- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f149815fe60 msgr2=0x7f14980b7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f149815fe60 0x7f14980b7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 -- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f149815fa50 msgr2=0x7f149815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f149815fa50 0x7f149815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 -- 172.21.15.22:0/1758612578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14980b2580 con 0x7f14981f9760 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149ce13640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 0x7f14980b1de0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1494011a60 tx=0x7f1494011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149401e070 con 0x7f14981f9760 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f149400fc70 con 0x7f14981f9760 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149815ef90 con 0x7f14981f9760 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1494019410 con 0x7f14981f9760 2024-09-21T05:47:58.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.705+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149815f550 con 0x7f14981f9760 2024-09-21T05:47:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.706+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14980b97e0 con 0x7f14981f9760 2024-09-21T05:47:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.707+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1494021070 con 0x7f14981f9760 2024-09-21T05:47:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.708+0000 7f148d7fa640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1484078b90 0x7f148407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:47:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.708+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f149409b490 con 0x7f14981f9760 2024-09-21T05:47:58.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.708+0000 7f148ffff640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1484078b90 0x7f148407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:47:58.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.709+0000 7f148ffff640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1484078b90 0x7f148407b0a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14980a21c0 tx=0x7f1488001060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:47:58.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.713+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14940633a0 con 0x7f14981f9760 2024-09-21T05:47:58.898 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.896+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1498069a50 con 0x7f1484078b90 2024-09-21T05:47:58.903 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.901+0000 7f148d7fa640 1 -- 172.21.15.22:0/1758612578 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f1498069a50 con 0x7f1484078b90 2024-09-21T05:47:58.906 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:47:58.907 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:47:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1484078b90 msgr2=0x7f148407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.908 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1484078b90 0x7f148407b0a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f14980a21c0 tx=0x7f1488001060 comp rx=0 tx=0).stop 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 msgr2=0x7f14980b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 0x7f14980b1de0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f1494011a60 tx=0x7f1494011f20 comp rx=0 tx=0).stop 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 shutdown_connections 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1484078b90 0x7f148407b0a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f149815fe60 0x7f14980b7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f149815fa50 0x7f149815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 --2- 172.21.15.22:0/1758612578 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f14981f9760 0x7f14980b1de0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 >> 172.21.15.22:0/1758612578 conn(0x7f149809d840 msgr2=0x7f149806e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 shutdown_connections 2024-09-21T05:47:58.909 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:47:58.904+0000 7f149f077640 1 -- 172.21.15.22:0/1758612578 wait complete. 2024-09-21T05:47:58.917 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-21T05:47:57.878028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282100,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754692620,"statfs":{"total":773094113280,"available":772805242880,"internally_reserved":0,"allocated":12034048,"data_stored":5857196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001519"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421407+0000","last_change":"2024-09-21T05:47:53.329400+0000","last_active":"2024-09-21T05:47:54.421407+0000","last_peered":"2024-09-21T05:47:54.421407+0000","last_clean":"2024-09-21T05:47:54.421407+0000","last_became_active":"2024-09-21T05:47:53.328865+0000","last_became_peered":"2024-09-21T05:47:53.328865+0000","last_unstale":"2024-09-21T05:47:54.421407+0000","last_undegraded":"2024-09-21T05:47:54.421407+0000","last_fullsized":"2024-09-21T05:47:54.421407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:33.666746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322207+0000","last_change":"2024-09-21T05:47:53.315211+0000","last_active":"2024-09-21T05:47:54.322207+0000","last_peered":"2024-09-21T05:47:54.322207+0000","last_clean":"2024-09-21T05:47:54.322207+0000","last_became_active":"2024-09-21T05:47:53.314268+0000","last_became_peered":"2024-09-21T05:47:53.314268+0000","last_unstale":"2024-09-21T05:47:54.322207+0000","last_undegraded":"2024-09-21T05:47:54.322207+0000","last_fullsized":"2024-09-21T05:47:54.322207+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:15.893666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420190+0000","last_change":"2024-09-21T05:47:53.322252+0000","last_active":"2024-09-21T05:47:54.420190+0000","last_peered":"2024-09-21T05:47:54.420190+0000","last_clean":"2024-09-21T05:47:54.420190+0000","last_became_active":"2024-09-21T05:47:53.321925+0000","last_became_peered":"2024-09-21T05:47:53.321925+0000","last_unstale":"2024-09-21T05:47:54.420190+0000","last_undegraded":"2024-09-21T05:47:54.420190+0000","last_fullsized":"2024-09-21T05:47:54.420190+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:11.917317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420973+0000","last_change":"2024-09-21T05:47:53.338050+0000","last_active":"2024-09-21T05:47:54.420973+0000","last_peered":"2024-09-21T05:47:54.420973+0000","last_clean":"2024-09-21T05:47:54.420973+0000","last_became_active":"2024-09-21T05:47:53.337279+0000","last_became_peered":"2024-09-21T05:47:53.337279+0000","last_unstale":"2024-09-21T05:47:54.420973+0000","last_undegraded":"2024-09-21T05:47:54.420973+0000","last_fullsized":"2024-09-21T05:47:54.420973+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:49.343256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320073+0000","last_change":"2024-09-21T05:47:53.330191+0000","last_active":"2024-09-21T05:47:54.320073+0000","last_peered":"2024-09-21T05:47:54.320073+0000","last_clean":"2024-09-21T05:47:54.320073+0000","last_became_active":"2024-09-21T05:47:53.329578+0000","last_became_peered":"2024-09-21T05:47:53.329578+0000","last_unstale":"2024-09-21T05:47:54.320073+0000","last_undegraded":"2024-09-21T05:47:54.320073+0000","last_fullsized":"2024-09-21T05:47:54.320073+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:51.237035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420158+0000","last_change":"2024-09-21T05:47:53.314749+0000","last_active":"2024-09-21T05:47:54.420158+0000","last_peered":"2024-09-21T05:47:54.420158+0000","last_clean":"2024-09-21T05:47:54.420158+0000","last_became_active":"2024-09-21T05:47:53.314457+0000","last_became_peered":"2024-09-21T05:47:53.314457+0000","last_unstale":"2024-09-21T05:47:54.420158+0000","last_undegraded":"2024-09-21T05:47:54.420158+0000","last_fullsized":"2024-09-21T05:47:54.420158+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:39.746882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419166+0000","last_change":"2024-09-21T05:47:53.322397+0000","last_active":"2024-09-21T05:47:54.419166+0000","last_peered":"2024-09-21T05:47:54.419166+0000","last_clean":"2024-09-21T05:47:54.419166+0000","last_became_active":"2024-09-21T05:47:53.322048+0000","last_became_peered":"2024-09-21T05:47:53.322048+0000","last_unstale":"2024-09-21T05:47:54.419166+0000","last_undegraded":"2024-09-21T05:47:54.419166+0000","last_fullsized":"2024-09-21T05:47:54.419166+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:37.914628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.323792+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.323271+0000","last_became_peered":"2024-09-21T05:47:53.323271+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:22.199819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421377+0000","last_change":"2024-09-21T05:47:53.333144+0000","last_active":"2024-09-21T05:47:54.421377+0000","last_peered":"2024-09-21T05:47:54.421377+0000","last_clean":"2024-09-21T05:47:54.421377+0000","last_became_active":"2024-09-21T05:47:53.332816+0000","last_became_peered":"2024-09-21T05:47:53.332816+0000","last_unstale":"2024-09-21T05:47:54.421377+0000","last_undegraded":"2024-09-21T05:47:54.421377+0000","last_fullsized":"2024-09-21T05:47:54.421377+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:44.276704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421287+0000","last_change":"2024-09-21T05:47:53.328426+0000","last_active":"2024-09-21T05:47:54.421287+0000","last_peered":"2024-09-21T05:47:54.421287+0000","last_clean":"2024-09-21T05:47:54.421287+0000","last_became_active":"2024-09-21T05:47:53.327992+0000","last_became_peered":"2024-09-21T05:47:53.327992+0000","last_unstale":"2024-09-21T05:47:54.421287+0000","last_undegraded":"2024-09-21T05:47:54.421287+0000","last_fullsized":"2024-09-21T05:47:54.421287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:02.621431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321989+0000","last_change":"2024-09-21T05:47:53.318301+0000","last_active":"2024-09-21T05:47:54.321989+0000","last_peered":"2024-09-21T05:47:54.321989+0000","last_clean":"2024-09-21T05:47:54.321989+0000","last_became_active":"2024-09-21T05:47:53.317941+0000","last_became_peered":"2024-09-21T05:47:53.317941+0000","last_unstale":"2024-09-21T05:47:54.321989+0000","last_undegraded":"2024-09-21T05:47:54.321989+0000","last_fullsized":"2024-09-21T05:47:54.321989+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421042+0000","last_change":"2024-09-21T05:47:53.338328+0000","last_active":"2024-09-21T05:47:54.421042+0000","last_peered":"2024-09-21T05:47:54.421042+0000","last_clean":"2024-09-21T05:47:54.421042+0000","last_became_active":"2024-09-21T05:47:53.337637+0000","last_became_peered":"2024-09-21T05:47:53.337637+0000","last_unstale":"2024-09-21T05:47:54.421042+0000","last_undegraded":"2024-09-21T05:47:54.421042+0000","last_fullsized":"2024-09-21T05:47:54.421042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:07.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420323+0000","last_change":"2024-09-21T05:47:53.329762+0000","last_active":"2024-09-21T05:47:54.420323+0000","last_peered":"2024-09-21T05:47:54.420323+0000","last_clean":"2024-09-21T05:47:54.420323+0000","last_became_active":"2024-09-21T05:47:53.329426+0000","last_became_peered":"2024-09-21T05:47:53.329426+0000","last_unstale":"2024-09-21T05:47:54.420323+0000","last_undegraded":"2024-09-21T05:47:54.420323+0000","last_fullsized":"2024-09-21T05:47:54.420323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:44.894421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419422+0000","last_change":"2024-09-21T05:47:53.328646+0000","last_active":"2024-09-21T05:47:54.419422+0000","last_peered":"2024-09-21T05:47:54.419422+0000","last_clean":"2024-09-21T05:47:54.419422+0000","last_became_active":"2024-09-21T05:47:53.328313+0000","last_became_peered":"2024-09-21T05:47:53.328313+0000","last_unstale":"2024-09-21T05:47:54.419422+0000","last_undegraded":"2024-09-21T05:47:54.419422+0000","last_fullsized":"2024-09-21T05:47:54.419422+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:14.951874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419472+0000","last_change":"2024-09-21T05:47:53.314013+0000","last_active":"2024-09-21T05:47:54.419472+0000","last_peered":"2024-09-21T05:47:54.419472+0000","last_clean":"2024-09-21T05:47:54.419472+0000","last_became_active":"2024-09-21T05:47:53.313613+0000","last_became_peered":"2024-09-21T05:47:53.313613+0000","last_unstale":"2024-09-21T05:47:54.419472+0000","last_undegraded":"2024-09-21T05:47:54.419472+0000","last_fullsized":"2024-09-21T05:47:54.419472+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:09.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420533+0000","last_change":"2024-09-21T05:47:53.340669+0000","last_active":"2024-09-21T05:47:54.420533+0000","last_peered":"2024-09-21T05:47:54.420533+0000","last_clean":"2024-09-21T05:47:54.420533+0000","last_became_active":"2024-09-21T05:47:53.340336+0000","last_became_peered":"2024-09-21T05:47:53.340336+0000","last_unstale":"2024-09-21T05:47:54.420533+0000","last_undegraded":"2024-09-21T05:47:54.420533+0000","last_fullsized":"2024-09-21T05:47:54.420533+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:28.758224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322315+0000","last_change":"2024-09-21T05:47:53.320885+0000","last_active":"2024-09-21T05:47:54.322315+0000","last_peered":"2024-09-21T05:47:54.322315+0000","last_clean":"2024-09-21T05:47:54.322315+0000","last_became_active":"2024-09-21T05:47:53.320175+0000","last_became_peered":"2024-09-21T05:47:53.320175+0000","last_unstale":"2024-09-21T05:47:54.322315+0000","last_undegraded":"2024-09-21T05:47:54.322315+0000","last_fullsized":"2024-09-21T05:47:54.322315+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:57.310115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323869+0000","last_change":"2024-09-21T05:47:53.317110+0000","last_active":"2024-09-21T05:47:54.323869+0000","last_peered":"2024-09-21T05:47:54.323869+0000","last_clean":"2024-09-21T05:47:54.323869+0000","last_became_active":"2024-09-21T05:47:53.316725+0000","last_became_peered":"2024-09-21T05:47:53.316725+0000","last_unstale":"2024-09-21T05:47:54.323869+0000","last_undegraded":"2024-09-21T05:47:54.323869+0000","last_fullsized":"2024-09-21T05:47:54.323869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.043120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420287+0000","last_change":"2024-09-21T05:47:53.332561+0000","last_active":"2024-09-21T05:47:54.420287+0000","last_peered":"2024-09-21T05:47:54.420287+0000","last_clean":"2024-09-21T05:47:54.420287+0000","last_became_active":"2024-09-21T05:47:53.331867+0000","last_became_peered":"2024-09-21T05:47:53.331867+0000","last_unstale":"2024-09-21T05:47:54.420287+0000","last_undegraded":"2024-09-21T05:47:54.420287+0000","last_fullsized":"2024-09-21T05:47:54.420287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:16.303376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322107+0000","last_change":"2024-09-21T05:47:53.314765+0000","last_active":"2024-09-21T05:47:54.322107+0000","last_peered":"2024-09-21T05:47:54.322107+0000","last_clean":"2024-09-21T05:47:54.322107+0000","last_became_active":"2024-09-21T05:47:53.314417+0000","last_became_peered":"2024-09-21T05:47:53.314417+0000","last_unstale":"2024-09-21T05:47:54.322107+0000","last_undegraded":"2024-09-21T05:47:54.322107+0000","last_fullsized":"2024-09-21T05:47:54.322107+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:44.328561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322342+0000","last_change":"2024-09-21T05:47:53.323870+0000","last_active":"2024-09-21T05:47:54.322342+0000","last_peered":"2024-09-21T05:47:54.322342+0000","last_clean":"2024-09-21T05:47:54.322342+0000","last_became_active":"2024-09-21T05:47:53.323552+0000","last_became_peered":"2024-09-21T05:47:53.323552+0000","last_unstale":"2024-09-21T05:47:54.322342+0000","last_undegraded":"2024-09-21T05:47:54.322342+0000","last_fullsized":"2024-09-21T05:47:54.322342+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:33.010490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419862+0000","last_change":"2024-09-21T05:47:53.322155+0000","last_active":"2024-09-21T05:47:54.419862+0000","last_peered":"2024-09-21T05:47:54.419862+0000","last_clean":"2024-09-21T05:47:54.419862+0000","last_became_active":"2024-09-21T05:47:53.321833+0000","last_became_peered":"2024-09-21T05:47:53.321833+0000","last_unstale":"2024-09-21T05:47:54.419862+0000","last_undegraded":"2024-09-21T05:47:54.419862+0000","last_fullsized":"2024-09-21T05:47:54.419862+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:01.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321454+0000","last_change":"2024-09-21T05:47:53.325437+0000","last_active":"2024-09-21T05:47:54.321454+0000","last_peered":"2024-09-21T05:47:54.321454+0000","last_clean":"2024-09-21T05:47:54.321454+0000","last_became_active":"2024-09-21T05:47:53.318729+0000","last_became_peered":"2024-09-21T05:47:53.318729+0000","last_unstale":"2024-09-21T05:47:54.321454+0000","last_undegraded":"2024-09-21T05:47:54.321454+0000","last_fullsized":"2024-09-21T05:47:54.321454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:23.462572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419203+0000","last_change":"2024-09-21T05:47:53.320400+0000","last_active":"2024-09-21T05:47:54.419203+0000","last_peered":"2024-09-21T05:47:54.419203+0000","last_clean":"2024-09-21T05:47:54.419203+0000","last_became_active":"2024-09-21T05:47:53.320056+0000","last_became_peered":"2024-09-21T05:47:53.320056+0000","last_unstale":"2024-09-21T05:47:54.419203+0000","last_undegraded":"2024-09-21T05:47:54.419203+0000","last_fullsized":"2024-09-21T05:47:54.419203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:51.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322541+0000","last_change":"2024-09-21T05:47:53.321337+0000","last_active":"2024-09-21T05:47:54.322541+0000","last_peered":"2024-09-21T05:47:54.322541+0000","last_clean":"2024-09-21T05:47:54.322541+0000","last_became_active":"2024-09-21T05:47:53.320583+0000","last_became_peered":"2024-09-21T05:47:53.320583+0000","last_unstale":"2024-09-21T05:47:54.322541+0000","last_undegraded":"2024-09-21T05:47:54.322541+0000","last_fullsized":"2024-09-21T05:47:54.322541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:57.315042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421261+0000","last_change":"2024-09-21T05:47:53.335796+0000","last_active":"2024-09-21T05:47:54.421261+0000","last_peered":"2024-09-21T05:47:54.421261+0000","last_clean":"2024-09-21T05:47:54.421261+0000","last_became_active":"2024-09-21T05:47:53.335426+0000","last_became_peered":"2024-09-21T05:47:53.335426+0000","last_unstale":"2024-09-21T05:47:54.421261+0000","last_undegraded":"2024-09-21T05:47:54.421261+0000","last_fullsized":"2024-09-21T05:47:54.421261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:32.715401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419441+0000","last_change":"2024-09-21T05:47:53.328721+0000","last_active":"2024-09-21T05:47:54.419441+0000","last_peered":"2024-09-21T05:47:54.419441+0000","last_clean":"2024-09-21T05:47:54.419441+0000","last_became_active":"2024-09-21T05:47:53.328431+0000","last_became_peered":"2024-09-21T05:47:53.328431+0000","last_unstale":"2024-09-21T05:47:54.419441+0000","last_undegraded":"2024-09-21T05:47:54.419441+0000","last_fullsized":"2024-09-21T05:47:54.419441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:16.400560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420065+0000","last_change":"2024-09-21T05:47:53.339459+0000","last_active":"2024-09-21T05:47:54.420065+0000","last_peered":"2024-09-21T05:47:54.420065+0000","last_clean":"2024-09-21T05:47:54.420065+0000","last_became_active":"2024-09-21T05:47:53.339054+0000","last_became_peered":"2024-09-21T05:47:53.339054+0000","last_unstale":"2024-09-21T05:47:54.420065+0000","last_undegraded":"2024-09-21T05:47:54.420065+0000","last_fullsized":"2024-09-21T05:47:54.420065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:16.576540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321766+0000","last_change":"2024-09-21T05:47:53.319552+0000","last_active":"2024-09-21T05:47:54.321766+0000","last_peered":"2024-09-21T05:47:54.321766+0000","last_clean":"2024-09-21T05:47:54.321766+0000","last_became_active":"2024-09-21T05:47:53.319101+0000","last_became_peered":"2024-09-21T05:47:53.319101+0000","last_unstale":"2024-09-21T05:47:54.321766+0000","last_undegraded":"2024-09-21T05:47:54.321766+0000","last_fullsized":"2024-09-21T05:47:54.321766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:44.232318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419434+0000","last_change":"2024-09-21T05:47:53.323957+0000","last_active":"2024-09-21T05:47:54.419434+0000","last_peered":"2024-09-21T05:47:54.419434+0000","last_clean":"2024-09-21T05:47:54.419434+0000","last_became_active":"2024-09-21T05:47:53.323432+0000","last_became_peered":"2024-09-21T05:47:53.323432+0000","last_unstale":"2024-09-21T05:47:54.419434+0000","last_undegraded":"2024-09-21T05:47:54.419434+0000","last_fullsized":"2024-09-21T05:47:54.419434+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:57.449767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420771+0000","last_change":"2024-09-21T05:47:53.340040+0000","last_active":"2024-09-21T05:47:54.420771+0000","last_peered":"2024-09-21T05:47:54.420771+0000","last_clean":"2024-09-21T05:47:54.420771+0000","last_became_active":"2024-09-21T05:47:53.339702+0000","last_became_peered":"2024-09-21T05:47:53.339702+0000","last_unstale":"2024-09-21T05:47:54.420771+0000","last_undegraded":"2024-09-21T05:47:54.420771+0000","last_fullsized":"2024-09-21T05:47:54.420771+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:02.133659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.320519+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.320182+0000","last_became_peered":"2024-09-21T05:47:53.320182+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:16.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419700+0000","last_change":"2024-09-21T05:47:53.327229+0000","last_active":"2024-09-21T05:47:54.419700+0000","last_peered":"2024-09-21T05:47:54.419700+0000","last_clean":"2024-09-21T05:47:54.419700+0000","last_became_active":"2024-09-21T05:47:53.326881+0000","last_became_peered":"2024-09-21T05:47:53.326881+0000","last_unstale":"2024-09-21T05:47:54.419700+0000","last_undegraded":"2024-09-21T05:47:54.419700+0000","last_fullsized":"2024-09-21T05:47:54.419700+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:43.659296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":19,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419430+0000","last_change":"2024-09-21T05:47:52.315229+0000","last_active":"2024-09-21T05:47:54.419430+0000","last_peered":"2024-09-21T05:47:54.419430+0000","last_clean":"2024-09-21T05:47:54.419430+0000","last_became_active":"2024-09-21T05:47:52.314905+0000","last_became_peered":"2024-09-21T05:47:52.314905+0000","last_unstale":"2024-09-21T05:47:54.419430+0000","last_undegraded":"2024-09-21T05:47:54.419430+0000","last_fullsized":"2024-09-21T05:47:54.419430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:00.549432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323756+0000","last_change":"2024-09-21T05:47:53.325534+0000","last_active":"2024-09-21T05:47:54.323756+0000","last_peered":"2024-09-21T05:47:54.323756+0000","last_clean":"2024-09-21T05:47:54.323756+0000","last_became_active":"2024-09-21T05:47:53.325200+0000","last_became_peered":"2024-09-21T05:47:53.325200+0000","last_unstale":"2024-09-21T05:47:54.323756+0000","last_undegraded":"2024-09-21T05:47:54.323756+0000","last_fullsized":"2024-09-21T05:47:54.323756+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:49.932618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420018+0000","last_change":"2024-09-21T05:47:52.317856+0000","last_active":"2024-09-21T05:47:54.420018+0000","last_peered":"2024-09-21T05:47:54.420018+0000","last_clean":"2024-09-21T05:47:54.420018+0000","last_became_active":"2024-09-21T05:47:52.317462+0000","last_became_peered":"2024-09-21T05:47:52.317462+0000","last_unstale":"2024-09-21T05:47:54.420018+0000","last_undegraded":"2024-09-21T05:47:54.420018+0000","last_fullsized":"2024-09-21T05:47:54.420018+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:58.602683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420098+0000","last_change":"2024-09-21T05:47:53.314734+0000","last_active":"2024-09-21T05:47:54.420098+0000","last_peered":"2024-09-21T05:47:54.420098+0000","last_clean":"2024-09-21T05:47:54.420098+0000","last_became_active":"2024-09-21T05:47:53.314373+0000","last_became_peered":"2024-09-21T05:47:53.314373+0000","last_unstale":"2024-09-21T05:47:54.420098+0000","last_undegraded":"2024-09-21T05:47:54.420098+0000","last_fullsized":"2024-09-21T05:47:54.420098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:32.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319979+0000","last_change":"2024-09-21T05:47:52.320521+0000","last_active":"2024-09-21T05:47:54.319979+0000","last_peered":"2024-09-21T05:47:54.319979+0000","last_clean":"2024-09-21T05:47:54.319979+0000","last_became_active":"2024-09-21T05:47:52.320077+0000","last_became_peered":"2024-09-21T05:47:52.320077+0000","last_unstale":"2024-09-21T05:47:54.319979+0000","last_undegraded":"2024-09-21T05:47:54.319979+0000","last_fullsized":"2024-09-21T05:47:54.319979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:11.737474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420363+0000","last_change":"2024-09-21T05:47:53.329736+0000","last_active":"2024-09-21T05:47:54.420363+0000","last_peered":"2024-09-21T05:47:54.420363+0000","last_clean":"2024-09-21T05:47:54.420363+0000","last_became_active":"2024-09-21T05:47:53.329390+0000","last_became_peered":"2024-09-21T05:47:53.329390+0000","last_unstale":"2024-09-21T05:47:54.420363+0000","last_undegraded":"2024-09-21T05:47:54.420363+0000","last_fullsized":"2024-09-21T05:47:54.420363+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:27.003776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319826+0000","last_change":"2024-09-21T05:47:52.321048+0000","last_active":"2024-09-21T05:47:54.319826+0000","last_peered":"2024-09-21T05:47:54.319826+0000","last_clean":"2024-09-21T05:47:54.319826+0000","last_became_active":"2024-09-21T05:47:52.320701+0000","last_became_peered":"2024-09-21T05:47:52.320701+0000","last_unstale":"2024-09-21T05:47:54.319826+0000","last_undegraded":"2024-09-21T05:47:54.319826+0000","last_fullsized":"2024-09-21T05:47:54.319826+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:30.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420672+0000","last_change":"2024-09-21T05:47:53.336978+0000","last_active":"2024-09-21T05:47:54.420672+0000","last_peered":"2024-09-21T05:47:54.420672+0000","last_clean":"2024-09-21T05:47:54.420672+0000","last_became_active":"2024-09-21T05:47:53.336304+0000","last_became_peered":"2024-09-21T05:47:53.336304+0000","last_unstale":"2024-09-21T05:47:54.420672+0000","last_undegraded":"2024-09-21T05:47:54.420672+0000","last_fullsized":"2024-09-21T05:47:54.420672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:23.474145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419148+0000","last_change":"2024-09-21T05:47:52.316047+0000","last_active":"2024-09-21T05:47:54.419148+0000","last_peered":"2024-09-21T05:47:54.419148+0000","last_clean":"2024-09-21T05:47:54.419148+0000","last_became_active":"2024-09-21T05:47:52.315653+0000","last_became_peered":"2024-09-21T05:47:52.315653+0000","last_unstale":"2024-09-21T05:47:54.419148+0000","last_undegraded":"2024-09-21T05:47:54.419148+0000","last_fullsized":"2024-09-21T05:47:54.419148+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:29.936206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321828+0000","last_change":"2024-09-21T05:47:53.321821+0000","last_active":"2024-09-21T05:47:54.321828+0000","last_peered":"2024-09-21T05:47:54.321828+0000","last_clean":"2024-09-21T05:47:54.321828+0000","last_became_active":"2024-09-21T05:47:53.321524+0000","last_became_peered":"2024-09-21T05:47:53.321524+0000","last_unstale":"2024-09-21T05:47:54.321828+0000","last_undegraded":"2024-09-21T05:47:54.321828+0000","last_fullsized":"2024-09-21T05:47:54.321828+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:49.030281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420062+0000","last_change":"2024-09-21T05:47:52.315884+0000","last_active":"2024-09-21T05:47:54.420062+0000","last_peered":"2024-09-21T05:47:54.420062+0000","last_clean":"2024-09-21T05:47:54.420062+0000","last_became_active":"2024-09-21T05:47:52.315492+0000","last_became_peered":"2024-09-21T05:47:52.315492+0000","last_unstale":"2024-09-21T05:47:54.420062+0000","last_undegraded":"2024-09-21T05:47:54.420062+0000","last_fullsized":"2024-09-21T05:47:54.420062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:47.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420539+0000","last_change":"2024-09-21T05:47:53.339451+0000","last_active":"2024-09-21T05:47:54.420539+0000","last_peered":"2024-09-21T05:47:54.420539+0000","last_clean":"2024-09-21T05:47:54.420539+0000","last_became_active":"2024-09-21T05:47:53.339042+0000","last_became_peered":"2024-09-21T05:47:53.339042+0000","last_unstale":"2024-09-21T05:47:54.420539+0000","last_undegraded":"2024-09-21T05:47:54.420539+0000","last_fullsized":"2024-09-21T05:47:54.420539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:24.227545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321401+0000","last_change":"2024-09-21T05:47:52.322566+0000","last_active":"2024-09-21T05:47:54.321401+0000","last_peered":"2024-09-21T05:47:54.321401+0000","last_clean":"2024-09-21T05:47:54.321401+0000","last_became_active":"2024-09-21T05:47:52.322294+0000","last_became_peered":"2024-09-21T05:47:52.322294+0000","last_unstale":"2024-09-21T05:47:54.321401+0000","last_undegraded":"2024-09-21T05:47:54.321401+0000","last_fullsized":"2024-09-21T05:47:54.321401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:33.895532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419785+0000","last_change":"2024-09-21T05:47:53.324736+0000","last_active":"2024-09-21T05:47:54.419785+0000","last_peered":"2024-09-21T05:47:54.419785+0000","last_clean":"2024-09-21T05:47:54.419785+0000","last_became_active":"2024-09-21T05:47:53.324398+0000","last_became_peered":"2024-09-21T05:47:53.324398+0000","last_unstale":"2024-09-21T05:47:54.419785+0000","last_undegraded":"2024-09-21T05:47:54.419785+0000","last_fullsized":"2024-09-21T05:47:54.419785+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:14.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420656+0000","last_change":"2024-09-21T05:47:52.315887+0000","last_active":"2024-09-21T05:47:54.420656+0000","last_peered":"2024-09-21T05:47:54.420656+0000","last_clean":"2024-09-21T05:47:54.420656+0000","last_became_active":"2024-09-21T05:47:52.315504+0000","last_became_peered":"2024-09-21T05:47:52.315504+0000","last_unstale":"2024-09-21T05:47:54.420656+0000","last_undegraded":"2024-09-21T05:47:54.420656+0000","last_fullsized":"2024-09-21T05:47:54.420656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:57.957523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419923+0000","last_change":"2024-09-21T05:47:53.329242+0000","last_active":"2024-09-21T05:47:54.419923+0000","last_peered":"2024-09-21T05:47:54.419923+0000","last_clean":"2024-09-21T05:47:54.419923+0000","last_became_active":"2024-09-21T05:47:53.327266+0000","last_became_peered":"2024-09-21T05:47:53.327266+0000","last_unstale":"2024-09-21T05:47:54.419923+0000","last_undegraded":"2024-09-21T05:47:54.419923+0000","last_fullsized":"2024-09-21T05:47:54.419923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:53.808702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320068+0000","last_change":"2024-09-21T05:47:52.318716+0000","last_active":"2024-09-21T05:47:54.320068+0000","last_peered":"2024-09-21T05:47:54.320068+0000","last_clean":"2024-09-21T05:47:54.320068+0000","last_became_active":"2024-09-21T05:47:52.318376+0000","last_became_peered":"2024-09-21T05:47:52.318376+0000","last_unstale":"2024-09-21T05:47:54.320068+0000","last_undegraded":"2024-09-21T05:47:54.320068+0000","last_fullsized":"2024-09-21T05:47:54.320068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:36.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323810+0000","last_change":"2024-09-21T05:47:53.330894+0000","last_active":"2024-09-21T05:47:54.323810+0000","last_peered":"2024-09-21T05:47:54.323810+0000","last_clean":"2024-09-21T05:47:54.323810+0000","last_became_active":"2024-09-21T05:47:53.330525+0000","last_became_peered":"2024-09-21T05:47:53.330525+0000","last_unstale":"2024-09-21T05:47:54.323810+0000","last_undegraded":"2024-09-21T05:47:54.323810+0000","last_fullsized":"2024-09-21T05:47:54.323810+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:53.569567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419648+0000","last_change":"2024-09-21T05:47:52.315933+0000","last_active":"2024-09-21T05:47:54.419648+0000","last_peered":"2024-09-21T05:47:54.419648+0000","last_clean":"2024-09-21T05:47:54.419648+0000","last_became_active":"2024-09-21T05:47:52.315578+0000","last_became_peered":"2024-09-21T05:47:52.315578+0000","last_unstale":"2024-09-21T05:47:54.419648+0000","last_undegraded":"2024-09-21T05:47:54.419648+0000","last_fullsized":"2024-09-21T05:47:54.419648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:07.116156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420303+0000","last_change":"2024-09-21T05:47:53.320523+0000","last_active":"2024-09-21T05:47:54.420303+0000","last_peered":"2024-09-21T05:47:54.420303+0000","last_clean":"2024-09-21T05:47:54.420303+0000","last_became_active":"2024-09-21T05:47:53.320183+0000","last_became_peered":"2024-09-21T05:47:53.320183+0000","last_unstale":"2024-09-21T05:47:54.420303+0000","last_undegraded":"2024-09-21T05:47:54.420303+0000","last_fullsized":"2024-09-21T05:47:54.420303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:58.959425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421035+0000","last_change":"2024-09-21T05:47:52.322650+0000","last_active":"2024-09-21T05:47:54.421035+0000","last_peered":"2024-09-21T05:47:54.421035+0000","last_clean":"2024-09-21T05:47:54.421035+0000","last_became_active":"2024-09-21T05:47:52.322310+0000","last_became_peered":"2024-09-21T05:47:52.322310+0000","last_unstale":"2024-09-21T05:47:54.421035+0000","last_undegraded":"2024-09-21T05:47:54.421035+0000","last_fullsized":"2024-09-21T05:47:54.421035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:23.430441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419420+0000","last_change":"2024-09-21T05:47:53.326552+0000","last_active":"2024-09-21T05:47:54.419420+0000","last_peered":"2024-09-21T05:47:54.419420+0000","last_clean":"2024-09-21T05:47:54.419420+0000","last_became_active":"2024-09-21T05:47:53.326200+0000","last_became_peered":"2024-09-21T05:47:53.326200+0000","last_unstale":"2024-09-21T05:47:54.419420+0000","last_undegraded":"2024-09-21T05:47:54.419420+0000","last_fullsized":"2024-09-21T05:47:54.419420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:13.011382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":21,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421178+0000","last_change":"2024-09-21T05:47:52.324477+0000","last_active":"2024-09-21T05:47:54.421178+0000","last_peered":"2024-09-21T05:47:54.421178+0000","last_clean":"2024-09-21T05:47:54.421178+0000","last_became_active":"2024-09-21T05:47:52.324166+0000","last_became_peered":"2024-09-21T05:47:52.324166+0000","last_unstale":"2024-09-21T05:47:54.421178+0000","last_undegraded":"2024-09-21T05:47:54.421178+0000","last_fullsized":"2024-09-21T05:47:54.421178+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:00.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419994+0000","last_change":"2024-09-21T05:47:53.326873+0000","last_active":"2024-09-21T05:47:54.419994+0000","last_peered":"2024-09-21T05:47:54.419994+0000","last_clean":"2024-09-21T05:47:54.419994+0000","last_became_active":"2024-09-21T05:47:53.326548+0000","last_became_peered":"2024-09-21T05:47:53.326548+0000","last_unstale":"2024-09-21T05:47:54.419994+0000","last_undegraded":"2024-09-21T05:47:54.419994+0000","last_fullsized":"2024-09-21T05:47:54.419994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:34.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419775+0000","last_change":"2024-09-21T05:47:52.316414+0000","last_active":"2024-09-21T05:47:54.419775+0000","last_peered":"2024-09-21T05:47:54.419775+0000","last_clean":"2024-09-21T05:47:54.419775+0000","last_became_active":"2024-09-21T05:47:52.316075+0000","last_became_peered":"2024-09-21T05:47:52.316075+0000","last_unstale":"2024-09-21T05:47:54.419775+0000","last_undegraded":"2024-09-21T05:47:54.419775+0000","last_fullsized":"2024-09-21T05:47:54.419775+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:58.423946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419724+0000","last_change":"2024-09-21T05:47:53.331201+0000","last_active":"2024-09-21T05:47:54.419724+0000","last_peered":"2024-09-21T05:47:54.419724+0000","last_clean":"2024-09-21T05:47:54.419724+0000","last_became_active":"2024-09-21T05:47:53.330540+0000","last_became_peered":"2024-09-21T05:47:53.330540+0000","last_unstale":"2024-09-21T05:47:54.419724+0000","last_undegraded":"2024-09-21T05:47:54.419724+0000","last_fullsized":"2024-09-21T05:47:54.419724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:19.803831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323431+0000","last_change":"2024-09-21T05:47:52.327286+0000","last_active":"2024-09-21T05:47:54.323431+0000","last_peered":"2024-09-21T05:47:54.323431+0000","last_clean":"2024-09-21T05:47:54.323431+0000","last_became_active":"2024-09-21T05:47:52.326918+0000","last_became_peered":"2024-09-21T05:47:52.326918+0000","last_unstale":"2024-09-21T05:47:54.323431+0000","last_undegraded":"2024-09-21T05:47:54.323431+0000","last_fullsized":"2024-09-21T05:47:54.323431+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:21.234490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419301+0000","last_change":"2024-09-21T05:47:53.319222+0000","last_active":"2024-09-21T05:47:54.419301+0000","last_peered":"2024-09-21T05:47:54.419301+0000","last_clean":"2024-09-21T05:47:54.419301+0000","last_became_active":"2024-09-21T05:47:53.318912+0000","last_became_peered":"2024-09-21T05:47:53.318912+0000","last_unstale":"2024-09-21T05:47:54.419301+0000","last_undegraded":"2024-09-21T05:47:54.419301+0000","last_fullsized":"2024-09-21T05:47:54.419301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:55.408820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":23,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319663+0000","last_change":"2024-09-21T05:47:52.318238+0000","last_active":"2024-09-21T05:47:54.319663+0000","last_peered":"2024-09-21T05:47:54.319663+0000","last_clean":"2024-09-21T05:47:54.319663+0000","last_became_active":"2024-09-21T05:47:52.317230+0000","last_became_peered":"2024-09-21T05:47:52.317230+0000","last_unstale":"2024-09-21T05:47:54.319663+0000","last_undegraded":"2024-09-21T05:47:54.319663+0000","last_fullsized":"2024-09-21T05:47:54.319663+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:37.840490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320255+0000","last_change":"2024-09-21T05:47:53.321414+0000","last_active":"2024-09-21T05:47:54.320255+0000","last_peered":"2024-09-21T05:47:54.320255+0000","last_clean":"2024-09-21T05:47:54.320255+0000","last_became_active":"2024-09-21T05:47:53.321109+0000","last_became_peered":"2024-09-21T05:47:53.321109+0000","last_unstale":"2024-09-21T05:47:54.320255+0000","last_undegraded":"2024-09-21T05:47:54.320255+0000","last_fullsized":"2024-09-21T05:47:54.320255+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:49.957606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421568+0000","last_change":"2024-09-21T05:47:52.326635+0000","last_active":"2024-09-21T05:47:54.421568+0000","last_peered":"2024-09-21T05:47:54.421568+0000","last_clean":"2024-09-21T05:47:54.421568+0000","last_became_active":"2024-09-21T05:47:52.326303+0000","last_became_peered":"2024-09-21T05:47:52.326303+0000","last_unstale":"2024-09-21T05:47:54.421568+0000","last_undegraded":"2024-09-21T05:47:54.421568+0000","last_fullsized":"2024-09-21T05:47:54.421568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.319086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419800+0000","last_change":"2024-09-21T05:47:53.334483+0000","last_active":"2024-09-21T05:47:54.419800+0000","last_peered":"2024-09-21T05:47:54.419800+0000","last_clean":"2024-09-21T05:47:54.419800+0000","last_became_active":"2024-09-21T05:47:53.334127+0000","last_became_peered":"2024-09-21T05:47:53.334127+0000","last_unstale":"2024-09-21T05:47:54.419800+0000","last_undegraded":"2024-09-21T05:47:54.419800+0000","last_fullsized":"2024-09-21T05:47:54.419800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.795868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320154+0000","last_change":"2024-09-21T05:47:52.314440+0000","last_active":"2024-09-21T05:47:54.320154+0000","last_peered":"2024-09-21T05:47:54.320154+0000","last_clean":"2024-09-21T05:47:54.320154+0000","last_became_active":"2024-09-21T05:47:52.314083+0000","last_became_peered":"2024-09-21T05:47:52.314083+0000","last_unstale":"2024-09-21T05:47:54.320154+0000","last_undegraded":"2024-09-21T05:47:54.320154+0000","last_fullsized":"2024-09-21T05:47:54.320154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:05.210599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420081+0000","last_change":"2024-09-21T05:47:53.339114+0000","last_active":"2024-09-21T05:47:54.420081+0000","last_peered":"2024-09-21T05:47:54.420081+0000","last_clean":"2024-09-21T05:47:54.420081+0000","last_became_active":"2024-09-21T05:47:53.338663+0000","last_became_peered":"2024-09-21T05:47:53.338663+0000","last_unstale":"2024-09-21T05:47:54.420081+0000","last_undegraded":"2024-09-21T05:47:54.420081+0000","last_fullsized":"2024-09-21T05:47:54.420081+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:50.046862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323537+0000","last_change":"2024-09-21T05:47:52.318503+0000","last_active":"2024-09-21T05:47:54.323537+0000","last_peered":"2024-09-21T05:47:54.323537+0000","last_clean":"2024-09-21T05:47:54.323537+0000","last_became_active":"2024-09-21T05:47:52.318174+0000","last_became_peered":"2024-09-21T05:47:52.318174+0000","last_unstale":"2024-09-21T05:47:54.323537+0000","last_undegraded":"2024-09-21T05:47:54.323537+0000","last_fullsized":"2024-09-21T05:47:54.323537+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:30.834390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420460+0000","last_change":"2024-09-21T05:47:53.330917+0000","last_active":"2024-09-21T05:47:54.420460+0000","last_peered":"2024-09-21T05:47:54.420460+0000","last_clean":"2024-09-21T05:47:54.420460+0000","last_became_active":"2024-09-21T05:47:53.330486+0000","last_became_peered":"2024-09-21T05:47:53.330486+0000","last_unstale":"2024-09-21T05:47:54.420460+0000","last_undegraded":"2024-09-21T05:47:54.420460+0000","last_fullsized":"2024-09-21T05:47:54.420460+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:00.420835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419066+0000","last_change":"2024-09-21T05:47:52.316818+0000","last_active":"2024-09-21T05:47:54.419066+0000","last_peered":"2024-09-21T05:47:54.419066+0000","last_clean":"2024-09-21T05:47:54.419066+0000","last_became_active":"2024-09-21T05:47:52.316512+0000","last_became_peered":"2024-09-21T05:47:52.316512+0000","last_unstale":"2024-09-21T05:47:54.419066+0000","last_undegraded":"2024-09-21T05:47:54.419066+0000","last_fullsized":"2024-09-21T05:47:54.419066+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:56.759531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419677+0000","last_change":"2024-09-21T05:47:53.322629+0000","last_active":"2024-09-21T05:47:54.419677+0000","last_peered":"2024-09-21T05:47:54.419677+0000","last_clean":"2024-09-21T05:47:54.419677+0000","last_became_active":"2024-09-21T05:47:53.322243+0000","last_became_peered":"2024-09-21T05:47:53.322243+0000","last_unstale":"2024-09-21T05:47:54.419677+0000","last_undegraded":"2024-09-21T05:47:54.419677+0000","last_fullsized":"2024-09-21T05:47:54.419677+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.624866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419891+0000","last_change":"2024-09-21T05:47:52.321742+0000","last_active":"2024-09-21T05:47:54.419891+0000","last_peered":"2024-09-21T05:47:54.419891+0000","last_clean":"2024-09-21T05:47:54.419891+0000","last_became_active":"2024-09-21T05:47:52.321019+0000","last_became_peered":"2024-09-21T05:47:52.321019+0000","last_unstale":"2024-09-21T05:47:54.419891+0000","last_undegraded":"2024-09-21T05:47:54.419891+0000","last_fullsized":"2024-09-21T05:47:54.419891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:34.946405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420780+0000","last_change":"2024-09-21T05:47:53.335546+0000","last_active":"2024-09-21T05:47:54.420780+0000","last_peered":"2024-09-21T05:47:54.420780+0000","last_clean":"2024-09-21T05:47:54.420780+0000","last_became_active":"2024-09-21T05:47:53.335215+0000","last_became_peered":"2024-09-21T05:47:53.335215+0000","last_unstale":"2024-09-21T05:47:54.420780+0000","last_undegraded":"2024-09-21T05:47:54.420780+0000","last_fullsized":"2024-09-21T05:47:54.420780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:18.844619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319672+0000","last_change":"2024-09-21T05:47:52.319751+0000","last_active":"2024-09-21T05:47:54.319672+0000","last_peered":"2024-09-21T05:47:54.319672+0000","last_clean":"2024-09-21T05:47:54.319672+0000","last_became_active":"2024-09-21T05:47:52.319375+0000","last_became_peered":"2024-09-21T05:47:52.319375+0000","last_unstale":"2024-09-21T05:47:54.319672+0000","last_undegraded":"2024-09-21T05:47:54.319672+0000","last_fullsized":"2024-09-21T05:47:54.319672+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:30.482915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419571+0000","last_change":"2024-09-21T05:47:53.332300+0000","last_active":"2024-09-21T05:47:54.419571+0000","last_peered":"2024-09-21T05:47:54.419571+0000","last_clean":"2024-09-21T05:47:54.419571+0000","last_became_active":"2024-09-21T05:47:53.329284+0000","last_became_peered":"2024-09-21T05:47:53.329284+0000","last_unstale":"2024-09-21T05:47:54.419571+0000","last_undegraded":"2024-09-21T05:47:54.419571+0000","last_fullsized":"2024-09-21T05:47:54.419571+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:26.838302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420237+0000","last_change":"2024-09-21T05:47:52.321620+0000","last_active":"2024-09-21T05:47:54.420237+0000","last_peered":"2024-09-21T05:47:54.420237+0000","last_clean":"2024-09-21T05:47:54.420237+0000","last_became_active":"2024-09-21T05:47:52.321166+0000","last_became_peered":"2024-09-21T05:47:52.321166+0000","last_unstale":"2024-09-21T05:47:54.420237+0000","last_undegraded":"2024-09-21T05:47:54.420237+0000","last_fullsized":"2024-09-21T05:47:54.420237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.967667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419550+0000","last_change":"2024-09-21T05:47:53.340973+0000","last_active":"2024-09-21T05:47:54.419550+0000","last_peered":"2024-09-21T05:47:54.419550+0000","last_clean":"2024-09-21T05:47:54.419550+0000","last_became_active":"2024-09-21T05:47:53.340603+0000","last_became_peered":"2024-09-21T05:47:53.340603+0000","last_unstale":"2024-09-21T05:47:54.419550+0000","last_undegraded":"2024-09-21T05:47:54.419550+0000","last_fullsized":"2024-09-21T05:47:54.419550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.320987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420772+0000","last_change":"2024-09-21T05:47:52.326447+0000","last_active":"2024-09-21T05:47:54.420772+0000","last_peered":"2024-09-21T05:47:54.420772+0000","last_clean":"2024-09-21T05:47:54.420772+0000","last_became_active":"2024-09-21T05:47:52.326064+0000","last_became_peered":"2024-09-21T05:47:52.326064+0000","last_unstale":"2024-09-21T05:47:54.420772+0000","last_undegraded":"2024-09-21T05:47:54.420772+0000","last_fullsized":"2024-09-21T05:47:54.420772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:27.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323686+0000","last_change":"2024-09-21T05:47:53.322645+0000","last_active":"2024-09-21T05:47:54.323686+0000","last_peered":"2024-09-21T05:47:54.323686+0000","last_clean":"2024-09-21T05:47:54.323686+0000","last_became_active":"2024-09-21T05:47:53.322384+0000","last_became_peered":"2024-09-21T05:47:53.322384+0000","last_unstale":"2024-09-21T05:47:54.323686+0000","last_undegraded":"2024-09-21T05:47:54.323686+0000","last_fullsized":"2024-09-21T05:47:54.323686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:06.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419780+0000","last_change":"2024-09-21T05:47:52.316775+0000","last_active":"2024-09-21T05:47:54.419780+0000","last_peered":"2024-09-21T05:47:54.419780+0000","last_clean":"2024-09-21T05:47:54.419780+0000","last_became_active":"2024-09-21T05:47:52.315552+0000","last_became_peered":"2024-09-21T05:47:52.315552+0000","last_unstale":"2024-09-21T05:47:54.419780+0000","last_undegraded":"2024-09-21T05:47:54.419780+0000","last_fullsized":"2024-09-21T05:47:54.419780+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:56.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421499+0000","last_change":"2024-09-21T05:47:53.329320+0000","last_active":"2024-09-21T05:47:54.421499+0000","last_peered":"2024-09-21T05:47:54.421499+0000","last_clean":"2024-09-21T05:47:54.421499+0000","last_became_active":"2024-09-21T05:47:53.328785+0000","last_became_peered":"2024-09-21T05:47:53.328785+0000","last_unstale":"2024-09-21T05:47:54.421499+0000","last_undegraded":"2024-09-21T05:47:54.421499+0000","last_fullsized":"2024-09-21T05:47:54.421499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:11.061652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:55.306521+0000","last_change":"2024-09-21T05:47:52.317943+0000","last_active":"2024-09-21T05:47:55.306521+0000","last_peered":"2024-09-21T05:47:55.306521+0000","last_clean":"2024-09-21T05:47:55.306521+0000","last_became_active":"2024-09-21T05:47:52.317606+0000","last_became_peered":"2024-09-21T05:47:52.317606+0000","last_unstale":"2024-09-21T05:47:55.306521+0000","last_undegraded":"2024-09-21T05:47:55.306521+0000","last_fullsized":"2024-09-21T05:47:55.306521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:53.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421251+0000","last_change":"2024-09-21T05:47:53.337568+0000","last_active":"2024-09-21T05:47:54.421251+0000","last_peered":"2024-09-21T05:47:54.421251+0000","last_clean":"2024-09-21T05:47:54.421251+0000","last_became_active":"2024-09-21T05:47:53.337068+0000","last_became_peered":"2024-09-21T05:47:53.337068+0000","last_unstale":"2024-09-21T05:47:54.421251+0000","last_undegraded":"2024-09-21T05:47:54.421251+0000","last_fullsized":"2024-09-21T05:47:54.421251+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:19.441038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421108+0000","last_change":"2024-09-21T05:47:52.325423+0000","last_active":"2024-09-21T05:47:54.421108+0000","last_peered":"2024-09-21T05:47:54.421108+0000","last_clean":"2024-09-21T05:47:54.421108+0000","last_became_active":"2024-09-21T05:47:52.325118+0000","last_became_peered":"2024-09-21T05:47:52.325118+0000","last_unstale":"2024-09-21T05:47:54.421108+0000","last_undegraded":"2024-09-21T05:47:54.421108+0000","last_fullsized":"2024-09-21T05:47:54.421108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:54.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323673+0000","last_change":"2024-09-21T05:47:53.329564+0000","last_active":"2024-09-21T05:47:54.323673+0000","last_peered":"2024-09-21T05:47:54.323673+0000","last_clean":"2024-09-21T05:47:54.323673+0000","last_became_active":"2024-09-21T05:47:53.329189+0000","last_became_peered":"2024-09-21T05:47:53.329189+0000","last_unstale":"2024-09-21T05:47:54.323673+0000","last_undegraded":"2024-09-21T05:47:54.323673+0000","last_fullsized":"2024-09-21T05:47:54.323673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:30.665406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420349+0000","last_change":"2024-09-21T05:47:52.317088+0000","last_active":"2024-09-21T05:47:54.420349+0000","last_peered":"2024-09-21T05:47:54.420349+0000","last_clean":"2024-09-21T05:47:54.420349+0000","last_became_active":"2024-09-21T05:47:52.316526+0000","last_became_peered":"2024-09-21T05:47:52.316526+0000","last_unstale":"2024-09-21T05:47:54.420349+0000","last_undegraded":"2024-09-21T05:47:54.420349+0000","last_fullsized":"2024-09-21T05:47:54.420349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:20.821196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322095+0000","last_change":"2024-09-21T05:47:53.327666+0000","last_active":"2024-09-21T05:47:54.322095+0000","last_peered":"2024-09-21T05:47:54.322095+0000","last_clean":"2024-09-21T05:47:54.322095+0000","last_became_active":"2024-09-21T05:47:53.327242+0000","last_became_peered":"2024-09-21T05:47:53.327242+0000","last_unstale":"2024-09-21T05:47:54.322095+0000","last_undegraded":"2024-09-21T05:47:54.322095+0000","last_fullsized":"2024-09-21T05:47:54.322095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:43.684698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419272+0000","last_change":"2024-09-21T05:47:52.314266+0000","last_active":"2024-09-21T05:47:54.419272+0000","last_peered":"2024-09-21T05:47:54.419272+0000","last_clean":"2024-09-21T05:47:54.419272+0000","last_became_active":"2024-09-21T05:47:52.313950+0000","last_became_peered":"2024-09-21T05:47:52.313950+0000","last_unstale":"2024-09-21T05:47:54.419272+0000","last_undegraded":"2024-09-21T05:47:54.419272+0000","last_fullsized":"2024-09-21T05:47:54.419272+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:54.199451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322027+0000","last_change":"2024-09-21T05:47:53.328445+0000","last_active":"2024-09-21T05:47:54.322027+0000","last_peered":"2024-09-21T05:47:54.322027+0000","last_clean":"2024-09-21T05:47:54.322027+0000","last_became_active":"2024-09-21T05:47:53.328071+0000","last_became_peered":"2024-09-21T05:47:53.328071+0000","last_unstale":"2024-09-21T05:47:54.322027+0000","last_undegraded":"2024-09-21T05:47:54.322027+0000","last_fullsized":"2024-09-21T05:47:54.322027+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:34.151456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321715+0000","last_change":"2024-09-21T05:47:52.312743+0000","last_active":"2024-09-21T05:47:54.321715+0000","last_peered":"2024-09-21T05:47:54.321715+0000","last_clean":"2024-09-21T05:47:54.321715+0000","last_became_active":"2024-09-21T05:47:52.312378+0000","last_became_peered":"2024-09-21T05:47:52.312378+0000","last_unstale":"2024-09-21T05:47:54.321715+0000","last_undegraded":"2024-09-21T05:47:54.321715+0000","last_fullsized":"2024-09-21T05:47:54.321715+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.556704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419782+0000","last_change":"2024-09-21T05:47:53.332432+0000","last_active":"2024-09-21T05:47:54.419782+0000","last_peered":"2024-09-21T05:47:54.419782+0000","last_clean":"2024-09-21T05:47:54.419782+0000","last_became_active":"2024-09-21T05:47:53.331881+0000","last_became_peered":"2024-09-21T05:47:53.331881+0000","last_unstale":"2024-09-21T05:47:54.419782+0000","last_undegraded":"2024-09-21T05:47:54.419782+0000","last_fullsized":"2024-09-21T05:47:54.419782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:21.407811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319732+0000","last_change":"2024-09-21T05:47:52.319216+0000","last_active":"2024-09-21T05:47:54.319732+0000","last_peered":"2024-09-21T05:47:54.319732+0000","last_clean":"2024-09-21T05:47:54.319732+0000","last_became_active":"2024-09-21T05:47:52.318756+0000","last_became_peered":"2024-09-21T05:47:52.318756+0000","last_unstale":"2024-09-21T05:47:54.319732+0000","last_undegraded":"2024-09-21T05:47:54.319732+0000","last_fullsized":"2024-09-21T05:47:54.319732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:12.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420865+0000","last_change":"2024-09-21T05:47:53.337719+0000","last_active":"2024-09-21T05:47:54.420865+0000","last_peered":"2024-09-21T05:47:54.420865+0000","last_clean":"2024-09-21T05:47:54.420865+0000","last_became_active":"2024-09-21T05:47:53.329968+0000","last_became_peered":"2024-09-21T05:47:53.329968+0000","last_unstale":"2024-09-21T05:47:54.420865+0000","last_undegraded":"2024-09-21T05:47:54.420865+0000","last_fullsized":"2024-09-21T05:47:54.420865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:11.856321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420792+0000","last_change":"2024-09-21T05:47:52.324337+0000","last_active":"2024-09-21T05:47:54.420792+0000","last_peered":"2024-09-21T05:47:54.420792+0000","last_clean":"2024-09-21T05:47:54.420792+0000","last_became_active":"2024-09-21T05:47:52.324032+0000","last_became_peered":"2024-09-21T05:47:52.324032+0000","last_unstale":"2024-09-21T05:47:54.420792+0000","last_undegraded":"2024-09-21T05:47:54.420792+0000","last_fullsized":"2024-09-21T05:47:54.420792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:03.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419235+0000","last_change":"2024-09-21T05:47:53.319715+0000","last_active":"2024-09-21T05:47:54.419235+0000","last_peered":"2024-09-21T05:47:54.419235+0000","last_clean":"2024-09-21T05:47:54.419235+0000","last_became_active":"2024-09-21T05:47:53.319394+0000","last_became_peered":"2024-09-21T05:47:53.319394+0000","last_unstale":"2024-09-21T05:47:54.419235+0000","last_undegraded":"2024-09-21T05:47:54.419235+0000","last_fullsized":"2024-09-21T05:47:54.419235+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:24.432342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419478+0000","last_change":"2024-09-21T05:47:52.316016+0000","last_active":"2024-09-21T05:47:54.419478+0000","last_peered":"2024-09-21T05:47:54.419478+0000","last_clean":"2024-09-21T05:47:54.419478+0000","last_became_active":"2024-09-21T05:47:52.315661+0000","last_became_peered":"2024-09-21T05:47:52.315661+0000","last_unstale":"2024-09-21T05:47:54.419478+0000","last_undegraded":"2024-09-21T05:47:54.419478+0000","last_fullsized":"2024-09-21T05:47:54.419478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:23.229664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":271,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419782+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:47:54.419782+0000","last_peered":"2024-09-21T05:47:54.419782+0000","last_clean":"2024-09-21T05:47:54.419782+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:47:54.419782+0000","last_undegraded":"2024-09-21T05:47:54.419782+0000","last_fullsized":"2024-09-21T05:47:54.419782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509549,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1174939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1351680,"data_stored":584025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1355776,"data_stored":584559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1363968,"data_stored":585637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40840,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331000,"statfs":{"total":96636764160,"available":96594944000,"internally_reserved":0,"allocated":1355776,"data_stored":584079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542252,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":1176047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:48:01.962 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.091+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/1637506934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a780cf440 msgr2=0x7f3a7815c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.091+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/1637506934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a780cf440 0x7f3a7815c6a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c005fd0 tx=0x7f3a6c02a540 comp rx=0 tx=0).stop 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.092+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/1637506934 shutdown_connections 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.092+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/1637506934 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a780cf440 0x7f3a7815c6a0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.092+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/1637506934 >> 172.21.15.22:0/1637506934 conn(0x7f3a780710b0 msgr2=0x7f3a780714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.092+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/1637506934 shutdown_connections 2024-09-21T05:48:02.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.092+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/1637506934 wait complete. 2024-09-21T05:48:02.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.094+0000 7f3a7ca4d640 1 Processor -- start 2024-09-21T05:48:02.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.094+0000 7f3a7ca4d640 1 -- start start 2024-09-21T05:48:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7ca4d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a780b7fe0 0x7f3a78149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7ca4d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a780cd360 0x7f3a781f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7ca4d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 0x7f3a781f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7ca4d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f3a780d0dd0 con 0x7f3a7815e090 2024-09-21T05:48:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7ca4d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f3a780d0c50 con 0x7f3a780b7fe0 2024-09-21T05:48:02.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7ca4d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f3a780d0f50 con 0x7f3a780cd360 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 0x7f3a781f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a76d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 0x7f3a781f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50624/0 (socket says 172.21.15.22:50624) 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a76d9d640 1 -- 172.21.15.22:0/306615445 learned_addr learned my addr 172.21.15.22:0/306615445 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a7659c640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a780b7fe0 0x7f3a78149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.095+0000 7f3a75d9b640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a780cd360 0x7f3a781f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a76d9d640 1 -- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a780cd360 msgr2=0x7f3a781f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a76d9d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a780cd360 0x7f3a781f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a76d9d640 1 -- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a780b7fe0 msgr2=0x7f3a78149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a76d9d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a780b7fe0 0x7f3a78149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a76d9d640 1 -- 172.21.15.22:0/306615445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a781f7820 con 0x7f3a7815e090 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a76d9d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 0x7f3a781f6f90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c700 tx=0x7f3a7000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70003b70 con 0x7f3a7815e090 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a70003d10 con 0x7f3a7815e090 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000aab0 con 0x7f3a7815e090 2024-09-21T05:48:02.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a78149e70 con 0x7f3a7815e090 2024-09-21T05:48:02.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.096+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a780c5ed0 con 0x7f3a7815e090 2024-09-21T05:48:02.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.097+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a78149be0 con 0x7f3a7815e090 2024-09-21T05:48:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.098+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3a7000ac50 con 0x7f3a7815e090 2024-09-21T05:48:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.098+0000 7f3a66ffd640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a58078b90 0x7f3a5807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.098+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f3a7009aa30 con 0x7f3a7815e090 2024-09-21T05:48:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.099+0000 7f3a7659c640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a58078b90 0x7f3a5807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.099+0000 7f3a7659c640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a58078b90 0x7f3a5807b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3a68006fd0 tx=0x7f3a68009060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:02.105 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.103+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3a70062820 con 0x7f3a7815e090 2024-09-21T05:48:02.288 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.286+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a78146fb0 con 0x7f3a58078b90 2024-09-21T05:48:02.292 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.291+0000 7f3a66ffd640 1 -- 172.21.15.22:0/306615445 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252360 (secure 0 0 0) 0x7f3a68009550 con 0x7f3a58078b90 2024-09-21T05:48:02.295 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:02.296 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:48:02.297 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a58078b90 msgr2=0x7f3a5807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a58078b90 0x7f3a5807b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3a68006fd0 tx=0x7f3a68009060 comp rx=0 tx=0).stop 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 msgr2=0x7f3a781f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 0x7f3a781f6f90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c700 tx=0x7f3a7000cbc0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 shutdown_connections 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3a7815e090 0x7f3a781f6f90 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3a58078b90 0x7f3a5807b0a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3a780cd360 0x7f3a781f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 --2- 172.21.15.22:0/306615445 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3a780b7fe0 0x7f3a78149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 >> 172.21.15.22:0/306615445 conn(0x7f3a780710b0 msgr2=0x7f3a7815a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 shutdown_connections 2024-09-21T05:48:02.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.294+0000 7f3a7ca4d640 1 -- 172.21.15.22:0/306615445 wait complete. 2024-09-21T05:48:02.307 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-21T05:48:01.879047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282100,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754692620,"statfs":{"total":773094113280,"available":772805242880,"internally_reserved":0,"allocated":12034048,"data_stored":5857196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.574841"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421407+0000","last_change":"2024-09-21T05:47:53.329400+0000","last_active":"2024-09-21T05:47:54.421407+0000","last_peered":"2024-09-21T05:47:54.421407+0000","last_clean":"2024-09-21T05:47:54.421407+0000","last_became_active":"2024-09-21T05:47:53.328865+0000","last_became_peered":"2024-09-21T05:47:53.328865+0000","last_unstale":"2024-09-21T05:47:54.421407+0000","last_undegraded":"2024-09-21T05:47:54.421407+0000","last_fullsized":"2024-09-21T05:47:54.421407+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:33.666746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322207+0000","last_change":"2024-09-21T05:47:53.315211+0000","last_active":"2024-09-21T05:47:54.322207+0000","last_peered":"2024-09-21T05:47:54.322207+0000","last_clean":"2024-09-21T05:47:54.322207+0000","last_became_active":"2024-09-21T05:47:53.314268+0000","last_became_peered":"2024-09-21T05:47:53.314268+0000","last_unstale":"2024-09-21T05:47:54.322207+0000","last_undegraded":"2024-09-21T05:47:54.322207+0000","last_fullsized":"2024-09-21T05:47:54.322207+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:15.893666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420190+0000","last_change":"2024-09-21T05:47:53.322252+0000","last_active":"2024-09-21T05:47:54.420190+0000","last_peered":"2024-09-21T05:47:54.420190+0000","last_clean":"2024-09-21T05:47:54.420190+0000","last_became_active":"2024-09-21T05:47:53.321925+0000","last_became_peered":"2024-09-21T05:47:53.321925+0000","last_unstale":"2024-09-21T05:47:54.420190+0000","last_undegraded":"2024-09-21T05:47:54.420190+0000","last_fullsized":"2024-09-21T05:47:54.420190+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:11.917317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420973+0000","last_change":"2024-09-21T05:47:53.338050+0000","last_active":"2024-09-21T05:47:54.420973+0000","last_peered":"2024-09-21T05:47:54.420973+0000","last_clean":"2024-09-21T05:47:54.420973+0000","last_became_active":"2024-09-21T05:47:53.337279+0000","last_became_peered":"2024-09-21T05:47:53.337279+0000","last_unstale":"2024-09-21T05:47:54.420973+0000","last_undegraded":"2024-09-21T05:47:54.420973+0000","last_fullsized":"2024-09-21T05:47:54.420973+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:49.343256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320073+0000","last_change":"2024-09-21T05:47:53.330191+0000","last_active":"2024-09-21T05:47:54.320073+0000","last_peered":"2024-09-21T05:47:54.320073+0000","last_clean":"2024-09-21T05:47:54.320073+0000","last_became_active":"2024-09-21T05:47:53.329578+0000","last_became_peered":"2024-09-21T05:47:53.329578+0000","last_unstale":"2024-09-21T05:47:54.320073+0000","last_undegraded":"2024-09-21T05:47:54.320073+0000","last_fullsized":"2024-09-21T05:47:54.320073+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:51.237035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420158+0000","last_change":"2024-09-21T05:47:53.314749+0000","last_active":"2024-09-21T05:47:54.420158+0000","last_peered":"2024-09-21T05:47:54.420158+0000","last_clean":"2024-09-21T05:47:54.420158+0000","last_became_active":"2024-09-21T05:47:53.314457+0000","last_became_peered":"2024-09-21T05:47:53.314457+0000","last_unstale":"2024-09-21T05:47:54.420158+0000","last_undegraded":"2024-09-21T05:47:54.420158+0000","last_fullsized":"2024-09-21T05:47:54.420158+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:51:39.746882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419166+0000","last_change":"2024-09-21T05:47:53.322397+0000","last_active":"2024-09-21T05:47:54.419166+0000","last_peered":"2024-09-21T05:47:54.419166+0000","last_clean":"2024-09-21T05:47:54.419166+0000","last_became_active":"2024-09-21T05:47:53.322048+0000","last_became_peered":"2024-09-21T05:47:53.322048+0000","last_unstale":"2024-09-21T05:47:54.419166+0000","last_undegraded":"2024-09-21T05:47:54.419166+0000","last_fullsized":"2024-09-21T05:47:54.419166+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:37.914628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.323792+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.323271+0000","last_became_peered":"2024-09-21T05:47:53.323271+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:22.199819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421377+0000","last_change":"2024-09-21T05:47:53.333144+0000","last_active":"2024-09-21T05:47:54.421377+0000","last_peered":"2024-09-21T05:47:54.421377+0000","last_clean":"2024-09-21T05:47:54.421377+0000","last_became_active":"2024-09-21T05:47:53.332816+0000","last_became_peered":"2024-09-21T05:47:53.332816+0000","last_unstale":"2024-09-21T05:47:54.421377+0000","last_undegraded":"2024-09-21T05:47:54.421377+0000","last_fullsized":"2024-09-21T05:47:54.421377+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:44.276704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421287+0000","last_change":"2024-09-21T05:47:53.328426+0000","last_active":"2024-09-21T05:47:54.421287+0000","last_peered":"2024-09-21T05:47:54.421287+0000","last_clean":"2024-09-21T05:47:54.421287+0000","last_became_active":"2024-09-21T05:47:53.327992+0000","last_became_peered":"2024-09-21T05:47:53.327992+0000","last_unstale":"2024-09-21T05:47:54.421287+0000","last_undegraded":"2024-09-21T05:47:54.421287+0000","last_fullsized":"2024-09-21T05:47:54.421287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:35:02.621431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321989+0000","last_change":"2024-09-21T05:47:53.318301+0000","last_active":"2024-09-21T05:47:54.321989+0000","last_peered":"2024-09-21T05:47:54.321989+0000","last_clean":"2024-09-21T05:47:54.321989+0000","last_became_active":"2024-09-21T05:47:53.317941+0000","last_became_peered":"2024-09-21T05:47:53.317941+0000","last_unstale":"2024-09-21T05:47:54.321989+0000","last_undegraded":"2024-09-21T05:47:54.321989+0000","last_fullsized":"2024-09-21T05:47:54.321989+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:44.546336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421042+0000","last_change":"2024-09-21T05:47:53.338328+0000","last_active":"2024-09-21T05:47:54.421042+0000","last_peered":"2024-09-21T05:47:54.421042+0000","last_clean":"2024-09-21T05:47:54.421042+0000","last_became_active":"2024-09-21T05:47:53.337637+0000","last_became_peered":"2024-09-21T05:47:53.337637+0000","last_unstale":"2024-09-21T05:47:54.421042+0000","last_undegraded":"2024-09-21T05:47:54.421042+0000","last_fullsized":"2024-09-21T05:47:54.421042+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:36:07.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420323+0000","last_change":"2024-09-21T05:47:53.329762+0000","last_active":"2024-09-21T05:47:54.420323+0000","last_peered":"2024-09-21T05:47:54.420323+0000","last_clean":"2024-09-21T05:47:54.420323+0000","last_became_active":"2024-09-21T05:47:53.329426+0000","last_became_peered":"2024-09-21T05:47:53.329426+0000","last_unstale":"2024-09-21T05:47:54.420323+0000","last_undegraded":"2024-09-21T05:47:54.420323+0000","last_fullsized":"2024-09-21T05:47:54.420323+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:44.894421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419422+0000","last_change":"2024-09-21T05:47:53.328646+0000","last_active":"2024-09-21T05:47:54.419422+0000","last_peered":"2024-09-21T05:47:54.419422+0000","last_clean":"2024-09-21T05:47:54.419422+0000","last_became_active":"2024-09-21T05:47:53.328313+0000","last_became_peered":"2024-09-21T05:47:53.328313+0000","last_unstale":"2024-09-21T05:47:54.419422+0000","last_undegraded":"2024-09-21T05:47:54.419422+0000","last_fullsized":"2024-09-21T05:47:54.419422+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:14.951874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419472+0000","last_change":"2024-09-21T05:47:53.314013+0000","last_active":"2024-09-21T05:47:54.419472+0000","last_peered":"2024-09-21T05:47:54.419472+0000","last_clean":"2024-09-21T05:47:54.419472+0000","last_became_active":"2024-09-21T05:47:53.313613+0000","last_became_peered":"2024-09-21T05:47:53.313613+0000","last_unstale":"2024-09-21T05:47:54.419472+0000","last_undegraded":"2024-09-21T05:47:54.419472+0000","last_fullsized":"2024-09-21T05:47:54.419472+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:07:09.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420533+0000","last_change":"2024-09-21T05:47:53.340669+0000","last_active":"2024-09-21T05:47:54.420533+0000","last_peered":"2024-09-21T05:47:54.420533+0000","last_clean":"2024-09-21T05:47:54.420533+0000","last_became_active":"2024-09-21T05:47:53.340336+0000","last_became_peered":"2024-09-21T05:47:53.340336+0000","last_unstale":"2024-09-21T05:47:54.420533+0000","last_undegraded":"2024-09-21T05:47:54.420533+0000","last_fullsized":"2024-09-21T05:47:54.420533+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:28.758224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322315+0000","last_change":"2024-09-21T05:47:53.320885+0000","last_active":"2024-09-21T05:47:54.322315+0000","last_peered":"2024-09-21T05:47:54.322315+0000","last_clean":"2024-09-21T05:47:54.322315+0000","last_became_active":"2024-09-21T05:47:53.320175+0000","last_became_peered":"2024-09-21T05:47:53.320175+0000","last_unstale":"2024-09-21T05:47:54.322315+0000","last_undegraded":"2024-09-21T05:47:54.322315+0000","last_fullsized":"2024-09-21T05:47:54.322315+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:57.310115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323869+0000","last_change":"2024-09-21T05:47:53.317110+0000","last_active":"2024-09-21T05:47:54.323869+0000","last_peered":"2024-09-21T05:47:54.323869+0000","last_clean":"2024-09-21T05:47:54.323869+0000","last_became_active":"2024-09-21T05:47:53.316725+0000","last_became_peered":"2024-09-21T05:47:53.316725+0000","last_unstale":"2024-09-21T05:47:54.323869+0000","last_undegraded":"2024-09-21T05:47:54.323869+0000","last_fullsized":"2024-09-21T05:47:54.323869+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:11.043120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420287+0000","last_change":"2024-09-21T05:47:53.332561+0000","last_active":"2024-09-21T05:47:54.420287+0000","last_peered":"2024-09-21T05:47:54.420287+0000","last_clean":"2024-09-21T05:47:54.420287+0000","last_became_active":"2024-09-21T05:47:53.331867+0000","last_became_peered":"2024-09-21T05:47:53.331867+0000","last_unstale":"2024-09-21T05:47:54.420287+0000","last_undegraded":"2024-09-21T05:47:54.420287+0000","last_fullsized":"2024-09-21T05:47:54.420287+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:16.303376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322107+0000","last_change":"2024-09-21T05:47:53.314765+0000","last_active":"2024-09-21T05:47:54.322107+0000","last_peered":"2024-09-21T05:47:54.322107+0000","last_clean":"2024-09-21T05:47:54.322107+0000","last_became_active":"2024-09-21T05:47:53.314417+0000","last_became_peered":"2024-09-21T05:47:53.314417+0000","last_unstale":"2024-09-21T05:47:54.322107+0000","last_undegraded":"2024-09-21T05:47:54.322107+0000","last_fullsized":"2024-09-21T05:47:54.322107+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:44.328561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322342+0000","last_change":"2024-09-21T05:47:53.323870+0000","last_active":"2024-09-21T05:47:54.322342+0000","last_peered":"2024-09-21T05:47:54.322342+0000","last_clean":"2024-09-21T05:47:54.322342+0000","last_became_active":"2024-09-21T05:47:53.323552+0000","last_became_peered":"2024-09-21T05:47:53.323552+0000","last_unstale":"2024-09-21T05:47:54.322342+0000","last_undegraded":"2024-09-21T05:47:54.322342+0000","last_fullsized":"2024-09-21T05:47:54.322342+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:33.010490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419862+0000","last_change":"2024-09-21T05:47:53.322155+0000","last_active":"2024-09-21T05:47:54.419862+0000","last_peered":"2024-09-21T05:47:54.419862+0000","last_clean":"2024-09-21T05:47:54.419862+0000","last_became_active":"2024-09-21T05:47:53.321833+0000","last_became_peered":"2024-09-21T05:47:53.321833+0000","last_unstale":"2024-09-21T05:47:54.419862+0000","last_undegraded":"2024-09-21T05:47:54.419862+0000","last_fullsized":"2024-09-21T05:47:54.419862+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:01.122192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321454+0000","last_change":"2024-09-21T05:47:53.325437+0000","last_active":"2024-09-21T05:47:54.321454+0000","last_peered":"2024-09-21T05:47:54.321454+0000","last_clean":"2024-09-21T05:47:54.321454+0000","last_became_active":"2024-09-21T05:47:53.318729+0000","last_became_peered":"2024-09-21T05:47:53.318729+0000","last_unstale":"2024-09-21T05:47:54.321454+0000","last_undegraded":"2024-09-21T05:47:54.321454+0000","last_fullsized":"2024-09-21T05:47:54.321454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:03:23.462572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419203+0000","last_change":"2024-09-21T05:47:53.320400+0000","last_active":"2024-09-21T05:47:54.419203+0000","last_peered":"2024-09-21T05:47:54.419203+0000","last_clean":"2024-09-21T05:47:54.419203+0000","last_became_active":"2024-09-21T05:47:53.320056+0000","last_became_peered":"2024-09-21T05:47:53.320056+0000","last_unstale":"2024-09-21T05:47:54.419203+0000","last_undegraded":"2024-09-21T05:47:54.419203+0000","last_fullsized":"2024-09-21T05:47:54.419203+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:51.429652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322541+0000","last_change":"2024-09-21T05:47:53.321337+0000","last_active":"2024-09-21T05:47:54.322541+0000","last_peered":"2024-09-21T05:47:54.322541+0000","last_clean":"2024-09-21T05:47:54.322541+0000","last_became_active":"2024-09-21T05:47:53.320583+0000","last_became_peered":"2024-09-21T05:47:53.320583+0000","last_unstale":"2024-09-21T05:47:54.322541+0000","last_undegraded":"2024-09-21T05:47:54.322541+0000","last_fullsized":"2024-09-21T05:47:54.322541+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:57.315042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421261+0000","last_change":"2024-09-21T05:47:53.335796+0000","last_active":"2024-09-21T05:47:54.421261+0000","last_peered":"2024-09-21T05:47:54.421261+0000","last_clean":"2024-09-21T05:47:54.421261+0000","last_became_active":"2024-09-21T05:47:53.335426+0000","last_became_peered":"2024-09-21T05:47:53.335426+0000","last_unstale":"2024-09-21T05:47:54.421261+0000","last_undegraded":"2024-09-21T05:47:54.421261+0000","last_fullsized":"2024-09-21T05:47:54.421261+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:32.715401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419441+0000","last_change":"2024-09-21T05:47:53.328721+0000","last_active":"2024-09-21T05:47:54.419441+0000","last_peered":"2024-09-21T05:47:54.419441+0000","last_clean":"2024-09-21T05:47:54.419441+0000","last_became_active":"2024-09-21T05:47:53.328431+0000","last_became_peered":"2024-09-21T05:47:53.328431+0000","last_unstale":"2024-09-21T05:47:54.419441+0000","last_undegraded":"2024-09-21T05:47:54.419441+0000","last_fullsized":"2024-09-21T05:47:54.419441+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:21:16.400560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420065+0000","last_change":"2024-09-21T05:47:53.339459+0000","last_active":"2024-09-21T05:47:54.420065+0000","last_peered":"2024-09-21T05:47:54.420065+0000","last_clean":"2024-09-21T05:47:54.420065+0000","last_became_active":"2024-09-21T05:47:53.339054+0000","last_became_peered":"2024-09-21T05:47:53.339054+0000","last_unstale":"2024-09-21T05:47:54.420065+0000","last_undegraded":"2024-09-21T05:47:54.420065+0000","last_fullsized":"2024-09-21T05:47:54.420065+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:16.576540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321766+0000","last_change":"2024-09-21T05:47:53.319552+0000","last_active":"2024-09-21T05:47:54.321766+0000","last_peered":"2024-09-21T05:47:54.321766+0000","last_clean":"2024-09-21T05:47:54.321766+0000","last_became_active":"2024-09-21T05:47:53.319101+0000","last_became_peered":"2024-09-21T05:47:53.319101+0000","last_unstale":"2024-09-21T05:47:54.321766+0000","last_undegraded":"2024-09-21T05:47:54.321766+0000","last_fullsized":"2024-09-21T05:47:54.321766+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:36:44.232318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419434+0000","last_change":"2024-09-21T05:47:53.323957+0000","last_active":"2024-09-21T05:47:54.419434+0000","last_peered":"2024-09-21T05:47:54.419434+0000","last_clean":"2024-09-21T05:47:54.419434+0000","last_became_active":"2024-09-21T05:47:53.323432+0000","last_became_peered":"2024-09-21T05:47:53.323432+0000","last_unstale":"2024-09-21T05:47:54.419434+0000","last_undegraded":"2024-09-21T05:47:54.419434+0000","last_fullsized":"2024-09-21T05:47:54.419434+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:57.449767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420771+0000","last_change":"2024-09-21T05:47:53.340040+0000","last_active":"2024-09-21T05:47:54.420771+0000","last_peered":"2024-09-21T05:47:54.420771+0000","last_clean":"2024-09-21T05:47:54.420771+0000","last_became_active":"2024-09-21T05:47:53.339702+0000","last_became_peered":"2024-09-21T05:47:53.339702+0000","last_unstale":"2024-09-21T05:47:54.420771+0000","last_undegraded":"2024-09-21T05:47:54.420771+0000","last_fullsized":"2024-09-21T05:47:54.420771+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:02.133659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419366+0000","last_change":"2024-09-21T05:47:53.320519+0000","last_active":"2024-09-21T05:47:54.419366+0000","last_peered":"2024-09-21T05:47:54.419366+0000","last_clean":"2024-09-21T05:47:54.419366+0000","last_became_active":"2024-09-21T05:47:53.320182+0000","last_became_peered":"2024-09-21T05:47:53.320182+0000","last_unstale":"2024-09-21T05:47:54.419366+0000","last_undegraded":"2024-09-21T05:47:54.419366+0000","last_fullsized":"2024-09-21T05:47:54.419366+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:16.998601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419700+0000","last_change":"2024-09-21T05:47:53.327229+0000","last_active":"2024-09-21T05:47:54.419700+0000","last_peered":"2024-09-21T05:47:54.419700+0000","last_clean":"2024-09-21T05:47:54.419700+0000","last_became_active":"2024-09-21T05:47:53.326881+0000","last_became_peered":"2024-09-21T05:47:53.326881+0000","last_unstale":"2024-09-21T05:47:54.419700+0000","last_undegraded":"2024-09-21T05:47:54.419700+0000","last_fullsized":"2024-09-21T05:47:54.419700+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:43.659296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"108'4","reported_seq":19,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419430+0000","last_change":"2024-09-21T05:47:52.315229+0000","last_active":"2024-09-21T05:47:54.419430+0000","last_peered":"2024-09-21T05:47:54.419430+0000","last_clean":"2024-09-21T05:47:54.419430+0000","last_became_active":"2024-09-21T05:47:52.314905+0000","last_became_peered":"2024-09-21T05:47:52.314905+0000","last_unstale":"2024-09-21T05:47:54.419430+0000","last_undegraded":"2024-09-21T05:47:54.419430+0000","last_fullsized":"2024-09-21T05:47:54.419430+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:00.549432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323756+0000","last_change":"2024-09-21T05:47:53.325534+0000","last_active":"2024-09-21T05:47:54.323756+0000","last_peered":"2024-09-21T05:47:54.323756+0000","last_clean":"2024-09-21T05:47:54.323756+0000","last_became_active":"2024-09-21T05:47:53.325200+0000","last_became_peered":"2024-09-21T05:47:53.325200+0000","last_unstale":"2024-09-21T05:47:54.323756+0000","last_undegraded":"2024-09-21T05:47:54.323756+0000","last_fullsized":"2024-09-21T05:47:54.323756+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:49.932618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420018+0000","last_change":"2024-09-21T05:47:52.317856+0000","last_active":"2024-09-21T05:47:54.420018+0000","last_peered":"2024-09-21T05:47:54.420018+0000","last_clean":"2024-09-21T05:47:54.420018+0000","last_became_active":"2024-09-21T05:47:52.317462+0000","last_became_peered":"2024-09-21T05:47:52.317462+0000","last_unstale":"2024-09-21T05:47:54.420018+0000","last_undegraded":"2024-09-21T05:47:54.420018+0000","last_fullsized":"2024-09-21T05:47:54.420018+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:51:58.602683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420098+0000","last_change":"2024-09-21T05:47:53.314734+0000","last_active":"2024-09-21T05:47:54.420098+0000","last_peered":"2024-09-21T05:47:54.420098+0000","last_clean":"2024-09-21T05:47:54.420098+0000","last_became_active":"2024-09-21T05:47:53.314373+0000","last_became_peered":"2024-09-21T05:47:53.314373+0000","last_unstale":"2024-09-21T05:47:54.420098+0000","last_undegraded":"2024-09-21T05:47:54.420098+0000","last_fullsized":"2024-09-21T05:47:54.420098+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:44:32.893385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319979+0000","last_change":"2024-09-21T05:47:52.320521+0000","last_active":"2024-09-21T05:47:54.319979+0000","last_peered":"2024-09-21T05:47:54.319979+0000","last_clean":"2024-09-21T05:47:54.319979+0000","last_became_active":"2024-09-21T05:47:52.320077+0000","last_became_peered":"2024-09-21T05:47:52.320077+0000","last_unstale":"2024-09-21T05:47:54.319979+0000","last_undegraded":"2024-09-21T05:47:54.319979+0000","last_fullsized":"2024-09-21T05:47:54.319979+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:11.737474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420363+0000","last_change":"2024-09-21T05:47:53.329736+0000","last_active":"2024-09-21T05:47:54.420363+0000","last_peered":"2024-09-21T05:47:54.420363+0000","last_clean":"2024-09-21T05:47:54.420363+0000","last_became_active":"2024-09-21T05:47:53.329390+0000","last_became_peered":"2024-09-21T05:47:53.329390+0000","last_unstale":"2024-09-21T05:47:54.420363+0000","last_undegraded":"2024-09-21T05:47:54.420363+0000","last_fullsized":"2024-09-21T05:47:54.420363+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:07:27.003776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319826+0000","last_change":"2024-09-21T05:47:52.321048+0000","last_active":"2024-09-21T05:47:54.319826+0000","last_peered":"2024-09-21T05:47:54.319826+0000","last_clean":"2024-09-21T05:47:54.319826+0000","last_became_active":"2024-09-21T05:47:52.320701+0000","last_became_peered":"2024-09-21T05:47:52.320701+0000","last_unstale":"2024-09-21T05:47:54.319826+0000","last_undegraded":"2024-09-21T05:47:54.319826+0000","last_fullsized":"2024-09-21T05:47:54.319826+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:30.308217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420672+0000","last_change":"2024-09-21T05:47:53.336978+0000","last_active":"2024-09-21T05:47:54.420672+0000","last_peered":"2024-09-21T05:47:54.420672+0000","last_clean":"2024-09-21T05:47:54.420672+0000","last_became_active":"2024-09-21T05:47:53.336304+0000","last_became_peered":"2024-09-21T05:47:53.336304+0000","last_unstale":"2024-09-21T05:47:54.420672+0000","last_undegraded":"2024-09-21T05:47:54.420672+0000","last_fullsized":"2024-09-21T05:47:54.420672+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:23.474145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419148+0000","last_change":"2024-09-21T05:47:52.316047+0000","last_active":"2024-09-21T05:47:54.419148+0000","last_peered":"2024-09-21T05:47:54.419148+0000","last_clean":"2024-09-21T05:47:54.419148+0000","last_became_active":"2024-09-21T05:47:52.315653+0000","last_became_peered":"2024-09-21T05:47:52.315653+0000","last_unstale":"2024-09-21T05:47:54.419148+0000","last_undegraded":"2024-09-21T05:47:54.419148+0000","last_fullsized":"2024-09-21T05:47:54.419148+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:49:29.936206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321828+0000","last_change":"2024-09-21T05:47:53.321821+0000","last_active":"2024-09-21T05:47:54.321828+0000","last_peered":"2024-09-21T05:47:54.321828+0000","last_clean":"2024-09-21T05:47:54.321828+0000","last_became_active":"2024-09-21T05:47:53.321524+0000","last_became_peered":"2024-09-21T05:47:53.321524+0000","last_unstale":"2024-09-21T05:47:54.321828+0000","last_undegraded":"2024-09-21T05:47:54.321828+0000","last_fullsized":"2024-09-21T05:47:54.321828+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:49.030281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420062+0000","last_change":"2024-09-21T05:47:52.315884+0000","last_active":"2024-09-21T05:47:54.420062+0000","last_peered":"2024-09-21T05:47:54.420062+0000","last_clean":"2024-09-21T05:47:54.420062+0000","last_became_active":"2024-09-21T05:47:52.315492+0000","last_became_peered":"2024-09-21T05:47:52.315492+0000","last_unstale":"2024-09-21T05:47:54.420062+0000","last_undegraded":"2024-09-21T05:47:54.420062+0000","last_fullsized":"2024-09-21T05:47:54.420062+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:47.033990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420539+0000","last_change":"2024-09-21T05:47:53.339451+0000","last_active":"2024-09-21T05:47:54.420539+0000","last_peered":"2024-09-21T05:47:54.420539+0000","last_clean":"2024-09-21T05:47:54.420539+0000","last_became_active":"2024-09-21T05:47:53.339042+0000","last_became_peered":"2024-09-21T05:47:53.339042+0000","last_unstale":"2024-09-21T05:47:54.420539+0000","last_undegraded":"2024-09-21T05:47:54.420539+0000","last_fullsized":"2024-09-21T05:47:54.420539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:42:24.227545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321401+0000","last_change":"2024-09-21T05:47:52.322566+0000","last_active":"2024-09-21T05:47:54.321401+0000","last_peered":"2024-09-21T05:47:54.321401+0000","last_clean":"2024-09-21T05:47:54.321401+0000","last_became_active":"2024-09-21T05:47:52.322294+0000","last_became_peered":"2024-09-21T05:47:52.322294+0000","last_unstale":"2024-09-21T05:47:54.321401+0000","last_undegraded":"2024-09-21T05:47:54.321401+0000","last_fullsized":"2024-09-21T05:47:54.321401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:33.895532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419785+0000","last_change":"2024-09-21T05:47:53.324736+0000","last_active":"2024-09-21T05:47:54.419785+0000","last_peered":"2024-09-21T05:47:54.419785+0000","last_clean":"2024-09-21T05:47:54.419785+0000","last_became_active":"2024-09-21T05:47:53.324398+0000","last_became_peered":"2024-09-21T05:47:53.324398+0000","last_unstale":"2024-09-21T05:47:54.419785+0000","last_undegraded":"2024-09-21T05:47:54.419785+0000","last_fullsized":"2024-09-21T05:47:54.419785+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:14.690584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420656+0000","last_change":"2024-09-21T05:47:52.315887+0000","last_active":"2024-09-21T05:47:54.420656+0000","last_peered":"2024-09-21T05:47:54.420656+0000","last_clean":"2024-09-21T05:47:54.420656+0000","last_became_active":"2024-09-21T05:47:52.315504+0000","last_became_peered":"2024-09-21T05:47:52.315504+0000","last_unstale":"2024-09-21T05:47:54.420656+0000","last_undegraded":"2024-09-21T05:47:54.420656+0000","last_fullsized":"2024-09-21T05:47:54.420656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:57.957523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419923+0000","last_change":"2024-09-21T05:47:53.329242+0000","last_active":"2024-09-21T05:47:54.419923+0000","last_peered":"2024-09-21T05:47:54.419923+0000","last_clean":"2024-09-21T05:47:54.419923+0000","last_became_active":"2024-09-21T05:47:53.327266+0000","last_became_peered":"2024-09-21T05:47:53.327266+0000","last_unstale":"2024-09-21T05:47:54.419923+0000","last_undegraded":"2024-09-21T05:47:54.419923+0000","last_fullsized":"2024-09-21T05:47:54.419923+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:53.808702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320068+0000","last_change":"2024-09-21T05:47:52.318716+0000","last_active":"2024-09-21T05:47:54.320068+0000","last_peered":"2024-09-21T05:47:54.320068+0000","last_clean":"2024-09-21T05:47:54.320068+0000","last_became_active":"2024-09-21T05:47:52.318376+0000","last_became_peered":"2024-09-21T05:47:52.318376+0000","last_unstale":"2024-09-21T05:47:54.320068+0000","last_undegraded":"2024-09-21T05:47:54.320068+0000","last_fullsized":"2024-09-21T05:47:54.320068+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:36.407295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323810+0000","last_change":"2024-09-21T05:47:53.330894+0000","last_active":"2024-09-21T05:47:54.323810+0000","last_peered":"2024-09-21T05:47:54.323810+0000","last_clean":"2024-09-21T05:47:54.323810+0000","last_became_active":"2024-09-21T05:47:53.330525+0000","last_became_peered":"2024-09-21T05:47:53.330525+0000","last_unstale":"2024-09-21T05:47:54.323810+0000","last_undegraded":"2024-09-21T05:47:54.323810+0000","last_fullsized":"2024-09-21T05:47:54.323810+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:53.569567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419648+0000","last_change":"2024-09-21T05:47:52.315933+0000","last_active":"2024-09-21T05:47:54.419648+0000","last_peered":"2024-09-21T05:47:54.419648+0000","last_clean":"2024-09-21T05:47:54.419648+0000","last_became_active":"2024-09-21T05:47:52.315578+0000","last_became_peered":"2024-09-21T05:47:52.315578+0000","last_unstale":"2024-09-21T05:47:54.419648+0000","last_undegraded":"2024-09-21T05:47:54.419648+0000","last_fullsized":"2024-09-21T05:47:54.419648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:07.116156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420303+0000","last_change":"2024-09-21T05:47:53.320523+0000","last_active":"2024-09-21T05:47:54.420303+0000","last_peered":"2024-09-21T05:47:54.420303+0000","last_clean":"2024-09-21T05:47:54.420303+0000","last_became_active":"2024-09-21T05:47:53.320183+0000","last_became_peered":"2024-09-21T05:47:53.320183+0000","last_unstale":"2024-09-21T05:47:54.420303+0000","last_undegraded":"2024-09-21T05:47:54.420303+0000","last_fullsized":"2024-09-21T05:47:54.420303+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:58.959425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"108'3","reported_seq":18,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421035+0000","last_change":"2024-09-21T05:47:52.322650+0000","last_active":"2024-09-21T05:47:54.421035+0000","last_peered":"2024-09-21T05:47:54.421035+0000","last_clean":"2024-09-21T05:47:54.421035+0000","last_became_active":"2024-09-21T05:47:52.322310+0000","last_became_peered":"2024-09-21T05:47:52.322310+0000","last_unstale":"2024-09-21T05:47:54.421035+0000","last_undegraded":"2024-09-21T05:47:54.421035+0000","last_fullsized":"2024-09-21T05:47:54.421035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:23.430441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419420+0000","last_change":"2024-09-21T05:47:53.326552+0000","last_active":"2024-09-21T05:47:54.419420+0000","last_peered":"2024-09-21T05:47:54.419420+0000","last_clean":"2024-09-21T05:47:54.419420+0000","last_became_active":"2024-09-21T05:47:53.326200+0000","last_became_peered":"2024-09-21T05:47:53.326200+0000","last_unstale":"2024-09-21T05:47:54.419420+0000","last_undegraded":"2024-09-21T05:47:54.419420+0000","last_fullsized":"2024-09-21T05:47:54.419420+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:13.011382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"108'6","reported_seq":21,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421178+0000","last_change":"2024-09-21T05:47:52.324477+0000","last_active":"2024-09-21T05:47:54.421178+0000","last_peered":"2024-09-21T05:47:54.421178+0000","last_clean":"2024-09-21T05:47:54.421178+0000","last_became_active":"2024-09-21T05:47:52.324166+0000","last_became_peered":"2024-09-21T05:47:52.324166+0000","last_unstale":"2024-09-21T05:47:54.421178+0000","last_undegraded":"2024-09-21T05:47:54.421178+0000","last_fullsized":"2024-09-21T05:47:54.421178+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:00.120291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419994+0000","last_change":"2024-09-21T05:47:53.326873+0000","last_active":"2024-09-21T05:47:54.419994+0000","last_peered":"2024-09-21T05:47:54.419994+0000","last_clean":"2024-09-21T05:47:54.419994+0000","last_became_active":"2024-09-21T05:47:53.326548+0000","last_became_peered":"2024-09-21T05:47:53.326548+0000","last_unstale":"2024-09-21T05:47:54.419994+0000","last_undegraded":"2024-09-21T05:47:54.419994+0000","last_fullsized":"2024-09-21T05:47:54.419994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:34:34.754582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419775+0000","last_change":"2024-09-21T05:47:52.316414+0000","last_active":"2024-09-21T05:47:54.419775+0000","last_peered":"2024-09-21T05:47:54.419775+0000","last_clean":"2024-09-21T05:47:54.419775+0000","last_became_active":"2024-09-21T05:47:52.316075+0000","last_became_peered":"2024-09-21T05:47:52.316075+0000","last_unstale":"2024-09-21T05:47:54.419775+0000","last_undegraded":"2024-09-21T05:47:54.419775+0000","last_fullsized":"2024-09-21T05:47:54.419775+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:58.423946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419724+0000","last_change":"2024-09-21T05:47:53.331201+0000","last_active":"2024-09-21T05:47:54.419724+0000","last_peered":"2024-09-21T05:47:54.419724+0000","last_clean":"2024-09-21T05:47:54.419724+0000","last_became_active":"2024-09-21T05:47:53.330540+0000","last_became_peered":"2024-09-21T05:47:53.330540+0000","last_unstale":"2024-09-21T05:47:54.419724+0000","last_undegraded":"2024-09-21T05:47:54.419724+0000","last_fullsized":"2024-09-21T05:47:54.419724+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:13:19.803831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323431+0000","last_change":"2024-09-21T05:47:52.327286+0000","last_active":"2024-09-21T05:47:54.323431+0000","last_peered":"2024-09-21T05:47:54.323431+0000","last_clean":"2024-09-21T05:47:54.323431+0000","last_became_active":"2024-09-21T05:47:52.326918+0000","last_became_peered":"2024-09-21T05:47:52.326918+0000","last_unstale":"2024-09-21T05:47:54.323431+0000","last_undegraded":"2024-09-21T05:47:54.323431+0000","last_fullsized":"2024-09-21T05:47:54.323431+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:52:21.234490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419301+0000","last_change":"2024-09-21T05:47:53.319222+0000","last_active":"2024-09-21T05:47:54.419301+0000","last_peered":"2024-09-21T05:47:54.419301+0000","last_clean":"2024-09-21T05:47:54.419301+0000","last_became_active":"2024-09-21T05:47:53.318912+0000","last_became_peered":"2024-09-21T05:47:53.318912+0000","last_unstale":"2024-09-21T05:47:54.419301+0000","last_undegraded":"2024-09-21T05:47:54.419301+0000","last_fullsized":"2024-09-21T05:47:54.419301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:55.408820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"108'8","reported_seq":23,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319663+0000","last_change":"2024-09-21T05:47:52.318238+0000","last_active":"2024-09-21T05:47:54.319663+0000","last_peered":"2024-09-21T05:47:54.319663+0000","last_clean":"2024-09-21T05:47:54.319663+0000","last_became_active":"2024-09-21T05:47:52.317230+0000","last_became_peered":"2024-09-21T05:47:52.317230+0000","last_unstale":"2024-09-21T05:47:54.319663+0000","last_undegraded":"2024-09-21T05:47:54.319663+0000","last_fullsized":"2024-09-21T05:47:54.319663+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:37.840490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320255+0000","last_change":"2024-09-21T05:47:53.321414+0000","last_active":"2024-09-21T05:47:54.320255+0000","last_peered":"2024-09-21T05:47:54.320255+0000","last_clean":"2024-09-21T05:47:54.320255+0000","last_became_active":"2024-09-21T05:47:53.321109+0000","last_became_peered":"2024-09-21T05:47:53.321109+0000","last_unstale":"2024-09-21T05:47:54.320255+0000","last_undegraded":"2024-09-21T05:47:54.320255+0000","last_fullsized":"2024-09-21T05:47:54.320255+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:49.957606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421568+0000","last_change":"2024-09-21T05:47:52.326635+0000","last_active":"2024-09-21T05:47:54.421568+0000","last_peered":"2024-09-21T05:47:54.421568+0000","last_clean":"2024-09-21T05:47:54.421568+0000","last_became_active":"2024-09-21T05:47:52.326303+0000","last_became_peered":"2024-09-21T05:47:52.326303+0000","last_unstale":"2024-09-21T05:47:54.421568+0000","last_undegraded":"2024-09-21T05:47:54.421568+0000","last_fullsized":"2024-09-21T05:47:54.421568+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:22.319086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419800+0000","last_change":"2024-09-21T05:47:53.334483+0000","last_active":"2024-09-21T05:47:54.419800+0000","last_peered":"2024-09-21T05:47:54.419800+0000","last_clean":"2024-09-21T05:47:54.419800+0000","last_became_active":"2024-09-21T05:47:53.334127+0000","last_became_peered":"2024-09-21T05:47:53.334127+0000","last_unstale":"2024-09-21T05:47:54.419800+0000","last_undegraded":"2024-09-21T05:47:54.419800+0000","last_fullsized":"2024-09-21T05:47:54.419800+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:45.795868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.320154+0000","last_change":"2024-09-21T05:47:52.314440+0000","last_active":"2024-09-21T05:47:54.320154+0000","last_peered":"2024-09-21T05:47:54.320154+0000","last_clean":"2024-09-21T05:47:54.320154+0000","last_became_active":"2024-09-21T05:47:52.314083+0000","last_became_peered":"2024-09-21T05:47:52.314083+0000","last_unstale":"2024-09-21T05:47:54.320154+0000","last_undegraded":"2024-09-21T05:47:54.320154+0000","last_fullsized":"2024-09-21T05:47:54.320154+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:46:05.210599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420081+0000","last_change":"2024-09-21T05:47:53.339114+0000","last_active":"2024-09-21T05:47:54.420081+0000","last_peered":"2024-09-21T05:47:54.420081+0000","last_clean":"2024-09-21T05:47:54.420081+0000","last_became_active":"2024-09-21T05:47:53.338663+0000","last_became_peered":"2024-09-21T05:47:53.338663+0000","last_unstale":"2024-09-21T05:47:54.420081+0000","last_undegraded":"2024-09-21T05:47:54.420081+0000","last_fullsized":"2024-09-21T05:47:54.420081+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:50.046862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"108'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323537+0000","last_change":"2024-09-21T05:47:52.318503+0000","last_active":"2024-09-21T05:47:54.323537+0000","last_peered":"2024-09-21T05:47:54.323537+0000","last_clean":"2024-09-21T05:47:54.323537+0000","last_became_active":"2024-09-21T05:47:52.318174+0000","last_became_peered":"2024-09-21T05:47:52.318174+0000","last_unstale":"2024-09-21T05:47:54.323537+0000","last_undegraded":"2024-09-21T05:47:54.323537+0000","last_fullsized":"2024-09-21T05:47:54.323537+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:08:30.834390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420460+0000","last_change":"2024-09-21T05:47:53.330917+0000","last_active":"2024-09-21T05:47:54.420460+0000","last_peered":"2024-09-21T05:47:54.420460+0000","last_clean":"2024-09-21T05:47:54.420460+0000","last_became_active":"2024-09-21T05:47:53.330486+0000","last_became_peered":"2024-09-21T05:47:53.330486+0000","last_unstale":"2024-09-21T05:47:54.420460+0000","last_undegraded":"2024-09-21T05:47:54.420460+0000","last_fullsized":"2024-09-21T05:47:54.420460+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:00.420835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419066+0000","last_change":"2024-09-21T05:47:52.316818+0000","last_active":"2024-09-21T05:47:54.419066+0000","last_peered":"2024-09-21T05:47:54.419066+0000","last_clean":"2024-09-21T05:47:54.419066+0000","last_became_active":"2024-09-21T05:47:52.316512+0000","last_became_peered":"2024-09-21T05:47:52.316512+0000","last_unstale":"2024-09-21T05:47:54.419066+0000","last_undegraded":"2024-09-21T05:47:54.419066+0000","last_fullsized":"2024-09-21T05:47:54.419066+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:56.759531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419677+0000","last_change":"2024-09-21T05:47:53.322629+0000","last_active":"2024-09-21T05:47:54.419677+0000","last_peered":"2024-09-21T05:47:54.419677+0000","last_clean":"2024-09-21T05:47:54.419677+0000","last_became_active":"2024-09-21T05:47:53.322243+0000","last_became_peered":"2024-09-21T05:47:53.322243+0000","last_unstale":"2024-09-21T05:47:54.419677+0000","last_undegraded":"2024-09-21T05:47:54.419677+0000","last_fullsized":"2024-09-21T05:47:54.419677+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:34.624866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419891+0000","last_change":"2024-09-21T05:47:52.321742+0000","last_active":"2024-09-21T05:47:54.419891+0000","last_peered":"2024-09-21T05:47:54.419891+0000","last_clean":"2024-09-21T05:47:54.419891+0000","last_became_active":"2024-09-21T05:47:52.321019+0000","last_became_peered":"2024-09-21T05:47:52.321019+0000","last_unstale":"2024-09-21T05:47:54.419891+0000","last_undegraded":"2024-09-21T05:47:54.419891+0000","last_fullsized":"2024-09-21T05:47:54.419891+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:34.946405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420780+0000","last_change":"2024-09-21T05:47:53.335546+0000","last_active":"2024-09-21T05:47:54.420780+0000","last_peered":"2024-09-21T05:47:54.420780+0000","last_clean":"2024-09-21T05:47:54.420780+0000","last_became_active":"2024-09-21T05:47:53.335215+0000","last_became_peered":"2024-09-21T05:47:53.335215+0000","last_unstale":"2024-09-21T05:47:54.420780+0000","last_undegraded":"2024-09-21T05:47:54.420780+0000","last_fullsized":"2024-09-21T05:47:54.420780+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:21:18.844619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319672+0000","last_change":"2024-09-21T05:47:52.319751+0000","last_active":"2024-09-21T05:47:54.319672+0000","last_peered":"2024-09-21T05:47:54.319672+0000","last_clean":"2024-09-21T05:47:54.319672+0000","last_became_active":"2024-09-21T05:47:52.319375+0000","last_became_peered":"2024-09-21T05:47:52.319375+0000","last_unstale":"2024-09-21T05:47:54.319672+0000","last_undegraded":"2024-09-21T05:47:54.319672+0000","last_fullsized":"2024-09-21T05:47:54.319672+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:49:30.482915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419571+0000","last_change":"2024-09-21T05:47:53.332300+0000","last_active":"2024-09-21T05:47:54.419571+0000","last_peered":"2024-09-21T05:47:54.419571+0000","last_clean":"2024-09-21T05:47:54.419571+0000","last_became_active":"2024-09-21T05:47:53.329284+0000","last_became_peered":"2024-09-21T05:47:53.329284+0000","last_unstale":"2024-09-21T05:47:54.419571+0000","last_undegraded":"2024-09-21T05:47:54.419571+0000","last_fullsized":"2024-09-21T05:47:54.419571+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:26.838302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420237+0000","last_change":"2024-09-21T05:47:52.321620+0000","last_active":"2024-09-21T05:47:54.420237+0000","last_peered":"2024-09-21T05:47:54.420237+0000","last_clean":"2024-09-21T05:47:54.420237+0000","last_became_active":"2024-09-21T05:47:52.321166+0000","last_became_peered":"2024-09-21T05:47:52.321166+0000","last_unstale":"2024-09-21T05:47:54.420237+0000","last_undegraded":"2024-09-21T05:47:54.420237+0000","last_fullsized":"2024-09-21T05:47:54.420237+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:10:25.967667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419550+0000","last_change":"2024-09-21T05:47:53.340973+0000","last_active":"2024-09-21T05:47:54.419550+0000","last_peered":"2024-09-21T05:47:54.419550+0000","last_clean":"2024-09-21T05:47:54.419550+0000","last_became_active":"2024-09-21T05:47:53.340603+0000","last_became_peered":"2024-09-21T05:47:53.340603+0000","last_unstale":"2024-09-21T05:47:54.419550+0000","last_undegraded":"2024-09-21T05:47:54.419550+0000","last_fullsized":"2024-09-21T05:47:54.419550+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:23.320987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420772+0000","last_change":"2024-09-21T05:47:52.326447+0000","last_active":"2024-09-21T05:47:54.420772+0000","last_peered":"2024-09-21T05:47:54.420772+0000","last_clean":"2024-09-21T05:47:54.420772+0000","last_became_active":"2024-09-21T05:47:52.326064+0000","last_became_peered":"2024-09-21T05:47:52.326064+0000","last_unstale":"2024-09-21T05:47:54.420772+0000","last_undegraded":"2024-09-21T05:47:54.420772+0000","last_fullsized":"2024-09-21T05:47:54.420772+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:47:27.164234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323686+0000","last_change":"2024-09-21T05:47:53.322645+0000","last_active":"2024-09-21T05:47:54.323686+0000","last_peered":"2024-09-21T05:47:54.323686+0000","last_clean":"2024-09-21T05:47:54.323686+0000","last_became_active":"2024-09-21T05:47:53.322384+0000","last_became_peered":"2024-09-21T05:47:53.322384+0000","last_unstale":"2024-09-21T05:47:54.323686+0000","last_undegraded":"2024-09-21T05:47:54.323686+0000","last_fullsized":"2024-09-21T05:47:54.323686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:59:06.323269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"108'2","reported_seq":17,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419780+0000","last_change":"2024-09-21T05:47:52.316775+0000","last_active":"2024-09-21T05:47:54.419780+0000","last_peered":"2024-09-21T05:47:54.419780+0000","last_clean":"2024-09-21T05:47:54.419780+0000","last_became_active":"2024-09-21T05:47:52.315552+0000","last_became_peered":"2024-09-21T05:47:52.315552+0000","last_unstale":"2024-09-21T05:47:54.419780+0000","last_undegraded":"2024-09-21T05:47:54.419780+0000","last_fullsized":"2024-09-21T05:47:54.419780+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:56.148820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421499+0000","last_change":"2024-09-21T05:47:53.329320+0000","last_active":"2024-09-21T05:47:54.421499+0000","last_peered":"2024-09-21T05:47:54.421499+0000","last_clean":"2024-09-21T05:47:54.421499+0000","last_became_active":"2024-09-21T05:47:53.328785+0000","last_became_peered":"2024-09-21T05:47:53.328785+0000","last_unstale":"2024-09-21T05:47:54.421499+0000","last_undegraded":"2024-09-21T05:47:54.421499+0000","last_fullsized":"2024-09-21T05:47:54.421499+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:38:11.061652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"109'1","reported_seq":16,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:55.306521+0000","last_change":"2024-09-21T05:47:52.317943+0000","last_active":"2024-09-21T05:47:55.306521+0000","last_peered":"2024-09-21T05:47:55.306521+0000","last_clean":"2024-09-21T05:47:55.306521+0000","last_became_active":"2024-09-21T05:47:52.317606+0000","last_became_peered":"2024-09-21T05:47:52.317606+0000","last_unstale":"2024-09-21T05:47:55.306521+0000","last_undegraded":"2024-09-21T05:47:55.306521+0000","last_fullsized":"2024-09-21T05:47:55.306521+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:53.649237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421251+0000","last_change":"2024-09-21T05:47:53.337568+0000","last_active":"2024-09-21T05:47:54.421251+0000","last_peered":"2024-09-21T05:47:54.421251+0000","last_clean":"2024-09-21T05:47:54.421251+0000","last_became_active":"2024-09-21T05:47:53.337068+0000","last_became_peered":"2024-09-21T05:47:53.337068+0000","last_unstale":"2024-09-21T05:47:54.421251+0000","last_undegraded":"2024-09-21T05:47:54.421251+0000","last_fullsized":"2024-09-21T05:47:54.421251+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:19.441038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.421108+0000","last_change":"2024-09-21T05:47:52.325423+0000","last_active":"2024-09-21T05:47:54.421108+0000","last_peered":"2024-09-21T05:47:54.421108+0000","last_clean":"2024-09-21T05:47:54.421108+0000","last_became_active":"2024-09-21T05:47:52.325118+0000","last_became_peered":"2024-09-21T05:47:52.325118+0000","last_unstale":"2024-09-21T05:47:54.421108+0000","last_undegraded":"2024-09-21T05:47:54.421108+0000","last_fullsized":"2024-09-21T05:47:54.421108+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:33:54.406919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.323673+0000","last_change":"2024-09-21T05:47:53.329564+0000","last_active":"2024-09-21T05:47:54.323673+0000","last_peered":"2024-09-21T05:47:54.323673+0000","last_clean":"2024-09-21T05:47:54.323673+0000","last_became_active":"2024-09-21T05:47:53.329189+0000","last_became_peered":"2024-09-21T05:47:53.329189+0000","last_unstale":"2024-09-21T05:47:54.323673+0000","last_undegraded":"2024-09-21T05:47:54.323673+0000","last_fullsized":"2024-09-21T05:47:54.323673+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:38:30.665406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420349+0000","last_change":"2024-09-21T05:47:52.317088+0000","last_active":"2024-09-21T05:47:54.420349+0000","last_peered":"2024-09-21T05:47:54.420349+0000","last_clean":"2024-09-21T05:47:54.420349+0000","last_became_active":"2024-09-21T05:47:52.316526+0000","last_became_peered":"2024-09-21T05:47:52.316526+0000","last_unstale":"2024-09-21T05:47:54.420349+0000","last_undegraded":"2024-09-21T05:47:54.420349+0000","last_fullsized":"2024-09-21T05:47:54.420349+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:20.821196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322095+0000","last_change":"2024-09-21T05:47:53.327666+0000","last_active":"2024-09-21T05:47:54.322095+0000","last_peered":"2024-09-21T05:47:54.322095+0000","last_clean":"2024-09-21T05:47:54.322095+0000","last_became_active":"2024-09-21T05:47:53.327242+0000","last_became_peered":"2024-09-21T05:47:53.327242+0000","last_unstale":"2024-09-21T05:47:54.322095+0000","last_undegraded":"2024-09-21T05:47:54.322095+0000","last_fullsized":"2024-09-21T05:47:54.322095+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:43.684698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419272+0000","last_change":"2024-09-21T05:47:52.314266+0000","last_active":"2024-09-21T05:47:54.419272+0000","last_peered":"2024-09-21T05:47:54.419272+0000","last_clean":"2024-09-21T05:47:54.419272+0000","last_became_active":"2024-09-21T05:47:52.313950+0000","last_became_peered":"2024-09-21T05:47:52.313950+0000","last_unstale":"2024-09-21T05:47:54.419272+0000","last_undegraded":"2024-09-21T05:47:54.419272+0000","last_fullsized":"2024-09-21T05:47:54.419272+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:04:54.199451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.322027+0000","last_change":"2024-09-21T05:47:53.328445+0000","last_active":"2024-09-21T05:47:54.322027+0000","last_peered":"2024-09-21T05:47:54.322027+0000","last_clean":"2024-09-21T05:47:54.322027+0000","last_became_active":"2024-09-21T05:47:53.328071+0000","last_became_peered":"2024-09-21T05:47:53.328071+0000","last_unstale":"2024-09-21T05:47:54.322027+0000","last_undegraded":"2024-09-21T05:47:54.322027+0000","last_fullsized":"2024-09-21T05:47:54.322027+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:34.151456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.321715+0000","last_change":"2024-09-21T05:47:52.312743+0000","last_active":"2024-09-21T05:47:54.321715+0000","last_peered":"2024-09-21T05:47:54.321715+0000","last_clean":"2024-09-21T05:47:54.321715+0000","last_became_active":"2024-09-21T05:47:52.312378+0000","last_became_peered":"2024-09-21T05:47:52.312378+0000","last_unstale":"2024-09-21T05:47:54.321715+0000","last_undegraded":"2024-09-21T05:47:54.321715+0000","last_fullsized":"2024-09-21T05:47:54.321715+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:22.556704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419782+0000","last_change":"2024-09-21T05:47:53.332432+0000","last_active":"2024-09-21T05:47:54.419782+0000","last_peered":"2024-09-21T05:47:54.419782+0000","last_clean":"2024-09-21T05:47:54.419782+0000","last_became_active":"2024-09-21T05:47:53.331881+0000","last_became_peered":"2024-09-21T05:47:53.331881+0000","last_unstale":"2024-09-21T05:47:54.419782+0000","last_undegraded":"2024-09-21T05:47:54.419782+0000","last_fullsized":"2024-09-21T05:47:54.419782+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:21.407811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.319732+0000","last_change":"2024-09-21T05:47:52.319216+0000","last_active":"2024-09-21T05:47:54.319732+0000","last_peered":"2024-09-21T05:47:54.319732+0000","last_clean":"2024-09-21T05:47:54.319732+0000","last_became_active":"2024-09-21T05:47:52.318756+0000","last_became_peered":"2024-09-21T05:47:52.318756+0000","last_unstale":"2024-09-21T05:47:54.319732+0000","last_undegraded":"2024-09-21T05:47:54.319732+0000","last_fullsized":"2024-09-21T05:47:54.319732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:12.306404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420865+0000","last_change":"2024-09-21T05:47:53.337719+0000","last_active":"2024-09-21T05:47:54.420865+0000","last_peered":"2024-09-21T05:47:54.420865+0000","last_clean":"2024-09-21T05:47:54.420865+0000","last_became_active":"2024-09-21T05:47:53.329968+0000","last_became_peered":"2024-09-21T05:47:53.329968+0000","last_unstale":"2024-09-21T05:47:54.420865+0000","last_undegraded":"2024-09-21T05:47:54.420865+0000","last_fullsized":"2024-09-21T05:47:54.420865+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:11.856321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.420792+0000","last_change":"2024-09-21T05:47:52.324337+0000","last_active":"2024-09-21T05:47:54.420792+0000","last_peered":"2024-09-21T05:47:54.420792+0000","last_clean":"2024-09-21T05:47:54.420792+0000","last_became_active":"2024-09-21T05:47:52.324032+0000","last_became_peered":"2024-09-21T05:47:52.324032+0000","last_unstale":"2024-09-21T05:47:54.420792+0000","last_undegraded":"2024-09-21T05:47:54.420792+0000","last_fullsized":"2024-09-21T05:47:54.420792+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:03.622067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419235+0000","last_change":"2024-09-21T05:47:53.319715+0000","last_active":"2024-09-21T05:47:54.419235+0000","last_peered":"2024-09-21T05:47:54.419235+0000","last_clean":"2024-09-21T05:47:54.419235+0000","last_became_active":"2024-09-21T05:47:53.319394+0000","last_became_peered":"2024-09-21T05:47:53.319394+0000","last_unstale":"2024-09-21T05:47:54.419235+0000","last_undegraded":"2024-09-21T05:47:54.419235+0000","last_fullsized":"2024-09-21T05:47:54.419235+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:52.291430+0000","last_clean_scrub_stamp":"2024-09-21T05:47:52.291430+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:32:24.432342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419478+0000","last_change":"2024-09-21T05:47:52.316016+0000","last_active":"2024-09-21T05:47:54.419478+0000","last_peered":"2024-09-21T05:47:54.419478+0000","last_clean":"2024-09-21T05:47:54.419478+0000","last_became_active":"2024-09-21T05:47:52.315661+0000","last_became_peered":"2024-09-21T05:47:52.315661+0000","last_unstale":"2024-09-21T05:47:54.419478+0000","last_undegraded":"2024-09-21T05:47:54.419478+0000","last_fullsized":"2024-09-21T05:47:54.419478+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:47:51.289529+0000","last_clean_scrub_stamp":"2024-09-21T05:47:51.289529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:39:23.229664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":271,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-21T05:47:54.419782+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:47:54.419782+0000","last_peered":"2024-09-21T05:47:54.419782+0000","last_clean":"2024-09-21T05:47:54.419782+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:47:54.419782+0000","last_undegraded":"2024-09-21T05:47:54.419782+0000","last_fullsized":"2024-09-21T05:47:54.419782+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509550,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1174939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542253,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1351680,"data_stored":584025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1355776,"data_stored":584559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542252,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36688,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335152,"statfs":{"total":96636764160,"available":96599195648,"internally_reserved":0,"allocated":1363968,"data_stored":585637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36672,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335168,"statfs":{"total":96636764160,"available":96599212032,"internally_reserved":0,"allocated":1347584,"data_stored":583955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40840,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331000,"statfs":{"total":96636764160,"available":96594944000,"internally_reserved":0,"allocated":1355776,"data_stored":584079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542253,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37264,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334576,"statfs":{"total":96636764160,"available":96598605824,"internally_reserved":0,"allocated":1961984,"data_stored":1176047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:48:02.311 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:48:02.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 -- 172.21.15.22:0/4102595711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 msgr2=0x7f281800ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 --2- 172.21.15.22:0/4102595711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f281800ba00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2828006fd0 tx=0x7f2818005830 comp rx=0 tx=0).stop 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 -- 172.21.15.22:0/4102595711 shutdown_connections 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 --2- 172.21.15.22:0/4102595711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f281800ba00 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 -- 172.21.15.22:0/4102595711 >> 172.21.15.22:0/4102595711 conn(0x7f28300b1cb0 msgr2=0x7f28300b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 -- 172.21.15.22:0/4102595711 shutdown_connections 2024-09-21T05:48:02.473 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.470+0000 7f2836332640 1 -- 172.21.15.22:0/4102595711 wait complete. 2024-09-21T05:48:02.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.472+0000 7f2836332640 1 Processor -- start 2024-09-21T05:48:02.474 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.472+0000 7f2836332640 1 -- start start 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f2836332640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f283015e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f2836332640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f283015b450 0x7f28301f3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f2836332640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28301f4530 0x7f283014dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f2836332640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28300cc830 con 0x7f2818005350 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f2836332640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28300cbd70 con 0x7f283015b450 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f2836332640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f28300cc9b0 con 0x7f28301f4530 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f283015e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.477 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f283015e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50634/0 (socket says 172.21.15.22:50634) 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 -- 172.21.15.22:0/4201404166 learned_addr learned my addr 172.21.15.22:0/4201404166 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f28348cf640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28301f4530 0x7f283014dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 -- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28301f4530 msgr2=0x7f283014dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28301f4530 0x7f283014dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 -- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f283015b450 msgr2=0x7f28301f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f283015b450 0x7f28301f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.474+0000 7f282ffff640 1 -- 172.21.15.22:0/4201404166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283015c790 con 0x7f2818005350 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.475+0000 7f282ffff640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f283015e6e0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f282000b420 tx=0x7f282000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.475+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820011830 con 0x7f2818005350 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.475+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2820016070 con 0x7f2818005350 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.475+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283015ca70 con 0x7f2818005350 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.475+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820011d80 con 0x7f2818005350 2024-09-21T05:48:02.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.475+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28300ad270 con 0x7f2818005350 2024-09-21T05:48:02.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.476+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28300ccf10 con 0x7f2818005350 2024-09-21T05:48:02.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.477+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f282001b460 con 0x7f2818005350 2024-09-21T05:48:02.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.477+0000 7f282cff9640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f281c078b90 0x7f281c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.477+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7f282009a8e0 con 0x7f2818005350 2024-09-21T05:48:02.479 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.477+0000 7f282f7fe640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f281c078b90 0x7f281c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.478+0000 7f282f7fe640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f281c078b90 0x7f281c07b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2824000c00 tx=0x7f28240039c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:02.484 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.482+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2820062db0 con 0x7f2818005350 2024-09-21T05:48:02.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.744+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f28300cb4d0 con 0x7f2818005350 2024-09-21T05:48:02.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.747+0000 7f282cff9640 1 -- 172.21.15.22:0/4201404166 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f28200626b0 con 0x7f2818005350 2024-09-21T05:48:02.749 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-09-21T05:48:02.749 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQDPW+5mfLjeLxAAsXLXCBX1u1BVQ8jC2Wc6RA== 2024-09-21T05:48:02.749 INFO:teuthology.orchestra.run.smithi022.stdout: caps mds = "allow" 2024-09-21T05:48:02.749 INFO:teuthology.orchestra.run.smithi022.stdout: caps mon = "allow r" 2024-09-21T05:48:02.750 INFO:teuthology.orchestra.run.smithi022.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T05:48:02.750 INFO:teuthology.orchestra.run.smithi022.stderr:updated caps for client.0 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f281c078b90 msgr2=0x7f281c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f281c078b90 0x7f281c07b0a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2824000c00 tx=0x7f28240039c0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 msgr2=0x7f283015e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f283015e6e0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f282000b420 tx=0x7f282000b8e0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 shutdown_connections 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f281c078b90 0x7f281c07b0a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28301f4530 0x7f283014dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f283015b450 0x7f28301f3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 --2- 172.21.15.22:0/4201404166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f2818005350 0x7f283015e6e0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 >> 172.21.15.22:0/4201404166 conn(0x7f28300b1cb0 msgr2=0x7f28300b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 shutdown_connections 2024-09-21T05:48:02.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.750+0000 7f2836332640 1 -- 172.21.15.22:0/4201404166 wait complete. 2024-09-21T05:48:02.763 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 -- 172.21.15.22:0/103290422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 msgr2=0x7fdb780d14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 --2- 172.21.15.22:0/103290422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780d14f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c005fd0 tx=0x7fdb6c02a4a0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 -- 172.21.15.22:0/103290422 shutdown_connections 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 --2- 172.21.15.22:0/103290422 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780d14f0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 --2- 172.21.15.22:0/103290422 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb780b9b60 0x7fdb780d0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 -- 172.21.15.22:0/103290422 >> 172.21.15.22:0/103290422 conn(0x7fdb7809d8f0 msgr2=0x7fdb780d45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 -- 172.21.15.22:0/103290422 shutdown_connections 2024-09-21T05:48:02.881 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.878+0000 7fdb7e777640 1 -- 172.21.15.22:0/103290422 wait complete. 2024-09-21T05:48:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.880+0000 7fdb7e777640 1 Processor -- start 2024-09-21T05:48:02.882 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.880+0000 7fdb7e777640 1 -- start start 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.881+0000 7fdb7e777640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.881+0000 7fdb7e777640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb780b9b60 0x7fdb78157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb7e777640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78162430 0x7fdb7815d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb7e777640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781630d0 con 0x7fdb780b3a10 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb7e777640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78162f50 con 0x7fdb78162430 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb7e777640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fdb78163250 con 0x7fdb780b9b60 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50648/0 (socket says 172.21.15.22:50648) 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 -- 172.21.15.22:0/2279208193 learned_addr learned my addr 172.21.15.22:0/2279208193 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb777fe640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb780b9b60 0x7fdb78157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb7cd14640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78162430 0x7fdb7815d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 -- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb780b9b60 msgr2=0x7fdb78157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb780b9b60 0x7fdb78157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 -- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78162430 msgr2=0x7fdb7815d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78162430 0x7fdb7815d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 -- 172.21.15.22:0/2279208193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb781f5f40 con 0x7fdb780b3a10 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb77fff640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780b47f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800c970 tx=0x7fdb6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.882+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68007bb0 con 0x7fdb780b3a10 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.883+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb68007e40 con 0x7fdb780b3a10 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.883+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6800f690 con 0x7fdb780b3a10 2024-09-21T05:48:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.883+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb781f61c0 con 0x7fdb780b3a10 2024-09-21T05:48:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.883+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781f6780 con 0x7fdb780b3a10 2024-09-21T05:48:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.884+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb78162840 con 0x7fdb780b3a10 2024-09-21T05:48:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.884+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb6800f830 con 0x7fdb780b3a10 2024-09-21T05:48:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.885+0000 7fdb74ff9640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdb64078b90 0x7fdb6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.885+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fdb6809b5c0 con 0x7fdb780b3a10 2024-09-21T05:48:02.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.885+0000 7fdb777fe640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdb64078b90 0x7fdb6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.886+0000 7fdb777fe640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdb64078b90 0x7fdb6407b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c033000 tx=0x7fdb6c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:02.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:02.890+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb781f5f40 con 0x7fdb780b3a10 2024-09-21T05:48:03.118 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.116+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb780ba0d0 con 0x7fdb780b3a10 2024-09-21T05:48:03.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.117+0000 7fdb74ff9640 1 -- 172.21.15.22:0/2279208193 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5230 (secure 0 0 0) 0x7fdb68014030 con 0x7fdb780b3a10 2024-09-21T05:48:03.119 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:03.119 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":101,"btime":"2024-09-21T05:47:56:326350+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8049,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/3772214991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3772214991},{"type":"v1","addr":"172.21.15.22:6835","nonce":3772214991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:47:55.710024+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8055},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8055":{"gid":8055,"name":"b","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.134:6837/4249862141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":4249862141},{"type":"v1","addr":"172.21.15.134:6837","nonce":4249862141}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8055,"qdb_cluster":[8055]},"id":8}]} 2024-09-21T05:48:03.120 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 101 2024-09-21T05:48:03.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdb64078b90 msgr2=0x7fdb6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdb64078b90 0x7fdb6407b0a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c033000 tx=0x7fdb6c004730 comp rx=0 tx=0).stop 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 msgr2=0x7fdb780b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780b47f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800c970 tx=0x7fdb6800ce30 comp rx=0 tx=0).stop 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 shutdown_connections 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fdb78162430 0x7fdb7815d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fdb64078b90 0x7fdb6407b0a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fdb780b9b60 0x7fdb78157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 --2- 172.21.15.22:0/2279208193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fdb780b3a10 0x7fdb780b47f0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 >> 172.21.15.22:0/2279208193 conn(0x7fdb7809d8f0 msgr2=0x7fdb780b8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 shutdown_connections 2024-09-21T05:48:03.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.119+0000 7fdb7e777640 1 -- 172.21.15.22:0/2279208193 wait complete. 2024-09-21T05:48:03.535 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:47:55.710024+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8055}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8055': {'gid': 8055, 'name': 'b', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.134:6837/4249862141', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.134:6835', 'nonce': 4249862141}, {'type': 'v1', 'addr': '172.21.15.134:6837', 'nonce': 4249862141}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8055, 'qdb_cluster': [8055]} 2024-09-21T05:48:03.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:48:03.536 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T05:48:03.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 -- 172.21.15.22:0/532079136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 msgr2=0x7fb3440ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 --2- 172.21.15.22:0/532079136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440ccbc0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb334005fd0 tx=0x7fb33402a520 comp rx=0 tx=0).stop 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 -- 172.21.15.22:0/532079136 shutdown_connections 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 --2- 172.21.15.22:0/532079136 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440b7cc0 0x7fb3440cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 --2- 172.21.15.22:0/532079136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440ccbc0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 -- 172.21.15.22:0/532079136 >> 172.21.15.22:0/532079136 conn(0x7fb34406d4f0 msgr2=0x7fb344071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 -- 172.21.15.22:0/532079136 shutdown_connections 2024-09-21T05:48:03.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.659+0000 7fb34a207640 1 -- 172.21.15.22:0/532079136 wait complete. 2024-09-21T05:48:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.661+0000 7fb34a207640 1 Processor -- start 2024-09-21T05:48:03.663 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.661+0000 7fb34a207640 1 -- start start 2024-09-21T05:48:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb34a207640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440b7cc0 0x7fb3440a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb34a207640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3440cf150 0x7fb3441582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb34a207640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb34a207640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440b4460 con 0x7fb3441600b0 2024-09-21T05:48:03.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb34a207640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440d1ba0 con 0x7fb3440cf150 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb34a207640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440b45e0 con 0x7fb3440b7cc0 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440b0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50662/0 (socket says 172.21.15.22:50662) 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.662+0000 7fb343fff640 1 -- 172.21.15.22:0/3176824665 learned_addr learned my addr 172.21.15.22:0/3176824665 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb342ffd640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3440cf150 0x7fb3441582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440b7cc0 0x7fb3440a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb343fff640 1 -- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440b7cc0 msgr2=0x7fb3440a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb343fff640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440b7cc0 0x7fb3440a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb343fff640 1 -- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3440cf150 msgr2=0x7fb3441582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb343fff640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3440cf150 0x7fb3441582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb343fff640 1 -- 172.21.15.22:0/3176824665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34415b3b0 con 0x7fb3441600b0 2024-09-21T05:48:03.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb343fff640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440b0af0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00ba10 tx=0x7fb33c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb32bfff640 1 -- 172.21.15.22:0/3176824665 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c019070 con 0x7fb3441600b0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb32bfff640 1 -- 172.21.15.22:0/3176824665 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c00cc30 con 0x7fb3441600b0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34415b0d0 con 0x7fb3441600b0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb32bfff640 1 -- 172.21.15.22:0/3176824665 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0146a0 con 0x7fb3441600b0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.663+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3441f7630 con 0x7fb3441600b0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.665+0000 7fb32bfff640 1 -- 172.21.15.22:0/3176824665 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb33c01c070 con 0x7fb3441600b0 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.665+0000 7fb32bfff640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb330078b90 0x7fb33007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.667 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.666+0000 7fb32bfff640 1 -- 172.21.15.22:0/3176824665 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fb33c09a9b0 con 0x7fb3441600b0 2024-09-21T05:48:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.666+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb330078b90 0x7fb33007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.668 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.666+0000 7fb35124d740 1 Processor -- start 2024-09-21T05:48:03.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.667+0000 7fb35124d740 1 -- start start 2024-09-21T05:48:03.669 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.667+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb330078b90 0x7fb33007b0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb334033000 tx=0x7fb334004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c1ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb35124d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 0x55914c1c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb35124d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55914c0f3820 con 0x55914bdb8590 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55914c0e2c50 con 0x55914bdb03b0 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55914c0e6a60 con 0x55914bdd9270 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 0x55914c1c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 0x55914c1c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50676/0 (socket says 172.21.15.22:50676) 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 -- 172.21.15.22:0/618394666 learned_addr learned my addr 172.21.15.22:0/618394666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:03.671 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb342ffd640 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c1ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb3437fe640 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 -- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 msgr2=0x55914c1aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 -- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 msgr2=0x55914c1ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c1ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.668+0000 7fb343fff640 1 -- 172.21.15.22:0/618394666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914be44850 con 0x55914bdb8590 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.669+0000 7fb343fff640 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 0x55914c1c8400 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fb33c062700 tx=0x7fb33c061fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d935a6693dc4dc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.669+0000 7fb3297fa640 1 -- 172.21.15.22:0/618394666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c062e70 con 0x55914bdb8590 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.669+0000 7fb3297fa640 1 -- 172.21.15.22:0/618394666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c062770 con 0x55914bdb8590 2024-09-21T05:48:03.672 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.669+0000 7fb3297fa640 1 -- 172.21.15.22:0/618394666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c06beb0 con 0x55914bdb8590 2024-09-21T05:48:03.673 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 -- 172.21.15.22:0/618394666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 msgr2=0x55914c1c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 0x55914c1c8400 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fb33c062700 tx=0x7fb33c061fa0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 -- 172.21.15.22:0/618394666 shutdown_connections 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb8590 0x55914c1c8400 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c1ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 --2- 172.21.15.22:0/618394666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 -- 172.21.15.22:0/618394666 >> 172.21.15.22:0/618394666 conn(0x55914bdcd3c0 msgr2=0x55914c181280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 -- 172.21.15.22:0/618394666 shutdown_connections 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.670+0000 7fb35124d740 1 -- 172.21.15.22:0/618394666 wait complete. 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.671+0000 7fb35124d740 1 -- start start 2024-09-21T05:48:03.674 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.671+0000 7fb35124d740 1 Processor -- start 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 0x55914c1d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c1bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55914be53620 con 0x55914bdb03b0 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55914c0885f0 con 0x55914bdb8590 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55914c1aba40 con 0x55914bdd9270 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb35124d740 10 client.0 _ll_register_callbacks cb 0x55914c08c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 0x55914c1d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 0x55914c1d4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50690/0 (socket says 172.21.15.22:50690) 2024-09-21T05:48:03.678 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 -- 172.21.15.22:0/2723321627 learned_addr learned my addr 172.21.15.22:0/2723321627 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb342ffd640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c1bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb343fff640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 -- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 msgr2=0x55914c1c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 -- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 msgr2=0x55914c1bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c1bf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb3437fe640 1 -- 172.21.15.22:0/2723321627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914bebb6b0 con 0x55914bdb03b0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb342ffd640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c1bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.672+0000 7fb343fff640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3437fe640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 0x55914c1d4ae0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb3340099c0 tx=0x7fb334009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3415b935ac0ea2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3227fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3227fc640 1 -- 172.21.15.22:0/2723321627 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33404c020 con 0x55914bdb03b0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3227fc640 1 -- 172.21.15.22:0/2723321627 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fb334004bb0 con 0x55914bdb03b0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c062770 con 0x55914bdb03b0 2024-09-21T05:48:03.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3227fc640 1 -- 172.21.15.22:0/2723321627 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fb334056050 con 0x55914bdb03b0 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 10 client.8187 fetch_fsmap learned FSMap version 101 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55914c1be8d0 con 0x55914bdb03b0 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3227fc640 1 -- 172.21.15.22:0/2723321627 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c062770 con 0x55914bdb03b0 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb3227fc640 1 -- 172.21.15.22:0/2723321627 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fb334059020 con 0x55914bdb03b0 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 10 client.8187 fetch_fsmap finished waiting for FSMap version 101 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 10 client.8187 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 20 client.8187 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914bd97e40 0x55914c1c0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 4 client.8187 mds_command: new command op to 8055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.673+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 --> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914c0f5770 con 0x55914bd97e40 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.674+0000 7fb342ffd640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914bd97e40 0x55914c1c0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.674+0000 7fb342ffd640 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914bd97e40 0x55914c1c0390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12496612d8a820e server_cookie=71590544516f4fd0 in_seq=0 out_seq=0 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.674+0000 7fb3227fc640 10 client.8187 ms_handle_connect on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb3227fc640 1 -- 172.21.15.22:0/2723321627 <== mds.0 v2:172.21.15.134:6835/4249862141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55914c0f5770 con 0x55914bd97e40 2024-09-21T05:48:03.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb3227fc640 10 client.8187 handle_command_reply: tid=0 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 client.8187 shutdown 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 msgr2=0x55914c1d4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 0x55914c1d4ae0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb3340099c0 tx=0x7fb334009260 comp rx=0 tx=0).stop 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 shutdown_connections 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914bd97e40 0x55914c1c0390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdd9270 0x55914c1c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c1bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb3227fc640 0 client.8187 ms_handle_reset on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 --2- 172.21.15.22:0/2723321627 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdb03b0 0x55914c1d4ae0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 >> 172.21.15.22:0/2723321627 conn(0x55914bdcd3c0 msgr2=0x55914c17a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 shutdown_connections 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.675+0000 7fb35124d740 1 -- 172.21.15.22:0/2723321627 wait complete. 2024-09-21T05:48:03.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.676+0000 7fb35124d740 20 client.8187 trim_cache size 0 max 16384 2024-09-21T05:48:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.694+0000 7fb35124d740 1 Processor -- start 2024-09-21T05:48:03.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.694+0000 7fb35124d740 1 -- start start 2024-09-21T05:48:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.695+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb03b0 0x55914c17a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.695+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c17be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.695+0000 7fb35124d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.695+0000 7fb35124d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb3440c9e50 con 0x55914bdd9270 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.695+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441f4a00 con 0x55914bdb8590 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.695+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440c9800 con 0x55914bdb03b0 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50700/0 (socket says 172.21.15.22:50700) 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 -- 172.21.15.22:0/3933250719 learned_addr learned my addr 172.21.15.22:0/3933250719 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb342ffd640 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c17be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb03b0 0x55914c17a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 -- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb03b0 msgr2=0x55914c17a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb03b0 0x55914c17a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 -- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 msgr2=0x55914c17be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c17be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 -- 172.21.15.22:0/3933250719 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914c1bfc00 con 0x55914bdd9270 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb342ffd640 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c17be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:03.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb343fff640 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bbfa0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0671c0 tx=0x7fb33c098da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b17813dfc25e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb328ff9640 1 -- 172.21.15.22:0/3933250719 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c080030 con 0x55914bdd9270 2024-09-21T05:48:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb328ff9640 1 -- 172.21.15.22:0/3933250719 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c067250 con 0x55914bdd9270 2024-09-21T05:48:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.696+0000 7fb328ff9640 1 -- 172.21.15.22:0/3933250719 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c06c100 con 0x55914bdd9270 2024-09-21T05:48:03.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 -- 172.21.15.22:0/3933250719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 msgr2=0x55914c1bbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bbfa0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0671c0 tx=0x7fb33c098da0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 -- 172.21.15.22:0/3933250719 shutdown_connections 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bbfa0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb8590 0x55914c17be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 --2- 172.21.15.22:0/3933250719 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb03b0 0x55914c17a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 -- 172.21.15.22:0/3933250719 >> 172.21.15.22:0/3933250719 conn(0x55914bdcd3c0 msgr2=0x55914c1a80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 -- 172.21.15.22:0/3933250719 shutdown_connections 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.697+0000 7fb35124d740 1 -- 172.21.15.22:0/3933250719 wait complete. 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.698+0000 7fb35124d740 1 -- start start 2024-09-21T05:48:03.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.698+0000 7fb35124d740 1 Processor -- start 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.699+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c23f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.699+0000 7fb35124d740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb8590 0x55914c1bf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.699+0000 7fb35124d740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.699+0000 7fb35124d740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55914c1d9680 con 0x55914bdd9270 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.699+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55914c1c2740 con 0x55914bdb03b0 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.699+0000 7fb35124d740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55914c1bdc00 con 0x55914bdb8590 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb35124d740 10 client.0 _ll_register_callbacks cb 0x7fb334004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50714/0 (socket says 172.21.15.22:50714) 2024-09-21T05:48:03.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 -- 172.21.15.22:0/3204437650 learned_addr learned my addr 172.21.15.22:0/3204437650 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb342ffd640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb8590 0x55914c1bf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c23f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 -- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb8590 msgr2=0x55914c1bf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb8590 0x55914c1bf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 -- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 msgr2=0x55914c23f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c23f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 -- 172.21.15.22:0/3204437650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914c1e8ea0 con 0x55914bdd9270 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb343fff640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bf550 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b9e0 tx=0x7fb33c06bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676a70c05c7889d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb3227fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb3227fc640 1 -- 172.21.15.22:0/3204437650 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c072060 con 0x55914bdd9270 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb3227fc640 1 -- 172.21.15.22:0/3204437650 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 5709+0+0 (secure 0 0 0) 0x7fb33c069460 con 0x55914bdd9270 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c067250 con 0x55914bdd9270 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb3227fc640 1 -- 172.21.15.22:0/3204437650 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fb33c080070 con 0x55914bdd9270 2024-09-21T05:48:03.706 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb35124d740 10 client.8193 fetch_fsmap learned FSMap version 101 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.700+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55914c23fdd0 con 0x55914bdd9270 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb3227fc640 1 -- 172.21.15.22:0/3204437650 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb33c067250 con 0x55914bdd9270 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb3227fc640 1 -- 172.21.15.22:0/3204437650 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fb33c05e0c0 con 0x55914bdd9270 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb35124d740 10 client.8193 fetch_fsmap finished waiting for FSMap version 101 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb35124d740 10 client.8193 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb35124d740 20 client.8193 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb35124d740 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914c248ff0 0x55914c249400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb35124d740 4 client.8193 mds_command: new command op to 8055 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 --> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] -- command(tid 0: {"prefix": "status"}) -- 0x55914c0f1220 con 0x55914c248ff0 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914c248ff0 0x55914c249400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb3437fe640 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914c248ff0 0x55914c249400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae867f2545f3b18d server_cookie=1f6797791675577d in_seq=0 out_seq=0 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.701+0000 7fb3227fc640 10 client.8193 ms_handle_connect on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb3227fc640 1 -- 172.21.15.22:0/3204437650 <== mds.0 v2:172.21.15.134:6835/4249862141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55914c0f1220 con 0x55914c248ff0 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb3227fc640 10 client.8193 handle_command_reply: tid=0 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 client.8193 shutdown 2024-09-21T05:48:03.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 msgr2=0x55914c1bf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bf550 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b9e0 tx=0x7fb33c06bf20 comp rx=0 tx=0).stop 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 shutdown_connections 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x55914c248ff0 0x55914c249400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55914bdd9270 0x55914c1bf550 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55914bdb8590 0x55914c1bf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb3227fc640 0 client.8193 ms_handle_reset on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 --2- 172.21.15.22:0/3204437650 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55914bdb03b0 0x55914c23f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 >> 172.21.15.22:0/3204437650 conn(0x55914bdcd3c0 msgr2=0x55914c1a80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 shutdown_connections 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 1 -- 172.21.15.22:0/3204437650 wait complete. 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb35124d740 20 client.8193 trim_cache size 0 max 16384 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:03.708 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 8055, 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 10.401466750000001, 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 101, 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 109, 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 109, 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 15.026249178 2024-09-21T05:48:03.709 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb330078b90 msgr2=0x7fb33007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb34a207640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb330078b90 0x7fb33007b0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb334033000 tx=0x7fb334004730 comp rx=0 tx=0).stop 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 msgr2=0x7fb3440b0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.702+0000 7fb34a207640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440b0af0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00ba10 tx=0x7fb33c00bed0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 shutdown_connections 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb3441600b0 0x7fb3440b0af0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb3440cf150 0x7fb3441582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb330078b90 0x7fb33007b0a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 --2- 172.21.15.22:0/3176824665 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb3440b7cc0 0x7fb3440a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 >> 172.21.15.22:0/3176824665 conn(0x7fb34406d4f0 msgr2=0x7fb3440b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 shutdown_connections 2024-09-21T05:48:03.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.703+0000 7fb34a207640 1 -- 172.21.15.22:0/3176824665 wait complete. 2024-09-21T05:48:03.716 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.831+0000 7f25680d3640 1 -- 172.21.15.22:0/1826588094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 msgr2=0x7f25600b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.831+0000 7f25680d3640 1 --2- 172.21.15.22:0/1826588094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b3710 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f256008d760 tx=0x7f255c02a540 comp rx=0 tx=0).stop 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.831+0000 7f25680d3640 1 -- 172.21.15.22:0/1826588094 shutdown_connections 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.831+0000 7f25680d3640 1 --2- 172.21.15.22:0/1826588094 >> v2:172.21.15.134:3300/0 conn(0x7f256006e0e0 0x7f256006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.831+0000 7f25680d3640 1 --2- 172.21.15.22:0/1826588094 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b3710 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.831+0000 7f25680d3640 1 -- 172.21.15.22:0/1826588094 >> 172.21.15.22:0/1826588094 conn(0x7f256009d7f0 msgr2=0x7f25600d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.832+0000 7f25680d3640 1 -- 172.21.15.22:0/1826588094 shutdown_connections 2024-09-21T05:48:03.834 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.832+0000 7f25680d3640 1 -- 172.21.15.22:0/1826588094 wait complete. 2024-09-21T05:48:03.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.833+0000 7f25680d3640 1 Processor -- start 2024-09-21T05:48:03.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.833+0000 7f25680d3640 1 -- start start 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f256006e0e0 0x7f256015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25600b5fa0 0x7f25601529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f256006ea30 con 0x7f25600b9970 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f25600d5010 con 0x7f25600b5fa0 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f256006ebb0 con 0x7f256006e0e0 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:50720/0 (socket says 172.21.15.22:50720) 2024-09-21T05:48:03.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 -- 172.21.15.22:0/1660586821 learned_addr learned my addr 172.21.15.22:0/1660586821 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f256566e640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25600b5fa0 0x7f25601529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2565e6f640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f256006e0e0 0x7f256015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 -- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f256006e0e0 msgr2=0x7f256015efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f256006e0e0 0x7f256015efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 -- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25600b5fa0 msgr2=0x7f25601529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25600b5fa0 0x7f25601529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 -- 172.21.15.22:0/1660586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25601e84c0 con 0x7f25600b9970 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f2566670640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b1fa0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f255800b9a0 tx=0x7f255800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558019070 con 0x7f25600b9970 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f255800ccb0 con 0x7f25600b9970 2024-09-21T05:48:03.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25601e8930 con 0x7f25600b9970 2024-09-21T05:48:03.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580146a0 con 0x7f25600b9970 2024-09-21T05:48:03.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.835+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25600c98d0 con 0x7f25600b9970 2024-09-21T05:48:03.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.836+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f256006f080 con 0x7f25600b9970 2024-09-21T05:48:03.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.837+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f255801c070 con 0x7f25600b9970 2024-09-21T05:48:03.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.838+0000 7f25567fc640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2544078bc0 0x7f254407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:03.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.838+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5767+0+0 (secure 0 0 0) 0x7f255809aa50 con 0x7f25600b9970 2024-09-21T05:48:03.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.838+0000 7f2565e6f640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2544078bc0 0x7f254407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:03.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.838+0000 7f2565e6f640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2544078bc0 0x7f254407b0d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f255c034000 tx=0x7f255c004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:03.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:03.842+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2558062ee0 con 0x7f25600b9970 2024-09-21T05:48:04.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.070+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f25601e8690 con 0x7f25600b9970 2024-09-21T05:48:04.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.070+0000 7f25567fc640 1 -- 172.21.15.22:0/1660586821 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7f25580627e0 con 0x7f25600b9970 2024-09-21T05:48:04.072 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2544078bc0 msgr2=0x7f254407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2544078bc0 0x7f254407b0d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f255c034000 tx=0x7f255c004730 comp rx=0 tx=0).stop 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 msgr2=0x7f25600b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b1fa0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f255800b9a0 tx=0x7f255800be60 comp rx=0 tx=0).stop 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 shutdown_connections 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f25600b9970 0x7f25600b1fa0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f25600b5fa0 0x7f25601529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:04.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2544078bc0 0x7f254407b0d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:04.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 --2- 172.21.15.22:0/1660586821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f256006e0e0 0x7f256015efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:04.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 >> 172.21.15.22:0/1660586821 conn(0x7f256009d7f0 msgr2=0x7f25600b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:04.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 shutdown_connections 2024-09-21T05:48:04.076 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04.073+0000 7f25680d3640 1 -- 172.21.15.22:0/1660586821 wait complete. 2024-09-21T05:48:04.086 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:48:04.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:48:04.087 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:04.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.087 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:48:04.103 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:48:04.103 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42260sec preferred_lft 42260sec 2024-09-21T05:48:04.104 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.105 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:04.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.106 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:48:04.168 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:04.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.169 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:48:04.227 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:48:04.227 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:48:04.227 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:48:04.227 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:48:04.227 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:48:04.227 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:48:04.227 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:48:04.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:48:04.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:48:04.227 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:48:04.228 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:48:04.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:48:04.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:04.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.356 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:48:04.372 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:48:04.372 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:48:04.372 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:04.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.372 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:48:04.437 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:04.437 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:04.494 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:04.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:04.494 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:04.549 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:48:04.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.550 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:04.645 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:04.645 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:04.645 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:04.656 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:04.656 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:04.657 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:04.657 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:04.657 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:04.669 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:04.669 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:04.670 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:04.670 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:04.670 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:04.670 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:48:04.670 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:48:04.670 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:48:04.671 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:48:04.671 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:48:04.671 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:48:04.671 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:48:04.671 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:48:04.672 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:48:04.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:04.699 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:48:04.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:48:04.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:04.808 INFO:teuthology.orchestra.run.smithi022.stdout:client6851 2024-09-21T05:48:04.809 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:04.809 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:04.825 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:04.825 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:04.825 DEBUG:teuthology.orchestra.run.smithi022:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-09-21T05:48:04.825 DEBUG:teuthology.orchestra.run.smithi022:> parent = os.path.join(mount_point, "parent") 2024-09-21T05:48:04.825 DEBUG:teuthology.orchestra.run.smithi022:> os.mkdir(parent) 2024-09-21T05:48:04.826 DEBUG:teuthology.orchestra.run.smithi022:> flushed = os.path.join(parent, "flushed") 2024-09-21T05:48:04.826 DEBUG:teuthology.orchestra.run.smithi022:> os.mkdir(flushed) 2024-09-21T05:48:04.826 DEBUG:teuthology.orchestra.run.smithi022:> for f in ["alpha", "bravo", "charlie"]: 2024-09-21T05:48:04.826 DEBUG:teuthology.orchestra.run.smithi022:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-09-21T05:48:04.826 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:04.951 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:04.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:04.957 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-21T05:48:05.019 INFO:teuthology.orchestra.run.smithi022.stdout:./ 2024-09-21T05:48:05.020 INFO:teuthology.orchestra.run.smithi022.stdout:./parent 2024-09-21T05:48:05.020 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed 2024-09-21T05:48:05.020 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/charlie 2024-09-21T05:48:05.020 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/alpha 2024-09-21T05:48:05.020 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/bravo 2024-09-21T05:48:05.020 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:05.020 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:05.020 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:05.020 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:05.020 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-21T05:48:05.020 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:05.127 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:05.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:05.136 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:05.136 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:05.136 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:05.136 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:05.136 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-21T05:48:05.136 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:05.244 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:48:05.247 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:05.253 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:05.253 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:05.254 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:05.254 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:05.254 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-21T05:48:05.254 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:05.360 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:48:05.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:05.370 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:05.370 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:05.370 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:05.370 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:05.370 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-21T05:48:05.370 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:05.477 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628280 2024-09-21T05:48:05.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:05.486 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:05.486 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:05.486 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:05.487 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:05.487 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-21T05:48:05.487 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:05.593 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627778 2024-09-21T05:48:05.596 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:05.602 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:05.602 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:05.602 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:05.602 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:05.602 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-09-21T05:48:05.602 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:05.709 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628279 2024-09-21T05:48:05.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:05.717 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:05.717 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:05.773 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:05.773 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:05.830 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:48:05.987 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 -- 172.21.15.22:0/3473264166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81600a0 msgr2=0x7faaa8071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 --2- 172.21.15.22:0/3473264166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81600a0 0x7faaa8071250 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7faaa809f280 tx=0x7faa9802a5f0 comp rx=0 tx=0).stop 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 -- 172.21.15.22:0/3473264166 shutdown_connections 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 --2- 172.21.15.22:0/3473264166 >> v2:172.21.15.134:3300/0 conn(0x7faaa806d4f0 0x7faaa80cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 --2- 172.21.15.22:0/3473264166 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81600a0 0x7faaa8071250 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 -- 172.21.15.22:0/3473264166 >> 172.21.15.22:0/3473264166 conn(0x7faaa80cdcb0 msgr2=0x7faaa80ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 -- 172.21.15.22:0/3473264166 shutdown_connections 2024-09-21T05:48:05.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.985+0000 7faaae5ba640 1 -- 172.21.15.22:0/3473264166 wait complete. 2024-09-21T05:48:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.987+0000 7faaae5ba640 1 Processor -- start 2024-09-21T05:48:05.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.987+0000 7faaae5ba640 1 -- start start 2024-09-21T05:48:05.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaae5ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaa80ce7d0 0x7faaa81f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaae5ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa81600a0 0x7faaa80cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaae5ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 0x7faaa80c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaae5ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faaa80d0370 con 0x7faaa81f5560 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaae5ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faaa80d01f0 con 0x7faaa80ce7d0 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaae5ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faaa80d04f0 con 0x7faaa81600a0 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaacb57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 0x7faaa80c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.988+0000 7faaacb57640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 0x7faaa80c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44814/0 (socket says 172.21.15.22:44814) 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 -- 172.21.15.22:0/4143904964 learned_addr learned my addr 172.21.15.22:0/4143904964 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:05.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaa7fff640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaa80ce7d0 0x7faaa81f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faa9ffff640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa81600a0 0x7faaa80cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 -- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa81600a0 msgr2=0x7faaa80cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa81600a0 0x7faaa80cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 -- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaa80ce7d0 msgr2=0x7faaa81f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaa80ce7d0 0x7faaa81f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 -- 172.21.15.22:0/4143904964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa81f5f90 con 0x7faaa81f5560 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaacb57640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 0x7faaa80c9aa0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7faaa000caa0 tx=0x7faaa000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa000adf0 con 0x7faaa81f5560 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaa000f540 con 0x7faaa81f5560 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa000f990 con 0x7faaa81f5560 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa81f6270 con 0x7faaa81f5560 2024-09-21T05:48:05.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.989+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa80c9580 con 0x7faaa81f5560 2024-09-21T05:48:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.990+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa80d09a0 con 0x7faaa81f5560 2024-09-21T05:48:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.991+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faaa000fb30 con 0x7faaa81f5560 2024-09-21T05:48:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.992+0000 7faaa57fa640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa88078bc0 0x7faa8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.992+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7faaa009b690 con 0x7faaa81f5560 2024-09-21T05:48:05.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.992+0000 7faaa7fff640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa88078bc0 0x7faa8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:05.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.993+0000 7faaa7fff640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa88078bc0 0x7faa8807b0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa98033000 tx=0x7faa98004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:05.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:05.996+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faaa0014030 con 0x7faaa81f5560 2024-09-21T05:48:06.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.193+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faaa80b5da0 con 0x7faaa81f5560 2024-09-21T05:48:06.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.194+0000 7faaa57fa640 1 -- 172.21.15.22:0/4143904964 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) ==== 80+0+0 (secure 0 0 0) 0x7faaa000fe20 con 0x7faaa81f5560 2024-09-21T05:48:06.196 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:48:06.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa88078bc0 msgr2=0x7faa8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa88078bc0 0x7faa8807b0d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa98033000 tx=0x7faa98004730 comp rx=0 tx=0).stop 2024-09-21T05:48:06.198 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 msgr2=0x7faaa80c9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 0x7faaa80c9aa0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7faaa000caa0 tx=0x7faaa000cf60 comp rx=0 tx=0).stop 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 shutdown_connections 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faaa81f5560 0x7faaa80c9aa0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa88078bc0 0x7faa8807b0d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faaa81600a0 0x7faaa80cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 --2- 172.21.15.22:0/4143904964 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faaa80ce7d0 0x7faaa81f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 >> 172.21.15.22:0/4143904964 conn(0x7faaa80cdcb0 msgr2=0x7faaa80ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 shutdown_connections 2024-09-21T05:48:06.199 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.196+0000 7faaae5ba640 1 -- 172.21.15.22:0/4143904964 wait complete. 2024-09-21T05:48:06.209 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:48:06.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:06.210 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:06.263 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:48:06.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:06.263 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:48:06.290 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 -- 172.21.15.22:0/1378156758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc398160f60 msgr2=0x7fc39815da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 --2- 172.21.15.22:0/1378156758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc398160f60 0x7fc39815da00 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fc380005fd0 tx=0x7fc38002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 -- 172.21.15.22:0/1378156758 shutdown_connections 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 --2- 172.21.15.22:0/1378156758 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc398160f60 0x7fc39815da00 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 --2- 172.21.15.22:0/1378156758 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc3980cd3b0 0x7fc398157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 -- 172.21.15.22:0/1378156758 >> 172.21.15.22:0/1378156758 conn(0x7fc39806d550 msgr2=0x7fc398071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 -- 172.21.15.22:0/1378156758 shutdown_connections 2024-09-21T05:48:06.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.448+0000 7fc39d913640 1 -- 172.21.15.22:0/1378156758 wait complete. 2024-09-21T05:48:06.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.449+0000 7fc39d913640 1 Processor -- start 2024-09-21T05:48:06.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.450+0000 7fc39d913640 1 -- start start 2024-09-21T05:48:06.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc39d913640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 0x7fc3980cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc39d913640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3980ced00 0x7fc3980c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc39d913640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc398160f60 0x7fc39814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc39d913640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3980d07e0 con 0x7fc3980cd3b0 2024-09-21T05:48:06.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc39d913640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3980d0660 con 0x7fc3980ced00 2024-09-21T05:48:06.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc39d913640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3980d0960 con 0x7fc398160f60 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 0x7fc3980cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 0x7fc3980cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44826/0 (socket says 172.21.15.22:44826) 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 -- 172.21.15.22:0/2183422349 learned_addr learned my addr 172.21.15.22:0/2183422349 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc38ffff640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3980ced00 0x7fc3980c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 -- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc398160f60 msgr2=0x7fc39814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc3977fe640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc398160f60 0x7fc39814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc398160f60 0x7fc39814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 -- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3980ced00 msgr2=0x7fc3980c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3980ced00 0x7fc3980c7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc396ffd640 1 -- 172.21.15.22:0/2183422349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3980b2ab0 con 0x7fc3980cd3b0 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.451+0000 7fc3977fe640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc398160f60 0x7fc39814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.452+0000 7fc396ffd640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 0x7fc3980cbf00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc38800d750 tx=0x7fc38800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.452+0000 7fc394ff9640 1 -- 172.21.15.22:0/2183422349 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388017070 con 0x7fc3980cd3b0 2024-09-21T05:48:06.455 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.452+0000 7fc394ff9640 1 -- 172.21.15.22:0/2183422349 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc388004310 con 0x7fc3980cd3b0 2024-09-21T05:48:06.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.452+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3980b1cf0 con 0x7fc3980cd3b0 2024-09-21T05:48:06.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.452+0000 7fc394ff9640 1 -- 172.21.15.22:0/2183422349 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38800de60 con 0x7fc3980cd3b0 2024-09-21T05:48:06.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.452+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3980b22b0 con 0x7fc3980cd3b0 2024-09-21T05:48:06.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.454+0000 7fc394ff9640 1 -- 172.21.15.22:0/2183422349 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc388009c30 con 0x7fc3980cd3b0 2024-09-21T05:48:06.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.454+0000 7fc394ff9640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc378078b90 0x7fc37807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.456 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.454+0000 7fc394ff9640 1 -- 172.21.15.22:0/2183422349 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fc38809b160 con 0x7fc3980cd3b0 2024-09-21T05:48:06.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.454+0000 7fc38ffff640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc378078b90 0x7fc37807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.455+0000 7fc38ffff640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc378078b90 0x7fc37807b0a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc380033000 tx=0x7fc380004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.455+0000 7fc3a4882740 1 Processor -- start 2024-09-21T05:48:06.457 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.455+0000 7fc3a4882740 1 -- start start 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x5620166a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016675860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x562016675da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3a4882740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5620165cea60 con 0x5620162936b0 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5620165bde90 con 0x5620162b4390 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5620165c1ca0 con 0x56201628b4d0 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x562016675da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x5620166a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x562016675da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44836/0 (socket says 172.21.15.22:44836) 2024-09-21T05:48:06.460 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 -- 172.21.15.22:0/366754302 learned_addr learned my addr 172.21.15.22:0/366754302 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc38ffff640 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016675860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 -- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 msgr2=0x562016675860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016675860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 -- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 msgr2=0x5620166a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x5620166a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc3977fe640 1 -- 172.21.15.22:0/366754302 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620163967d0 con 0x5620162936b0 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.457+0000 7fc396ffd640 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x5620166a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.458+0000 7fc3977fe640 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x562016675da0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc390001d90 tx=0x7fc39000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530014628875a7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.458+0000 7fc38d7fa640 1 -- 172.21.15.22:0/366754302 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390018070 con 0x5620162936b0 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.458+0000 7fc38d7fa640 1 -- 172.21.15.22:0/366754302 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc390006b50 con 0x5620162936b0 2024-09-21T05:48:06.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.458+0000 7fc38d7fa640 1 -- 172.21.15.22:0/366754302 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39000d980 con 0x5620162936b0 2024-09-21T05:48:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 -- 172.21.15.22:0/366754302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 msgr2=0x562016675da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x562016675da0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc390001d90 tx=0x7fc39000d150 comp rx=0 tx=0).stop 2024-09-21T05:48:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 -- 172.21.15.22:0/366754302 shutdown_connections 2024-09-21T05:48:06.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x562016675da0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016675860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 --2- 172.21.15.22:0/366754302 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x5620166a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 -- 172.21.15.22:0/366754302 >> 172.21.15.22:0/366754302 conn(0x5620162a84e0 msgr2=0x56201665bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 -- 172.21.15.22:0/366754302 shutdown_connections 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.459+0000 7fc3a4882740 1 -- 172.21.15.22:0/366754302 wait complete. 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.460+0000 7fc3a4882740 1 -- start start 2024-09-21T05:48:06.463 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.460+0000 7fc3a4882740 1 Processor -- start 2024-09-21T05:48:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x56201669a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x56201665bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x56201665a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x56201632e740 con 0x5620162936b0 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562016563970 con 0x5620162b4390 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x56201669aca0 con 0x56201628b4d0 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3a4882740 10 client.0 _ll_register_callbacks cb 0x562016572780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x56201665bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x56201665bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44850/0 (socket says 172.21.15.22:44850) 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 -- 172.21.15.22:0/2185953410 learned_addr learned my addr 172.21.15.22:0/2185953410 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc3977fe640 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x56201665a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 -- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 msgr2=0x56201669a760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x56201669a760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 -- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 msgr2=0x56201665a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x56201665a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.461+0000 7fc38ffff640 1 -- 172.21.15.22:0/2185953410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201631f970 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc38ffff640 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x56201665bc00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc38002abc0 tx=0x7fc380009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9428e7354225997c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3767fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3767fc640 1 -- 172.21.15.22:0/2185953410 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38004c070 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3767fc640 1 -- 172.21.15.22:0/2185953410 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fc3800049a0 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc390006b50 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3767fc640 1 -- 172.21.15.22:0/2185953410 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fc380057050 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 10 client.8220 fetch_fsmap learned FSMap version 101 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x5620162fd6a0 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3767fc640 1 -- 172.21.15.22:0/2185953410 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc390006b50 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3767fc640 1 -- 172.21.15.22:0/2185953410 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fc38005a020 con 0x5620162936b0 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 10 client.8220 fetch_fsmap finished waiting for FSMap version 101 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 10 client.8220 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-09-21T05:48:06.467 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 20 client.8220 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:06.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016272f60 0x562016667a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 4 client.8220 mds_command: new command op to 8055 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:48:06.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.462+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 --> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620165d09b0 con 0x562016272f60 2024-09-21T05:48:06.468 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.464+0000 7fc396ffd640 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016272f60 0x562016667a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.465+0000 7fc3767fc640 10 client.8220 ms_handle_connect on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.465+0000 7fc396ffd640 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016272f60 0x562016667a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654ca75481ca4c14 server_cookie=58862085b96b5a23 in_seq=0 out_seq=0 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3767fc640 1 -- 172.21.15.22:0/2185953410 <== mds.0 v2:172.21.15.134:6835/4249862141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620165d09b0 con 0x562016272f60 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3767fc640 10 client.8220 handle_command_reply: tid=0 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 client.8220 shutdown 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 msgr2=0x56201665bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x56201665bc00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc38002abc0 tx=0x7fc380009260 comp rx=0 tx=0).stop 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 shutdown_connections 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016272f60 0x562016667a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.469 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x56201665a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162936b0 0x56201665bc00 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3767fc640 0 client.8220 ms_handle_reset on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 --2- 172.21.15.22:0/2185953410 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x56201669a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 >> 172.21.15.22:0/2185953410 conn(0x5620162a84e0 msgr2=0x562016654e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 shutdown_connections 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 1 -- 172.21.15.22:0/2185953410 wait complete. 2024-09-21T05:48:06.470 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.466+0000 7fc3a4882740 20 client.8220 trim_cache size 0 max 16384 2024-09-21T05:48:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.481+0000 7fc3a4882740 1 Processor -- start 2024-09-21T05:48:06.483 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.481+0000 7fc3a4882740 1 -- start start 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 0x562016655af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5620162936b0 0x56201665b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x562016673aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3a4882740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc3980b3d20 con 0x56201628b4d0 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981f8080 con 0x5620162b4390 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc3981f73e0 con 0x5620162936b0 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 0x562016655af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 0x562016655af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44860/0 (socket says 172.21.15.22:44860) 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 -- 172.21.15.22:0/236928320 learned_addr learned my addr 172.21.15.22:0/236928320 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3977fe640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x562016673aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc38ffff640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5620162936b0 0x56201665b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 -- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5620162936b0 msgr2=0x56201665b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5620162936b0 0x56201665b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 -- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 msgr2=0x562016673aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x562016673aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 -- 172.21.15.22:0/236928320 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201665cc50 con 0x56201628b4d0 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc3977fe640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x562016673aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc38ffff640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5620162936b0 0x56201665b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.483+0000 7fc396ffd640 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 0x562016655af0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc388062e30 tx=0x7fc38806a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709b034c5f80d55d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc38cff9640 1 -- 172.21.15.22:0/236928320 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38804e070 con 0x56201628b4d0 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc38cff9640 1 -- 172.21.15.22:0/236928320 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3880679d0 con 0x56201628b4d0 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc38cff9640 1 -- 172.21.15.22:0/236928320 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38807d1b0 con 0x56201628b4d0 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 -- 172.21.15.22:0/236928320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 msgr2=0x562016655af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 0x562016655af0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fc388062e30 tx=0x7fc38806a040 comp rx=0 tx=0).stop 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 -- 172.21.15.22:0/236928320 shutdown_connections 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162b4390 0x562016673aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5620162936b0 0x56201665b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 --2- 172.21.15.22:0/236928320 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x56201628b4d0 0x562016655af0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 -- 172.21.15.22:0/236928320 >> 172.21.15.22:0/236928320 conn(0x5620162a84e0 msgr2=0x5620166c3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 -- 172.21.15.22:0/236928320 shutdown_connections 2024-09-21T05:48:06.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.484+0000 7fc3a4882740 1 -- 172.21.15.22:0/236928320 wait complete. 2024-09-21T05:48:06.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.485+0000 7fc3a4882740 1 -- start start 2024-09-21T05:48:06.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.485+0000 7fc3a4882740 1 Processor -- start 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016720540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162936b0 0x5620166b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 0x5620166b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5620165c41d0 con 0x5620162b4390 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc3981f7a30 con 0x5620162936b0 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562016675650 con 0x56201628b4d0 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3a4882740 10 client.0 _ll_register_callbacks cb 0x7fc3800056a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 0x5620166b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 0x5620166b1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44870/0 (socket says 172.21.15.22:44870) 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 -- 172.21.15.22:0/3897946139 learned_addr learned my addr 172.21.15.22:0/3897946139 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc396ffd640 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016720540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 -- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 msgr2=0x562016720540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016720540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.492 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 -- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162936b0 msgr2=0x5620166b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162936b0 0x5620166b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 -- 172.21.15.22:0/3897946139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201665b740 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3977fe640 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 0x5620166b1f70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc390001d60 tx=0x7fc39000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540ad68756eab679 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.487+0000 7fc3767fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3767fc640 1 -- 172.21.15.22:0/3897946139 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390018040 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3767fc640 1 -- 172.21.15.22:0/3897946139 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fc3900052e0 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3880679d0 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3767fc640 1 -- 172.21.15.22:0/3897946139 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fc39002c050 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 10 client.8226 fetch_fsmap learned FSMap version 101 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x5620166ae5c0 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3767fc640 1 -- 172.21.15.22:0/3897946139 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3880679d0 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3767fc640 1 -- 172.21.15.22:0/3897946139 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fc39002f020 con 0x5620162b4390 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 10 client.8226 fetch_fsmap finished waiting for FSMap version 101 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 10 client.8226 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-09-21T05:48:06.493 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 20 client.8226 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016720a80 0x5620166c2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 4 client.8226 mds_command: new command op to 8055 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:48:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 --> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620165cc460 con 0x562016720a80 2024-09-21T05:48:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.488+0000 7fc396ffd640 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016720a80 0x5620166c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.489+0000 7fc3767fc640 10 client.8226 ms_handle_connect on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:06.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.489+0000 7fc396ffd640 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016720a80 0x5620166c2420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2dba1bab769c40 server_cookie=1e8f6099b3e11b84 in_seq=0 out_seq=0 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3767fc640 1 -- 172.21.15.22:0/3897946139 <== mds.0 v2:172.21.15.134:6835/4249862141 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620165cc460 con 0x562016720a80 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3767fc640 10 client.8226 handle_command_reply: tid=0 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 client.8226 shutdown 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 msgr2=0x5620166b1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 0x5620166b1f70 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fc390001d60 tx=0x7fc39000dbb0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 shutdown_connections 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:6835/4249862141,v1:172.21.15.134:6837/4249862141] conn(0x562016720a80 0x5620166c2420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5620162b4390 0x5620166b1f70 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5620162936b0 0x5620166b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3767fc640 0 client.8226 ms_handle_reset on v2:172.21.15.134:6835/4249862141 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.497+0000 7fc3a4882740 1 --2- 172.21.15.22:0/3897946139 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x56201628b4d0 0x562016720540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 >> 172.21.15.22:0/3897946139 conn(0x5620162a84e0 msgr2=0x56201669a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 shutdown_connections 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc3a4882740 1 -- 172.21.15.22:0/3897946139 wait complete. 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc3a4882740 20 client.8226 trim_cache size 0 max 16384 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "", 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0 2024-09-21T05:48:06.501 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:06.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc378078b90 msgr2=0x7fc37807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc378078b90 0x7fc37807b0a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc380033000 tx=0x7fc380004730 comp rx=0 tx=0).stop 2024-09-21T05:48:06.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 msgr2=0x7fc3980cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 0x7fc3980cbf00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc38800d750 tx=0x7fc38800dc10 comp rx=0 tx=0).stop 2024-09-21T05:48:06.502 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 shutdown_connections 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc378078b90 0x7fc37807b0a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc398160f60 0x7fc39814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc3980ced00 0x7fc3980c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 --2- 172.21.15.22:0/2183422349 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc3980cd3b0 0x7fc3980cbf00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 >> 172.21.15.22:0/2183422349 conn(0x7fc39806d550 msgr2=0x7fc3980b6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 shutdown_connections 2024-09-21T05:48:06.503 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.498+0000 7fc39d913640 1 -- 172.21.15.22:0/2183422349 wait complete. 2024-09-21T05:48:06.512 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:48:06.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 -- 172.21.15.22:0/4038155458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde840cfc00 msgr2=0x7fde840ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 --2- 172.21.15.22:0/4038155458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde840cfc00 0x7fde840ce1c0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fde84076550 tx=0x7fde7802a580 comp rx=0 tx=0).stop 2024-09-21T05:48:06.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 -- 172.21.15.22:0/4038155458 shutdown_connections 2024-09-21T05:48:06.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 --2- 172.21.15.22:0/4038155458 >> v2:172.21.15.134:3300/0 conn(0x7fde84073e50 0x7fde84074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 --2- 172.21.15.22:0/4038155458 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde840cfc00 0x7fde840ce1c0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 --2- 172.21.15.22:0/4038155458 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde84153970 0x7fde8415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 -- 172.21.15.22:0/4038155458 >> 172.21.15.22:0/4038155458 conn(0x7fde840d15f0 msgr2=0x7fde84073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 -- 172.21.15.22:0/4038155458 shutdown_connections 2024-09-21T05:48:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.625+0000 7fde8a920640 1 -- 172.21.15.22:0/4038155458 wait complete. 2024-09-21T05:48:06.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.627+0000 7fde8a920640 1 Processor -- start 2024-09-21T05:48:06.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.627+0000 7fde8a920640 1 -- start start 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde8a920640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde84073e50 0x7fde84157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde8a920640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde840cfc00 0x7fde8415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde8a920640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 0x7fde8415f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde8a920640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fde8415e0d0 con 0x7fde84153970 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde8a920640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fde8415df50 con 0x7fde84073e50 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde8a920640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fde8415e250 con 0x7fde840cfc00 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 0x7fde8415f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 0x7fde8415f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44888/0 (socket says 172.21.15.22:44888) 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 -- 172.21.15.22:0/1822049899 learned_addr learned my addr 172.21.15.22:0/1822049899 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 -- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde840cfc00 msgr2=0x7fde8415a150 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde840cfc00 0x7fde8415a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 -- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde84073e50 msgr2=0x7fde84157e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.628+0000 7fde88ebd640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde84073e50 0x7fde84157e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde88ebd640 1 -- 172.21.15.22:0/1822049899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde841f7190 con 0x7fde84153970 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde88ebd640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 0x7fde8415f7b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e9c0 tx=0x7fde7c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c004d20 con 0x7fde84153970 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde7c00cb20 con 0x7fde84153970 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c005020 con 0x7fde84153970 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde841f7470 con 0x7fde84153970 2024-09-21T05:48:06.633 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.629+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde841f7a30 con 0x7fde84153970 2024-09-21T05:48:06.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.630+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde840ce1c0 con 0x7fde84153970 2024-09-21T05:48:06.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.631+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde7c0051c0 con 0x7fde84153970 2024-09-21T05:48:06.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.631+0000 7fde80ff9640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fde70078bc0 0x7fde7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:06.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.631+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7fde7c09b6a0 con 0x7fde84153970 2024-09-21T05:48:06.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.632+0000 7fde83fff640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fde70078bc0 0x7fde7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:06.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.632+0000 7fde83fff640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fde70078bc0 0x7fde7007b0d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde740095d0 tx=0x7fde74006b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:06.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.636+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde7c016030 con 0x7fde84153970 2024-09-21T05:48:06.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.863+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fde8415c750 con 0x7fde84153970 2024-09-21T05:48:06.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.864+0000 7fde80ff9640 1 -- 172.21.15.22:0/1822049899 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7fde7c063aa0 con 0x7fde84153970 2024-09-21T05:48:06.867 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fde70078bc0 msgr2=0x7fde7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fde70078bc0 0x7fde7007b0d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fde740095d0 tx=0x7fde74006b30 comp rx=0 tx=0).stop 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 msgr2=0x7fde8415f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 0x7fde8415f7b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e9c0 tx=0x7fde7c00ee80 comp rx=0 tx=0).stop 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 shutdown_connections 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fde84153970 0x7fde8415f7b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fde70078bc0 0x7fde7007b0d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fde840cfc00 0x7fde8415a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 --2- 172.21.15.22:0/1822049899 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fde84073e50 0x7fde84157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 >> 172.21.15.22:0/1822049899 conn(0x7fde840d15f0 msgr2=0x7fde84152d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 shutdown_connections 2024-09-21T05:48:06.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:06.866+0000 7fde8a920640 1 -- 172.21.15.22:0/1822049899 wait complete. 2024-09-21T05:48:06.880 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:48:06.880 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:48:06.880 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:48:06.880 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:48:06.880 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:06.881 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:48:06.881 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:48:06.881 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:48:06.881 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:06.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:06.881 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:48:06.897 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:48:06.897 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:06.898 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42257sec preferred_lft 42257sec 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:06.899 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:06.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:06.900 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:48:06.962 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:06.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:06.964 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:48:07.020 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:48:07.021 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:48:07.021 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:48:07.021 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:48:07.021 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:48:07.021 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:48:07.021 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:48:07.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:48:07.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:48:07.021 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:48:07.022 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:48:07.110 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:48:07.144 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:07.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:07.150 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:48:07.166 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:48:07.166 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:48:07.166 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:07.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:07.167 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:48:07.233 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:07.233 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:07.290 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:07.290 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:07.290 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:07.346 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:48:07.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:07.346 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:07.441 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:07.441 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:07.441 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:07.452 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:07.452 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:07.452 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:07.452 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:07.452 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:07.459 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:07.459 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:48:07.460 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:48:07.462 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:48:07.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:07.462 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:07.488 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:48:07.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:48:07.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:07.586 INFO:teuthology.orchestra.run.smithi022.stdout:client6866 2024-09-21T05:48:07.586 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:07.586 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:07.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:07.602 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-09-21T05:48:07.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:07.664 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-09-21T05:48:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:8+0 records in 2024-09-21T05:48:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:8+0 records out 2024-09-21T05:48:07.746 INFO:teuthology.orchestra.run.smithi022.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0238265 s, 352 MB/s 2024-09-21T05:48:07.747 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:07.747 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:07.747 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:07.747 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:07.747 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-21T05:48:07.747 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:07.854 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627779 2024-09-21T05:48:07.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:07.863 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:07.863 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:07.863 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:07.863 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:07.863 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-21T05:48:07.863 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:07.969 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627780 2024-09-21T05:48:07.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:07.978 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:07.978 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:08.034 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:08.034 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:08.091 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:48:08.248 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 -- 172.21.15.22:0/3013843175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40d0d50 msgr2=0x7faee40d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 --2- 172.21.15.22:0/3013843175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40d0d50 0x7faee40d1160 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7faedc005fd0 tx=0x7faedc02aa10 comp rx=0 tx=0).stop 2024-09-21T05:48:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 -- 172.21.15.22:0/3013843175 shutdown_connections 2024-09-21T05:48:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 --2- 172.21.15.22:0/3013843175 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40d0d50 0x7faee40d1160 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 -- 172.21.15.22:0/3013843175 >> 172.21.15.22:0/3013843175 conn(0x7faee409d680 msgr2=0x7faee40d4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 -- 172.21.15.22:0/3013843175 shutdown_connections 2024-09-21T05:48:08.249 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.245+0000 7faeea39d640 1 -- 172.21.15.22:0/3013843175 wait complete. 2024-09-21T05:48:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.247+0000 7faeea39d640 1 Processor -- start 2024-09-21T05:48:08.250 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.247+0000 7faeea39d640 1 -- start start 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faeea39d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee40b5520 0x7faee41f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faeea39d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 0x7faee415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faeea39d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee40d0d50 0x7faee41567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faeea39d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faee416a030 con 0x7faee40b71a0 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faeea39d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faee40d1c10 con 0x7faee40b5520 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faeea39d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faee416a1b0 con 0x7faee40d0d50 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.248+0000 7faee8b9a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 0x7faee415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 0x7faee415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44906/0 (socket says 172.21.15.22:44906) 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 -- 172.21.15.22:0/577148310 learned_addr learned my addr 172.21.15.22:0/577148310 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee9b9c640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee40d0d50 0x7faee41567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee939b640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee40b5520 0x7faee41f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.253 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 -- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee40d0d50 msgr2=0x7faee41567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee40d0d50 0x7faee41567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 -- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee40b5520 msgr2=0x7faee41f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee40b5520 0x7faee41f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 -- 172.21.15.22:0/577148310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee415ce90 con 0x7faee40b71a0 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee9b9c640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee40d0d50 0x7faee41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faee8b9a640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 0x7faee415c280 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7faed400c340 tx=0x7faed400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4017070 con 0x7faee40b71a0 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed40053c0 con 0x7faee40b71a0 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4003850 con 0x7faee40b71a0 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.249+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee415d170 con 0x7faee40b71a0 2024-09-21T05:48:08.254 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.250+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee415c7c0 con 0x7faee40b71a0 2024-09-21T05:48:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.250+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee415d690 con 0x7faee40b71a0 2024-09-21T05:48:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.251+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faed401a070 con 0x7faee40b71a0 2024-09-21T05:48:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.252+0000 7faed9ffb640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faec8078b90 0x7faec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.252+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7faed409a110 con 0x7faee40b71a0 2024-09-21T05:48:08.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.252+0000 7faee939b640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faec8078b90 0x7faec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.253+0000 7faee939b640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faec8078b90 0x7faec807b0a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faee000b7f0 tx=0x7faee0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:08.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.257+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faed4062470 con 0x7faee40b71a0 2024-09-21T05:48:08.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.455+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faee415fbe0 con 0x7faee40b71a0 2024-09-21T05:48:08.459 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.456+0000 7faed9ffb640 1 -- 172.21.15.22:0/577148310 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v111) ==== 80+0+0 (secure 0 0 0) 0x7faed4061d70 con 0x7faee40b71a0 2024-09-21T05:48:08.459 INFO:teuthology.orchestra.run.smithi022.stderr:listed 0 entries 2024-09-21T05:48:08.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faec8078b90 msgr2=0x7faec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faec8078b90 0x7faec807b0a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faee000b7f0 tx=0x7faee0008040 comp rx=0 tx=0).stop 2024-09-21T05:48:08.461 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 msgr2=0x7faee415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 0x7faee415c280 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7faed400c340 tx=0x7faed400c800 comp rx=0 tx=0).stop 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 shutdown_connections 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faec8078b90 0x7faec807b0a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faee40d0d50 0x7faee41567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faee40b71a0 0x7faee415c280 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 --2- 172.21.15.22:0/577148310 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faee40b5520 0x7faee41f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 >> 172.21.15.22:0/577148310 conn(0x7faee409d680 msgr2=0x7faee41583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 shutdown_connections 2024-09-21T05:48:08.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.458+0000 7faeea39d640 1 -- 172.21.15.22:0/577148310 wait complete. 2024-09-21T05:48:08.472 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:48:08.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:08.472 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:08.531 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:48:08.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:08.532 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:48:08.564 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:08.564 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:08.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:48:08.621 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:48:08.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:08.621 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:48:08.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:48:08.678 INFO:teuthology.orchestra.run.smithi022.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-21T05:48:08.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 -- 172.21.15.22:0/1574514014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40d1d40 msgr2=0x7f27d40b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 --2- 172.21.15.22:0/1574514014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40d1d40 0x7f27d40b5210 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f27d4074550 tx=0x7f27c402a5f0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 -- 172.21.15.22:0/1574514014 shutdown_connections 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 --2- 172.21.15.22:0/1574514014 >> v2:172.21.15.134:3300/0 conn(0x7f27d415c6f0 0x7f27d40ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 --2- 172.21.15.22:0/1574514014 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40d1d40 0x7f27d40b5210 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 -- 172.21.15.22:0/1574514014 >> 172.21.15.22:0/1574514014 conn(0x7f27d40b3f10 msgr2=0x7f27d40b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.834+0000 7f27da2cc640 1 -- 172.21.15.22:0/1574514014 shutdown_connections 2024-09-21T05:48:08.838 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.835+0000 7f27da2cc640 1 -- 172.21.15.22:0/1574514014 wait complete. 2024-09-21T05:48:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.836+0000 7f27da2cc640 1 Processor -- start 2024-09-21T05:48:08.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.836+0000 7f27da2cc640 1 -- start start 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27da2cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 0x7f27d40b25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27da2cc640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27d40d1d40 0x7f27d40b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27da2cc640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d415c6f0 0x7f27d4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27da2cc640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40d11f0 con 0x7f27d40ccba0 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27da2cc640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27d40d1070 con 0x7f27d415c6f0 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27da2cc640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27d40d1370 con 0x7f27d40d1d40 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 0x7f27d40b25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 0x7f27d40b25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44914/0 (socket says 172.21.15.22:44914) 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 -- 172.21.15.22:0/3397854054 learned_addr learned my addr 172.21.15.22:0/3397854054 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d8869640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d415c6f0 0x7f27d4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27cbfff640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27d40d1d40 0x7f27d40b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 -- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27d40d1d40 msgr2=0x7f27d40b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27d40d1d40 0x7f27d40b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 -- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d415c6f0 msgr2=0x7f27d4156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d415c6f0 0x7f27d4156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 -- 172.21.15.22:0/3397854054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d40b0da0 con 0x7f27d40ccba0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d3fff640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 0x7f27d40b25b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f27c4034040 tx=0x7f27c4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c403e030 con 0x7f27d40ccba0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.838+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27c4003e00 con 0x7f27d40ccba0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.839+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d41f8fb0 con 0x7f27d40ccba0 2024-09-21T05:48:08.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.839+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c4035040 con 0x7f27d40ccba0 2024-09-21T05:48:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.839+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d41f9250 con 0x7f27d40ccba0 2024-09-21T05:48:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.840+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d40d1870 con 0x7f27d40ccba0 2024-09-21T05:48:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.840+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27c4041030 con 0x7f27d40ccba0 2024-09-21T05:48:08.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.841+0000 7f27d17fa640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f27b4078b90 0x7f27b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:08.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.841+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5767+0+0 (secure 0 0 0) 0x7f27c40bb430 con 0x7f27d40ccba0 2024-09-21T05:48:08.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.841+0000 7f27cbfff640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f27b4078b90 0x7f27b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:08.845 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.842+0000 7f27cbfff640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f27b4078b90 0x7f27b407b0a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f27d4159840 tx=0x7f27bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:08.848 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:08.846+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27c4083300 con 0x7f27d40ccba0 2024-09-21T05:48:09.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.070+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f27d415c0c0 con 0x7f27d40ccba0 2024-09-21T05:48:09.073 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:48:09.070+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:48:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.072+0000 7f27d17fa640 1 -- 172.21.15.22:0/3397854054 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 161+0+0 (secure 0 0 0) 0x7f27c4087d30 con 0x7f27d40ccba0 2024-09-21T05:48:09.075 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.074+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f27b4078b90 msgr2=0x7f27b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.074+0000 7f27da2cc640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f27b4078b90 0x7f27b407b0a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f27d4159840 tx=0x7f27bc00b2e0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.074+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 msgr2=0x7f27d40b25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.074+0000 7f27da2cc640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 0x7f27d40b25b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f27c4034040 tx=0x7f27c4003910 comp rx=0 tx=0).stop 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.074+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 shutdown_connections 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f27d415c6f0 0x7f27d4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f27b4078b90 0x7f27b407b0a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f27d40d1d40 0x7f27d40b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 --2- 172.21.15.22:0/3397854054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f27d40ccba0 0x7f27d40b25b0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 >> 172.21.15.22:0/3397854054 conn(0x7f27d40b3f10 msgr2=0x7f27d40b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:09.078 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 shutdown_connections 2024-09-21T05:48:09.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.075+0000 7f27da2cc640 1 -- 172.21.15.22:0/3397854054 wait complete. 2024-09-21T05:48:09.082 INFO:tasks.ceph.mds.b.smithi134.stderr: -9771> 2024-09-21T05:47:55.282+0000 7f8c3e190640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:48:09.090 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:09.092 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:09.135 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-21T05:48:09.135 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:09.136 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:09.137 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on skipped message = False 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-09-21T05:48:09.138 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-09-21T05:48:09.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-09-21T05:48:09.140 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-09-21T05:48:09.141 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.142 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-09-21T05:48:09.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-09-21T05:48:09.144 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = False 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.145 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:09.146 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:09.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:09.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-09-21T05:48:09.150 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.151 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:09.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:09.153 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-09-21T05:48:09.154 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = False 2024-09-21T05:48:09.155 INFO:tasks.ceph.mds.b.smithi134.stdout:starting mds.b at 2024-09-21T05:48:09.157 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:09.198 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-09-21T05:48:09.241 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.242 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.243 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on skipped message = False 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-09-21T05:48:09.244 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-09-21T05:48:09.245 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-09-21T05:48:09.246 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.247 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:09.248 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.249 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-09-21T05:48:09.250 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = False 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = False 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-21T05:48:09.251 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:09.252 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-21T05:48:09.253 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:09.254 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-21T05:48:09.255 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.256 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:09.257 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-21T05:48:09.258 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:09.259 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-21T05:48:09.260 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = False 2024-09-21T05:48:09.261 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = False 2024-09-21T05:48:09.262 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-09-21T05:48:09.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.358+0000 7f3aee281f40 1 Processor -- start 2024-09-21T05:48:09.361 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.358+0000 7f3aee281f40 1 -- start start 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3aee281f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956e9bec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3aee281f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956e9b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3aee281f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956e9bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3aee281f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55956e924430 con 0x55956e9b7f90 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3aee281f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55956e9311f0 con 0x55956ea2d200 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3aee281f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55956e8c5d60 con 0x7f3ad4002ab0 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956e9bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae7d3d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956e9bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956e9bf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44926/0 (socket says 172.21.15.22:44926) 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 -- 172.21.15.22:0/3297596215 learned_addr learned my addr 172.21.15.22:0/3297596215 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae753c640 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956e9b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 -- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 msgr2=0x55956e9bec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956e9bec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.364 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 -- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 msgr2=0x55956e9b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.359+0000 7f3ae853e640 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956e9b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3ae853e640 1 -- 172.21.15.22:0/3297596215 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956e9c0af0 con 0x55956e9b7f90 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3ae853e640 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956e9bf6a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x55956e9793f0 tx=0x7f3ae0009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce71317d9918eb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3ae653a640 1 -- 172.21.15.22:0/3297596215 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000f810 con 0x55956e9b7f90 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3ae653a640 1 -- 172.21.15.22:0/3297596215 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ae0015070 con 0x55956e9b7f90 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3ae653a640 1 -- 172.21.15.22:0/3297596215 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae000fc70 con 0x55956e9b7f90 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3aee281f40 1 -- 172.21.15.22:0/3297596215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 msgr2=0x55956e9bf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3aee281f40 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956e9bf6a0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x55956e9793f0 tx=0x7f3ae0009940 comp rx=0 tx=0).stop 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3aee281f40 1 -- 172.21.15.22:0/3297596215 shutdown_connections 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.360+0000 7f3aee281f40 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956e9bf6a0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956e9b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 --2- 172.21.15.22:0/3297596215 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956e9bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 -- 172.21.15.22:0/3297596215 >> 172.21.15.22:0/3297596215 conn(0x55956e896ac0 msgr2=0x55956ea4d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:09.365 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 -- 172.21.15.22:0/3297596215 shutdown_connections 2024-09-21T05:48:09.366 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 -- 172.21.15.22:0/3297596215 wait complete. 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 Processor -- start 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.361+0000 7f3aee281f40 1 -- start start 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3aee281f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956ea41ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3aee281f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956ea44690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3aee281f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956ea44bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3aee281f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55956ea2d610 con 0x55956e9b7f90 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3aee281f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55956ea4cfe0 con 0x55956ea2d200 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3aee281f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55956eadac10 con 0x7f3ad4002ab0 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956ea41ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956ea41ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44934/0 (socket says 172.21.15.22:44934) 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 -- 172.21.15.22:0/986456249 learned_addr learned my addr 172.21.15.22:0/986456249 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae753c640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956ea44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae853e640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956ea44bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 -- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 msgr2=0x55956ea44bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f3ad4002ab0 0x55956ea44bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 -- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 msgr2=0x55956ea44690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55956ea2d200 0x55956ea44690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 -- 172.21.15.22:0/986456249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956ea2a490 con 0x55956e9b7f90 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.363+0000 7f3ae7d3d640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956ea41ef0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80117f0 tx=0x7f3ad8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.364+0000 7f3ae5538640 1 -- 172.21.15.22:0/986456249 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad801c070 con 0x55956e9b7f90 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.364+0000 7f3ae5538640 1 -- 172.21.15.22:0/986456249 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ad800f6b0 con 0x55956e9b7f90 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.364+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55956eadc280 con 0x55956e9b7f90 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.364+0000 7f3ae5538640 1 -- 172.21.15.22:0/986456249 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800fae0 con 0x55956e9b7f90 2024-09-21T05:48:09.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.364+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956eadc840 con 0x55956e9b7f90 2024-09-21T05:48:09.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.365+0000 7f3ae5538640 1 -- 172.21.15.22:0/986456249 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3ad801f070 con 0x55956e9b7f90 2024-09-21T05:48:09.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.366+0000 7f3ae5538640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ac80775a0 0x7f3ac8079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.370 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.366+0000 7f3ae5538640 1 -- 172.21.15.22:0/986456249 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f3ad80999b0 con 0x55956e9b7f90 2024-09-21T05:48:09.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.366+0000 7f3ae753c640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ac80775a0 0x7f3ac8079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.366+0000 7f3aee281f40 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55956ea4d930 0x55956eae6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.366+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112) -- 0x55956ea4d490 con 0x55956ea4d930 2024-09-21T05:48:09.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.366+0000 7f3ae853e640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55956ea4d930 0x55956eae6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.367+0000 7f3ae753c640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ac80775a0 0x7f3ac8079ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0055000 tx=0x7f3ad0002fb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:09.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.367+0000 7f3ae853e640 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55956ea4d930 0x55956eae6f30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3ae853e640 1 -- 172.21.15.22:0/986456249 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v112'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3ae000f610 con 0x55956ea4d930 2024-09-21T05:48:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55956ea4d930 msgr2=0x55956eae6f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x55956ea4d930 0x55956eae6f30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ac80775a0 msgr2=0x7f3ac8079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3ac80775a0 0x7f3ac8079ab0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0055000 tx=0x7f3ad0002fb0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 msgr2=0x55956ea41ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 --2- 172.21.15.22:0/986456249 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55956e9b7f90 0x55956ea41ef0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80117f0 tx=0x7f3ad8011cb0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3ae853e640 1 -- 172.21.15.22:0/986456249 reap_dead start 2024-09-21T05:48:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 shutdown_connections 2024-09-21T05:48:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 >> 172.21.15.22:0/986456249 conn(0x55956e896ac0 msgr2=0x55956e9bfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 shutdown_connections 2024-09-21T05:48:09.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.368+0000 7f3aee281f40 1 -- 172.21.15.22:0/986456249 wait complete. 2024-09-21T05:48:09.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 -- 172.21.15.22:0/3040875495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 msgr2=0x7fbd080cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 --2- 172.21.15.22:0/3040875495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 0x7fbd080cf3d0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009830 tx=0x7fbd0402a510 comp rx=0 tx=0).stop 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 -- 172.21.15.22:0/3040875495 shutdown_connections 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 --2- 172.21.15.22:0/3040875495 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080cf910 0x7fbd080cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 --2- 172.21.15.22:0/3040875495 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 0x7fbd080cf3d0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 -- 172.21.15.22:0/3040875495 >> 172.21.15.22:0/3040875495 conn(0x7fbd080d1220 msgr2=0x7fbd080d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 -- 172.21.15.22:0/3040875495 shutdown_connections 2024-09-21T05:48:09.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.491+0000 7fbd1007e640 1 -- 172.21.15.22:0/3040875495 wait complete. 2024-09-21T05:48:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.493+0000 7fbd1007e640 1 Processor -- start 2024-09-21T05:48:09.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.493+0000 7fbd1007e640 1 -- start start 2024-09-21T05:48:09.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd1007e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080b4120 0x7fbd0815e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd1007e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 0x7fbd08160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd1007e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 0x7fbd08160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd1007e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fbd080b4c40 con 0x7fbd080cf910 2024-09-21T05:48:09.498 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd1007e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fbd080d3820 con 0x7fbd080b5d30 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd1007e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fbd080b4dc0 con 0x7fbd080b4120 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0e61b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 0x7fbd08160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0de1a640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080b4120 0x7fbd0815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0d619640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 0x7fbd08160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0e61b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 0x7fbd08160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44940/0 (socket says 172.21.15.22:44940) 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0e61b640 1 -- 172.21.15.22:0/2334727655 learned_addr learned my addr 172.21.15.22:0/2334727655 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0de1a640 1 -- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 msgr2=0x7fbd08160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0de1a640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 0x7fbd08160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0de1a640 1 -- 172.21.15.22:0/2334727655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 msgr2=0x7fbd08160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0de1a640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 0x7fbd08160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0de1a640 1 -- 172.21.15.22:0/2334727655 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd080cb080 con 0x7fbd080b4120 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.494+0000 7fbd0e61b640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 0x7fbd08160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.495+0000 7fbd0de1a640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080b4120 0x7fbd0815e170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd04031000 tx=0x7fbd04033ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:09.499 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.495+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04031b50 con 0x7fbd080b4120 2024-09-21T05:48:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.495+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd04031e00 con 0x7fbd080b4120 2024-09-21T05:48:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.495+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd081e8b20 con 0x7fbd080b4120 2024-09-21T05:48:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.495+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04033330 con 0x7fbd080b4120 2024-09-21T05:48:09.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.495+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd081e90e0 con 0x7fbd080b4120 2024-09-21T05:48:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.496+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd080b52e0 con 0x7fbd080b4120 2024-09-21T05:48:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.497+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd040334d0 con 0x7fbd080b4120 2024-09-21T05:48:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.497+0000 7fbcfa7fc640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbcec078b90 0x7fbcec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.497+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fbd040bc000 con 0x7fbd080b4120 2024-09-21T05:48:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.497+0000 7fbd0d619640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbcec078b90 0x7fbcec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:09.501 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.498+0000 7fbd0d619640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbcec078b90 0x7fbcec07b0a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbd080c89c0 tx=0x7fbcfc005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:09.506 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.503+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd0403f030 con 0x7fbd080b4120 2024-09-21T05:48:09.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:09.729+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fbd080d09a0 con 0x7fbd080b4120 2024-09-21T05:48:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.076+0000 7fbcfa7fc640 1 -- 172.21.15.22:0/2334727655 <== mon.2 v2:172.21.15.134:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v103) ==== 162+0+0 (secure 0 0 0) 0x7fbd04083df0 con 0x7fbd080b4120 2024-09-21T05:48:10.079 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T05:48:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbcec078b90 msgr2=0x7fbcec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbcec078b90 0x7fbcec07b0a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fbd080c89c0 tx=0x7fbcfc005f90 comp rx=0 tx=0).stop 2024-09-21T05:48:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080b4120 msgr2=0x7fbd0815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080b4120 0x7fbd0815e170 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbd04031000 tx=0x7fbd04033ba0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 shutdown_connections 2024-09-21T05:48:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fbd080cf910 0x7fbd08160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fbd080b5d30 0x7fbd08160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fbcec078b90 0x7fbcec07b0a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 --2- 172.21.15.22:0/2334727655 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fbd080b4120 0x7fbd0815e170 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 >> 172.21.15.22:0/2334727655 conn(0x7fbd080d1220 msgr2=0x7fbd080b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:10.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 shutdown_connections 2024-09-21T05:48:10.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.079+0000 7fbd1007e640 1 -- 172.21.15.22:0/2334727655 wait complete. 2024-09-21T05:48:10.096 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:10.215 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.211+0000 7ff4f1a70640 1 -- 172.21.15.22:0/3648812887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec15f710 msgr2=0x7ff4ec0afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.211+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/3648812887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec15f710 0x7ff4ec0afa00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec073450 tx=0x7ff4dc02b860 comp rx=0 tx=0).stop 2024-09-21T05:48:10.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.212+0000 7ff4f1a70640 1 -- 172.21.15.22:0/3648812887 shutdown_connections 2024-09-21T05:48:10.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.212+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/3648812887 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec15f710 0x7ff4ec0afa00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.216 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.212+0000 7ff4f1a70640 1 -- 172.21.15.22:0/3648812887 >> 172.21.15.22:0/3648812887 conn(0x7ff4ec0b2320 msgr2=0x7ff4ec0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.212+0000 7ff4f1a70640 1 -- 172.21.15.22:0/3648812887 shutdown_connections 2024-09-21T05:48:10.217 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.212+0000 7ff4f1a70640 1 -- 172.21.15.22:0/3648812887 wait complete. 2024-09-21T05:48:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.213+0000 7ff4f1a70640 1 Processor -- start 2024-09-21T05:48:10.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.214+0000 7ff4f1a70640 1 -- start start 2024-09-21T05:48:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4f1a70640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 0x7ff4ec0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4f1a70640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4ec15f710 0x7ff4ec1f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4f1a70640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec1f6a80 0x7ff4ec1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4f1a70640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec06fad0 con 0x7ff4ec073760 2024-09-21T05:48:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4f1a70640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ec06f950 con 0x7ff4ec1f6a80 2024-09-21T05:48:10.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4f1a70640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ec15b760 con 0x7ff4ec15f710 2024-09-21T05:48:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 0x7ff4ec0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 0x7ff4ec0cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44950/0 (socket says 172.21.15.22:44950) 2024-09-21T05:48:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 -- 172.21.15.22:0/391207666 learned_addr learned my addr 172.21.15.22:0/391207666 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4ebfff640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec1f6a80 0x7ff4ec1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eaffd640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4ec15f710 0x7ff4ec1f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:10.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 -- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4ec15f710 msgr2=0x7ff4ec1f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4ec15f710 0x7ff4ec1f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 -- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec1f6a80 msgr2=0x7ff4ec1f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec1f6a80 0x7ff4ec1f6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 -- 172.21.15.22:0/391207666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ec1f73d0 con 0x7ff4ec073760 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4ebfff640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec1f6a80 0x7ff4ec1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eaffd640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4ec15f710 0x7ff4ec1f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:10.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.215+0000 7ff4eb7fe640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 0x7ff4ec0cf7a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00aa30 tx=0x7ff4dc030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.216+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc040070 con 0x7ff4ec073760 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.216+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ec0af260 con 0x7ff4ec073760 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.216+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4dc004510 con 0x7ff4ec073760 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.216+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc005c90 con 0x7ff4ec073760 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.216+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ec0b3a30 con 0x7ff4ec073760 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.217+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ec070040 con 0x7ff4ec073760 2024-09-21T05:48:10.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.217+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff4dc03a040 con 0x7ff4ec073760 2024-09-21T05:48:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.218+0000 7ff4d3fff640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff4d8078bc0 0x7ff4d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.218+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7ff4dc0bbc60 con 0x7ff4ec073760 2024-09-21T05:48:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.218+0000 7ff4eaffd640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff4d8078bc0 0x7ff4d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:10.227 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.219+0000 7ff4eaffd640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff4d8078bc0 0x7ff4d807b0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec06fec0 tx=0x7ff4e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:10.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.222+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4ec1f73d0 con 0x7ff4ec073760 2024-09-21T05:48:10.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.447+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4ec0ccde0 con 0x7ff4ec073760 2024-09-21T05:48:10.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.448+0000 7ff4d3fff640 1 -- 172.21.15.22:0/391207666 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5227 (secure 0 0 0) 0x7ff4dc084020 con 0x7ff4ec073760 2024-09-21T05:48:10.451 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:10.451 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":104,"btime":"2024-09-21T05:48:10:078027+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:10.078015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6835/3772214991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3772214991},{"type":"v1","addr":"172.21.15.22:6835","nonce":3772214991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-21T05:48:10.452 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 104 2024-09-21T05:48:10.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff4d8078bc0 msgr2=0x7ff4d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.453 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff4d8078bc0 0x7ff4d807b0d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec06fec0 tx=0x7ff4e0009040 comp rx=0 tx=0).stop 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 msgr2=0x7ff4ec0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 0x7ff4ec0cf7a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00aa30 tx=0x7ff4dc030ce0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 shutdown_connections 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff4ec1f6a80 0x7ff4ec1f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff4d8078bc0 0x7ff4d807b0d0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff4ec15f710 0x7ff4ec1f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 --2- 172.21.15.22:0/391207666 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff4ec073760 0x7ff4ec0cf7a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 >> 172.21.15.22:0/391207666 conn(0x7ff4ec0b2320 msgr2=0x7ff4ec155130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 shutdown_connections 2024-09-21T05:48:10.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:10.450+0000 7ff4f1a70640 1 -- 172.21.15.22:0/391207666 wait complete. 2024-09-21T05:48:10.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:48:10.078015+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8049': {'gid': 8049, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.22:6835/3772214991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 3772214991}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 3772214991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:48:10.868 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8049:up:replay 2024-09-21T05:48:11.869 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 -- 172.21.15.22:0/1451604605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 msgr2=0x7f57bc0cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 --2- 172.21.15.22:0/1451604605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 0x7f57bc0cfd20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009830 tx=0x7f57b002a510 comp rx=0 tx=0).stop 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 -- 172.21.15.22:0/1451604605 shutdown_connections 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 --2- 172.21.15.22:0/1451604605 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 0x7f57bc0cfd20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 --2- 172.21.15.22:0/1451604605 >> v2:172.21.15.134:3300/0 conn(0x7f57bc0b5d30 0x7f57bc0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 -- 172.21.15.22:0/1451604605 >> 172.21.15.22:0/1451604605 conn(0x7f57bc0d1220 msgr2=0x7f57bc0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 -- 172.21.15.22:0/1451604605 shutdown_connections 2024-09-21T05:48:12.001 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:11.998+0000 7f57c1571640 1 -- 172.21.15.22:0/1451604605 wait complete. 2024-09-21T05:48:12.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.000+0000 7f57c1571640 1 Processor -- start 2024-09-21T05:48:12.003 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.000+0000 7f57c1571640 1 -- start start 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57c1571640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 0x7f57bc15fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57c1571640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc0b5d30 0x7f57bc0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57c1571640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 0x7f57bc0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57c1571640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc0b4c40 con 0x7f57bc0b4120 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57c1571640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f57bc0d3820 con 0x7f57bc0b5d30 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57c1571640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f57bc0b4dc0 con 0x7f57bc0cf910 2024-09-21T05:48:12.005 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 0x7f57bc15fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57baffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 0x7f57bc15fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44962/0 (socket says 172.21.15.22:44962) 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57baffd640 1 -- 172.21.15.22:0/3184272291 learned_addr learned my addr 172.21.15.22:0/3184272291 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57ba7fc640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc0b5d30 0x7f57bc0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.001+0000 7f57bb7fe640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 0x7f57bc0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57baffd640 1 -- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 msgr2=0x7f57bc0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57baffd640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 0x7f57bc0cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57baffd640 1 -- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc0b5d30 msgr2=0x7f57bc0ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57baffd640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc0b5d30 0x7f57bc0ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57baffd640 1 -- 172.21.15.22:0/3184272291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57bc0a1fb0 con 0x7f57bc0b4120 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57baffd640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 0x7f57bc15fd40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00e790 tx=0x7f57ac00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac00eea0 con 0x7f57bc0b4120 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57ac00cdb0 con 0x7f57bc0b4120 2024-09-21T05:48:12.006 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57bc0a2290 con 0x7f57bc0b4120 2024-09-21T05:48:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac010690 con 0x7f57bc0b4120 2024-09-21T05:48:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.002+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57bc1f6a50 con 0x7f57bc0b4120 2024-09-21T05:48:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.003+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57bc1f6e60 con 0x7f57bc0b4120 2024-09-21T05:48:12.007 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.004+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f57ac010830 con 0x7f57bc0b4120 2024-09-21T05:48:12.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.004+0000 7f57a37fe640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57a8078b90 0x7f57a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:12.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.004+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f57ac09b7a0 con 0x7f57bc0b4120 2024-09-21T05:48:12.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.005+0000 7f57ba7fc640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57a8078b90 0x7f57a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:12.008 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.005+0000 7f57ba7fc640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57a8078b90 0x7f57a807b0a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57b0034000 tx=0x7f57b0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:12.012 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.009+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57ac063480 con 0x7f57bc0b4120 2024-09-21T05:48:12.241 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.238+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57bc0b52e0 con 0x7f57bc0b4120 2024-09-21T05:48:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.239+0000 7f57a37fe640 1 -- 172.21.15.22:0/3184272291 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5227 (secure 0 0 0) 0x7f57ac067eb0 con 0x7f57bc0b4120 2024-09-21T05:48:12.242 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:12.242 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":106,"btime":"2024-09-21T05:48:12:081780+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:11.084096+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":8,"addr":"172.21.15.22:6835/3772214991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3772214991},{"type":"v1","addr":"172.21.15.22:6835","nonce":3772214991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-21T05:48:12.242 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 106 2024-09-21T05:48:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57a8078b90 msgr2=0x7f57a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:12.244 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57a8078b90 0x7f57a807b0a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f57b0034000 tx=0x7f57b0004730 comp rx=0 tx=0).stop 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 msgr2=0x7f57bc15fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 0x7f57bc15fd40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f57ac00e790 tx=0x7f57ac00ec50 comp rx=0 tx=0).stop 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 shutdown_connections 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f57a8078b90 0x7f57a807b0a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f57bc0cf910 0x7f57bc0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f57bc0b5d30 0x7f57bc0ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 --2- 172.21.15.22:0/3184272291 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f57bc0b4120 0x7f57bc15fd40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 >> 172.21.15.22:0/3184272291 conn(0x7f57bc0d1220 msgr2=0x7f57bc06dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 shutdown_connections 2024-09-21T05:48:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:12.241+0000 7f57c1571640 1 -- 172.21.15.22:0/3184272291 wait complete. 2024-09-21T05:48:12.661 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:48:11.084096+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8049': {'gid': 8049, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:rejoin', 'state_seq': 8, 'addr': '172.21.15.22:6835/3772214991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 3772214991}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 3772214991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:48:12.661 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8049:up:rejoin 2024-09-21T05:48:13.662 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:13.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f6386332640 1 -- 172.21.15.22:0/916004679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6368003a00 msgr2=0x7f6368003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f6386332640 1 --2- 172.21.15.22:0/916004679 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6368003a00 0x7f6368003e70 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f637000c290 tx=0x7f6368005830 comp rx=0 tx=0).stop 2024-09-21T05:48:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f63848cf640 1 -- 172.21.15.22:0/916004679 reap_dead start 2024-09-21T05:48:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f6386332640 1 -- 172.21.15.22:0/916004679 shutdown_connections 2024-09-21T05:48:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f6386332640 1 -- 172.21.15.22:0/916004679 >> 172.21.15.22:0/916004679 conn(0x7f638006e060 msgr2=0x7f638006e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f6386332640 1 -- 172.21.15.22:0/916004679 shutdown_connections 2024-09-21T05:48:13.796 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.792+0000 7f6386332640 1 -- 172.21.15.22:0/916004679 wait complete. 2024-09-21T05:48:13.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.794+0000 7f6386332640 1 Processor -- start 2024-09-21T05:48:13.797 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.794+0000 7f6386332640 1 -- start start 2024-09-21T05:48:13.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f638014a390 0x7f63800abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:13.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 0x7f6380159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:13.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638015a290 0x7f63801f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:13.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6380070270 con 0x7f6380147b00 2024-09-21T05:48:13.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f63800700f0 con 0x7f638015a290 2024-09-21T05:48:13.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f63800703f0 con 0x7f638014a390 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 0x7f6380159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 0x7f6380159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:44972/0 (socket says 172.21.15.22:44972) 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 -- 172.21.15.22:0/732586186 learned_addr learned my addr 172.21.15.22:0/732586186 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f63848cf640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638015a290 0x7f63801f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637ffff640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f638014a390 0x7f63800abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 -- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f638014a390 msgr2=0x7f63800abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f638014a390 0x7f63800abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 -- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638015a290 msgr2=0x7f63801f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638015a290 0x7f63801f68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 -- 172.21.15.22:0/732586186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638015a6a0 con 0x7f6380147b00 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637f7fe640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 0x7f6380159d50 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6374003de0 tx=0x7f637400b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374005220 con 0x7f6380147b00 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6374015070 con 0x7f6380147b00 2024-09-21T05:48:13.801 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638014a890 con 0x7f6380147b00 2024-09-21T05:48:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6374005600 con 0x7f6380147b00 2024-09-21T05:48:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.797+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638014ae50 con 0x7f6380147b00 2024-09-21T05:48:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.798+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6380157a10 con 0x7f6380147b00 2024-09-21T05:48:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.799+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6374011830 con 0x7f6380147b00 2024-09-21T05:48:13.802 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.799+0000 7f637cff9640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f636c078b90 0x7f636c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:13.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.800+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f637409b5b0 con 0x7f6380147b00 2024-09-21T05:48:13.803 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.802+0000 7f637ffff640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f636c078b90 0x7f636c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:13.806 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.804+0000 7f637ffff640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f636c078b90 0x7f636c07b0a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6370033000 tx=0x7f637002d000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:13.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:13.805+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6380157a10 con 0x7f6380147b00 2024-09-21T05:48:14.036 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.034+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63800b54f0 con 0x7f6380147b00 2024-09-21T05:48:14.038 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.034+0000 7f637cff9640 1 -- 172.21.15.22:0/732586186 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5234 (secure 0 0 0) 0x7f638015a6a0 con 0x7f6380147b00 2024-09-21T05:48:14.038 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:14.038 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":107,"btime":"2024-09-21T05:48:13:088019+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:13.088009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":9,"addr":"172.21.15.22:6835/3772214991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3772214991},{"type":"v1","addr":"172.21.15.22:6835","nonce":3772214991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8049,"qdb_cluster":[8049]},"id":8}]} 2024-09-21T05:48:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 107 2024-09-21T05:48:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f636c078b90 msgr2=0x7f636c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f636c078b90 0x7f636c07b0a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6370033000 tx=0x7f637002d000 comp rx=0 tx=0).stop 2024-09-21T05:48:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 msgr2=0x7f6380159d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 0x7f6380159d50 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f6374003de0 tx=0x7f637400b980 comp rx=0 tx=0).stop 2024-09-21T05:48:14.039 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 shutdown_connections 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f638015a290 0x7f63801f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6380147b00 0x7f6380159d50 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f636c078b90 0x7f636c07b0a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 --2- 172.21.15.22:0/732586186 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f638014a390 0x7f63800abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 >> 172.21.15.22:0/732586186 conn(0x7f638006e060 msgr2=0x7f63800b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 shutdown_connections 2024-09-21T05:48:14.040 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.037+0000 7f6386332640 1 -- 172.21.15.22:0/732586186 wait complete. 2024-09-21T05:48:14.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:48:13.088009+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8049}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8049': {'gid': 8049, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.22:6835/3772214991', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 3772214991}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 3772214991}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8049, 'qdb_cluster': [8049]} 2024-09-21T05:48:14.453 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:48:14.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 -- 172.21.15.22:0/451581305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63415ebd0 msgr2=0x7fa6340d3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 --2- 172.21.15.22:0/451581305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63415ebd0 0x7fa6340d3ea0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fa624005fd0 tx=0x7fa62402a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 -- 172.21.15.22:0/451581305 shutdown_connections 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 --2- 172.21.15.22:0/451581305 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa63408d3d0 0x7fa634068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 --2- 172.21.15.22:0/451581305 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63415ebd0 0x7fa6340d3ea0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 -- 172.21.15.22:0/451581305 >> 172.21.15.22:0/451581305 conn(0x7fa6340d4820 msgr2=0x7fa6340d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 -- 172.21.15.22:0/451581305 shutdown_connections 2024-09-21T05:48:14.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.578+0000 7fa63a169640 1 -- 172.21.15.22:0/451581305 wait complete. 2024-09-21T05:48:14.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.580+0000 7fa63a169640 1 Processor -- start 2024-09-21T05:48:14.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.580+0000 7fa63a169640 1 -- start start 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa63a169640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 0x7fa6341f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa63a169640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa63408d7e0 0x7fa6340abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa63a169640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa63415ebd0 0x7fa63415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa63a169640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6340cf910 con 0x7fa63408d3d0 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa63a169640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6340cf790 con 0x7fa63408d7e0 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa63a169640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6340cfa90 con 0x7fa63415ebd0 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 0x7fa6341f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 0x7fa6341f5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48674/0 (socket says 172.21.15.22:48674) 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 -- 172.21.15.22:0/1372047449 learned_addr learned my addr 172.21.15.22:0/1372047449 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:14.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa632ffd640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa63408d7e0 0x7fa6340abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa633fff640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa63415ebd0 0x7fa63415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 -- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa63415ebd0 msgr2=0x7fa63415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa63415ebd0 0x7fa63415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 -- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa63408d7e0 msgr2=0x7fa6340abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa63408d7e0 0x7fa6340abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 -- 172.21.15.22:0/1372047449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341f9580 con 0x7fa63408d3d0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa632ffd640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa63408d7e0 0x7fa6340abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa6337fe640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 0x7fa6341f5f70 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa624034040 tx=0x7fa6240038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa61bfff640 1 -- 172.21.15.22:0/1372047449 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62403e030 con 0x7fa63408d3d0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.582+0000 7fa61bfff640 1 -- 172.21.15.22:0/1372047449 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa624003ca0 con 0x7fa63408d3d0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.583+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341f9860 con 0x7fa63408d3d0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.583+0000 7fa61bfff640 1 -- 172.21.15.22:0/1372047449 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624035040 con 0x7fa63408d3d0 2024-09-21T05:48:14.586 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.583+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341f5a50 con 0x7fa63408d3d0 2024-09-21T05:48:14.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.585+0000 7fa61bfff640 1 -- 172.21.15.22:0/1372047449 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa624041030 con 0x7fa63408d3d0 2024-09-21T05:48:14.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.585+0000 7fa61bfff640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa620078b90 0x7fa62007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.585+0000 7fa61bfff640 1 -- 172.21.15.22:0/1372047449 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fa6240bb390 con 0x7fa63408d3d0 2024-09-21T05:48:14.587 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.585+0000 7fa632ffd640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa620078b90 0x7fa62007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.586+0000 7fa632ffd640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa620078b90 0x7fa62007b0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ceda0 tx=0x7fa628005f10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.586+0000 7fa6410d8740 1 Processor -- start 2024-09-21T05:48:14.588 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.586+0000 7fa6410d8740 1 -- start start 2024-09-21T05:48:14.590 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.587+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 0x55658a0f8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589cfe3b0 0x55658a0f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a1163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6410d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55658a041820 con 0x556589d27270 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55658a030c50 con 0x556589cfe3b0 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55658a034a60 con 0x556589d06590 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 0x55658a0f8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 0x55658a0f8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48690/0 (socket says 172.21.15.22:48690) 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 -- 172.21.15.22:0/3080529244 learned_addr learned my addr 172.21.15.22:0/3080529244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa632ffd640 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589cfe3b0 0x55658a0f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa633fff640 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a1163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 -- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 msgr2=0x55658a1163e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a1163e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 -- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589cfe3b0 msgr2=0x55658a0f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589cfe3b0 0x55658a0f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 -- 172.21.15.22:0/3080529244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556589d92850 con 0x556589d27270 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa632ffd640 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589cfe3b0 0x55658a0f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6337fe640 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 0x55658a0f8fa0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b9830 tx=0x7fa62402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce18db1acb518ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6197fa640 1 -- 172.21.15.22:0/3080529244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624048040 con 0x556589d27270 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6197fa640 1 -- 172.21.15.22:0/3080529244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6240837d0 con 0x556589d27270 2024-09-21T05:48:14.592 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.588+0000 7fa6197fa640 1 -- 172.21.15.22:0/3080529244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6240830d0 con 0x556589d27270 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 -- 172.21.15.22:0/3080529244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 msgr2=0x55658a0f8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 0x55658a0f8fa0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa6240b9830 tx=0x7fa62402a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 -- 172.21.15.22:0/3080529244 shutdown_connections 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a1163e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589cfe3b0 0x55658a0f94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 --2- 172.21.15.22:0/3080529244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d27270 0x55658a0f8fa0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 -- 172.21.15.22:0/3080529244 >> 172.21.15.22:0/3080529244 conn(0x556589d1b3c0 msgr2=0x55658a0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.593 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 -- 172.21.15.22:0/3080529244 shutdown_connections 2024-09-21T05:48:14.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.589+0000 7fa6410d8740 1 -- 172.21.15.22:0/3080529244 wait complete. 2024-09-21T05:48:14.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.590+0000 7fa6410d8740 1 -- start start 2024-09-21T05:48:14.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.590+0000 7fa6410d8740 1 Processor -- start 2024-09-21T05:48:14.597 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589cfe3b0 0x55658a109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 0x55658a10e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a0e8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x556589da1620 con 0x556589d06590 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x556589fd65f0 con 0x556589d27270 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55658a122c90 con 0x556589cfe3b0 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6410d8740 10 client.0 _ll_register_callbacks cb 0x556589fda0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 0x55658a10e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 0x55658a10e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48692/0 (socket says 172.21.15.22:48692) 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 -- 172.21.15.22:0/2066328571 learned_addr learned my addr 172.21.15.22:0/2066328571 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa633fff640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a0e8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.598 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6337fe640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589cfe3b0 0x55658a109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 -- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589cfe3b0 msgr2=0x55658a109bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589cfe3b0 0x55658a109bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 -- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 msgr2=0x55658a0e8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a0e8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 -- 172.21.15.22:0/2066328571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556589e096b0 con 0x556589d06590 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6337fe640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589cfe3b0 0x55658a109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa632ffd640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 0x55658a10e420 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa6280114c0 tx=0x7fa628004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974d08fda7e0fa2a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.592+0000 7fa6127fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6127fc640 1 -- 172.21.15.22:0/2066328571 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62801f900 con 0x556589d06590 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6127fc640 1 -- 172.21.15.22:0/2066328571 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fa628028b40 con 0x556589d06590 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6240837d0 con 0x556589d06590 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6127fc640 1 -- 172.21.15.22:0/2066328571 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7fa628028d40 con 0x556589d06590 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 10 client.8286 fetch_fsmap learned FSMap version 107 2024-09-21T05:48:14.599 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55658a122a70 con 0x556589d06590 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6127fc640 1 -- 172.21.15.22:0/2066328571 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6240837d0 con 0x556589d06590 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6127fc640 1 -- 172.21.15.22:0/2066328571 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7fa62801faa0 con 0x556589d06590 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 10 client.8286 fetch_fsmap finished waiting for FSMap version 107 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 10 client.8286 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 20 client.8286 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x556589ce5e40 0x55658a10c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 4 client.8286 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 --> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658a043770 con 0x556589ce5e40 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.593+0000 7fa6337fe640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x556589ce5e40 0x55658a10c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.594+0000 7fa6127fc640 10 client.8286 ms_handle_connect on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.594+0000 7fa6337fe640 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x556589ce5e40 0x55658a10c980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5e5a049f569f158 server_cookie=e6a0169295d73c8c in_seq=0 out_seq=0 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.594+0000 7fa6127fc640 1 -- 172.21.15.22:0/2066328571 <== mds.0 v2:172.21.15.22:6833/3772214991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55658a043770 con 0x556589ce5e40 2024-09-21T05:48:14.600 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.594+0000 7fa6127fc640 10 client.8286 handle_command_reply: tid=0 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.594+0000 7fa6410d8740 1 client.8286 shutdown 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 msgr2=0x55658a10e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 0x55658a10e420 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa6280114c0 tx=0x7fa628004730 comp rx=0 tx=0).stop 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 shutdown_connections 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x556589ce5e40 0x55658a10c980 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a0e8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589d06590 0x55658a10e420 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6127fc640 0 client.8286 ms_handle_reset on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 --2- 172.21.15.22:0/2066328571 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589cfe3b0 0x55658a109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 >> 172.21.15.22:0/2066328571 conn(0x556589d1b3c0 msgr2=0x55658a1159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 shutdown_connections 2024-09-21T05:48:14.601 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 1 -- 172.21.15.22:0/2066328571 wait complete. 2024-09-21T05:48:14.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.595+0000 7fa6410d8740 20 client.8286 trim_cache size 0 max 16384 2024-09-21T05:48:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.613+0000 7fa6410d8740 1 Processor -- start 2024-09-21T05:48:14.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.613+0000 7fa6410d8740 1 -- start start 2024-09-21T05:48:14.617 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a10f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a10ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6410d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa634201540 con 0x556589cfe3b0 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa63415afa0 con 0x556589d27270 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa634200f50 con 0x556589d06590 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a10f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a10f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48700/0 (socket says 172.21.15.22:48700) 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 -- 172.21.15.22:0/681134056 learned_addr learned my addr 172.21.15.22:0/681134056 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 -- 172.21.15.22:0/681134056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 msgr2=0x55658a10ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a10ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 -- 172.21.15.22:0/681134056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 msgr2=0x55658a10b0d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:14.618 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10b0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 -- 172.21.15.22:0/681134056 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658a0c9d70 con 0x556589cfe3b0 2024-09-21T05:48:14.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa6337fe640 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a10f870 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa624080340 tx=0x7fa62408a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1c7bdd13a146bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.615+0000 7fa618ff9640 1 -- 172.21.15.22:0/681134056 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624048040 con 0x556589cfe3b0 2024-09-21T05:48:14.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa618ff9640 1 -- 172.21.15.22:0/681134056 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6240830d0 con 0x556589cfe3b0 2024-09-21T05:48:14.619 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa618ff9640 1 -- 172.21.15.22:0/681134056 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62407f340 con 0x556589cfe3b0 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 -- 172.21.15.22:0/681134056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 msgr2=0x55658a10f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a10f870 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fa624080340 tx=0x7fa62408a040 comp rx=0 tx=0).stop 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 -- 172.21.15.22:0/681134056 shutdown_connections 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 --2- 172.21.15.22:0/681134056 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a10f870 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 -- 172.21.15.22:0/681134056 >> 172.21.15.22:0/681134056 conn(0x556589d1b3c0 msgr2=0x55658a10d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 -- 172.21.15.22:0/681134056 shutdown_connections 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.616+0000 7fa6410d8740 1 -- 172.21.15.22:0/681134056 wait complete. 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.617+0000 7fa6410d8740 1 -- start start 2024-09-21T05:48:14.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.617+0000 7fa6410d8740 1 Processor -- start 2024-09-21T05:48:14.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a0f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a18fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55658a10ebe0 con 0x556589cfe3b0 2024-09-21T05:48:14.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55658a10a270 con 0x556589d27270 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55658a126420 con 0x556589d06590 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6410d8740 10 client.0 _ll_register_callbacks cb 0x7fa628015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a0f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a0f9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48704/0 (socket says 172.21.15.22:48704) 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 -- 172.21.15.22:0/661978741 learned_addr learned my addr 172.21.15.22:0/661978741 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa633fff640 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 -- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 msgr2=0x55658a18fca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a18fca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 -- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 msgr2=0x55658a10ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 -- 172.21.15.22:0/661978741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658a0cc960 con 0x556589cfe3b0 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6337fe640 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a0f9320 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa624083060 tx=0x7fa62402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bc65d87c969359 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6127fc640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:14.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.619+0000 7fa6127fc640 1 -- 172.21.15.22:0/661978741 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62409d390 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6127fc640 1 -- 172.21.15.22:0/661978741 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fa62407f340 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6240830d0 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6127fc640 1 -- 172.21.15.22:0/661978741 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7fa6240a4050 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 10 client.8292 fetch_fsmap learned FSMap version 107 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55658a0cc5d0 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6127fc640 1 -- 172.21.15.22:0/661978741 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6240830d0 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6127fc640 1 -- 172.21.15.22:0/661978741 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7fa62408c5d0 con 0x556589cfe3b0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 10 client.8292 fetch_fsmap finished waiting for FSMap version 107 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 10 client.8292 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 20 client.8292 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55658a195ed0 0x55658a1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 4 client.8292 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 --> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] -- command(tid 0: {"prefix": "status"}) -- 0x55658a03f220 con 0x55658a195ed0 2024-09-21T05:48:14.626 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa632ffd640 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55658a195ed0 0x55658a1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.620+0000 7fa6127fc640 10 client.8292 ms_handle_connect on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa632ffd640 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55658a195ed0 0x55658a1962e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bf0777000bcd65 server_cookie=a1cc4814cdf20694 in_seq=0 out_seq=0 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6127fc640 1 -- 172.21.15.22:0/661978741 <== mds.0 v2:172.21.15.22:6833/3772214991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55658a03f220 con 0x55658a195ed0 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6127fc640 10 client.8292 handle_command_reply: tid=0 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 client.8292 shutdown 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 msgr2=0x55658a0f9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a0f9320 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fa624083060 tx=0x7fa62402a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 shutdown_connections 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55658a195ed0 0x55658a1962e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x556589d27270 0x55658a10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x556589d06590 0x55658a18fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6127fc640 0 client.8292 ms_handle_reset on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 --2- 172.21.15.22:0/661978741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x556589cfe3b0 0x55658a0f9320 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.627 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 >> 172.21.15.22:0/661978741 conn(0x556589d1b3c0 msgr2=0x55658a0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 shutdown_connections 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 1 -- 172.21.15.22:0/661978741 wait complete. 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.621+0000 7fa6410d8740 20 client.8292 trim_cache size 0 max 16384 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 8049, 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:48:14.628 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 4.5423401170000002, 2024-09-21T05:48:14.629 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 107, 2024-09-21T05:48:14.629 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 112, 2024-09-21T05:48:14.629 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 112, 2024-09-21T05:48:14.629 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 25.95104302 2024-09-21T05:48:14.629 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:14.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa620078b90 msgr2=0x7fa62007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa620078b90 0x7fa62007b0a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa6340ceda0 tx=0x7fa628005f10 comp rx=0 tx=0).stop 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 msgr2=0x7fa6341f5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 0x7fa6341f5f70 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fa624034040 tx=0x7fa6240038a0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 shutdown_connections 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa620078b90 0x7fa62007b0a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa63415ebd0 0x7fa63415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa63408d7e0 0x7fa6340abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 --2- 172.21.15.22:0/1372047449 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa63408d3d0 0x7fa6341f5f70 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 >> 172.21.15.22:0/1372047449 conn(0x7fa6340d4820 msgr2=0x7fa6341603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 shutdown_connections 2024-09-21T05:48:14.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.622+0000 7fa63a169640 1 -- 172.21.15.22:0/1372047449 wait complete. 2024-09-21T05:48:14.635 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.750+0000 7f5aa6d62640 1 -- 172.21.15.22:0/4275225880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a880039e0 msgr2=0x7f5a88003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.750+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/4275225880 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5a880039e0 0x7f5a88003e10 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5a90001740 tx=0x7f5a88005830 comp rx=0 tx=0).stop 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.750+0000 7f5aa52ff640 1 -- 172.21.15.22:0/4275225880 reap_dead start 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.750+0000 7f5aa6d62640 1 -- 172.21.15.22:0/4275225880 shutdown_connections 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.751+0000 7f5aa6d62640 1 -- 172.21.15.22:0/4275225880 >> 172.21.15.22:0/4275225880 conn(0x7f5aa00d1220 msgr2=0x7f5aa00d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.751+0000 7f5aa6d62640 1 -- 172.21.15.22:0/4275225880 shutdown_connections 2024-09-21T05:48:14.753 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.751+0000 7f5aa6d62640 1 -- 172.21.15.22:0/4275225880 wait complete. 2024-09-21T05:48:14.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.752+0000 7f5aa6d62640 1 Processor -- start 2024-09-21T05:48:14.754 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.752+0000 7f5aa6d62640 1 -- start start 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa6d62640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a880039e0 0x7f5aa0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa6d62640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01ead70 0x7f5aa00c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa6d62640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 0x7f5aa01e8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa6d62640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa00d2d80 con 0x7f5aa01e7d80 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa6d62640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa00d2c00 con 0x7f5aa01ead70 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa6d62640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa00d2f00 con 0x7f5a880039e0 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 0x7f5aa01e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 0x7f5aa01e8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48734/0 (socket says 172.21.15.22:48734) 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 -- 172.21.15.22:0/434405711 learned_addr learned my addr 172.21.15.22:0/434405711 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:14.757 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5a9ffff640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01ead70 0x7f5aa00c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa4afe640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a880039e0 0x7f5aa0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 -- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a880039e0 msgr2=0x7f5aa0160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a880039e0 0x7f5aa0160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 -- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01ead70 msgr2=0x7f5aa00c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01ead70 0x7f5aa00c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.754+0000 7f5aa52ff640 1 -- 172.21.15.22:0/434405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01eb270 con 0x7f5aa01e7d80 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.755+0000 7f5aa52ff640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 0x7f5aa01e8190 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f5a98000f80 tx=0x7f5a98006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.755+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800fa80 con 0x7f5aa01e7d80 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.755+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a98017070 con 0x7f5aa01e7d80 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.755+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800fea0 con 0x7f5aa01e7d80 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.755+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01e9af0 con 0x7f5aa01e7d80 2024-09-21T05:48:14.758 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.755+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01e9d90 con 0x7f5aa01e7d80 2024-09-21T05:48:14.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.756+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa00767f0 con 0x7f5aa01e7d80 2024-09-21T05:48:14.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.757+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a9801c460 con 0x7f5aa01e7d80 2024-09-21T05:48:14.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.757+0000 7f5a9d7fa640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5a8c078b90 0x7f5a8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:14.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.757+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f5a9809b450 con 0x7f5aa01e7d80 2024-09-21T05:48:14.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.757+0000 7f5aa4afe640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5a8c078b90 0x7f5a8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:14.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.758+0000 7f5aa4afe640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5a8c078b90 0x7f5a8c07b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a90001710 tx=0x7f5a90001270 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:14.764 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.762+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5aa00767f0 con 0x7f5aa01e7d80 2024-09-21T05:48:14.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.988+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5aa00d3460 con 0x7f5aa01e7d80 2024-09-21T05:48:14.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.988+0000 7f5a9d7fa640 1 -- 172.21.15.22:0/434405711 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) ==== 53+0+73 (secure 0 0 0) 0x7f5aa01e9af0 con 0x7f5aa01e7d80 2024-09-21T05:48:14.990 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.990+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5a8c078b90 msgr2=0x7f5a8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.990+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5a8c078b90 0x7f5a8c07b0a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f5a90001710 tx=0x7f5a90001270 comp rx=0 tx=0).stop 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.990+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 msgr2=0x7f5aa01e8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.990+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 0x7f5aa01e8190 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f5a98000f80 tx=0x7f5a98006660 comp rx=0 tx=0).stop 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 shutdown_connections 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f5aa01e7d80 0x7f5aa01e8190 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f5aa01ead70 0x7f5aa00c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5a8c078b90 0x7f5a8c07b0a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 --2- 172.21.15.22:0/434405711 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f5a880039e0 0x7f5aa0160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 >> 172.21.15.22:0/434405711 conn(0x7f5aa00d1220 msgr2=0x7f5aa0161a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 shutdown_connections 2024-09-21T05:48:14.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:14.991+0000 7f5aa6d62640 1 -- 172.21.15.22:0/434405711 wait complete. 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:48:15.004 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:15.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.005 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:15.021 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42249sec preferred_lft 42249sec 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:15.022 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:48:15.023 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:15.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.024 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:48:15.085 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:15.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.087 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:48:15.143 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:48:15.143 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:48:15.143 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:48:15.143 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:48:15.143 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:48:15.144 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:48:15.144 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:48:15.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:48:15.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:48:15.144 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:48:15.144 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:48:15.233 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:48:15.266 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:15.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.273 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:48:15.283 INFO:tasks.ceph.mds.a.smithi022.stderr:2024-09-21T05:48:15.282+0000 7f35bdae3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:48:15.288 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:48:15.288 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:48:15.288 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:15.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.289 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:48:15.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:15.355 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:15.412 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:15.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:15.412 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:15.467 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:48:15.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.467 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:15.564 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:15.564 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:15.564 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:15.575 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:15.575 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:15.575 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:15.575 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:15.575 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:15.581 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:15.581 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:15.581 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:15.581 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:15.581 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:48:15.582 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:48:15.584 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:48:15.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.584 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:15.609 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:48:15.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:48:15.701 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:15.707 INFO:teuthology.orchestra.run.smithi022.stdout:client7624 2024-09-21T05:48:15.707 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:15.707 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:15.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:15.724 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./ 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./parent 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/charlie 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/alpha 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/unflushed 2024-09-21T05:48:15.783 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/unflushed/jfile 2024-09-21T05:48:15.784 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:15.784 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:15.784 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:15.784 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:15.784 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-21T05:48:15.784 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:15.889 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:15.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:15.899 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:15.899 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:15.899 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:15.899 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:15.899 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-21T05:48:15.899 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:16.007 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:48:16.010 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:16.016 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:16.016 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:16.016 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:16.016 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:16.016 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-21T05:48:16.016 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:16.124 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:48:16.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:16.133 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:16.133 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:16.133 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:16.133 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:16.133 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-21T05:48:16.133 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:16.239 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628280 2024-09-21T05:48:16.242 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:16.249 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:16.249 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:16.249 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:16.249 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:16.249 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-21T05:48:16.249 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:16.357 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627778 2024-09-21T05:48:16.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:16.366 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:16.366 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:16.366 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:16.366 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:16.366 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-21T05:48:16.366 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:16.473 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627779 2024-09-21T05:48:16.475 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:16.482 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:16.482 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:16.482 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:16.482 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:16.482 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-21T05:48:16.482 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:16.588 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627780 2024-09-21T05:48:16.591 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:16.597 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:16.597 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:16.653 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:16.653 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:16.710 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:48:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea434bd640 1 -- 172.21.15.22:0/3580349370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 msgr2=0x7fea28004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea434bd640 1 --2- 172.21.15.22:0/3580349370 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea28004c10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fea2c001790 tx=0x7fea28005830 comp rx=0 tx=0).stop 2024-09-21T05:48:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea41a5a640 1 -- 172.21.15.22:0/3580349370 reap_dead start 2024-09-21T05:48:16.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea434bd640 1 -- 172.21.15.22:0/3580349370 shutdown_connections 2024-09-21T05:48:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea434bd640 1 -- 172.21.15.22:0/3580349370 >> 172.21.15.22:0/3580349370 conn(0x7fea3c0d1220 msgr2=0x7fea3c0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea434bd640 1 -- 172.21.15.22:0/3580349370 shutdown_connections 2024-09-21T05:48:16.869 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.866+0000 7fea434bd640 1 -- 172.21.15.22:0/3580349370 wait complete. 2024-09-21T05:48:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.868+0000 7fea434bd640 1 Processor -- start 2024-09-21T05:48:16.870 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.868+0000 7fea434bd640 1 -- start start 2024-09-21T05:48:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea434bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea3c159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea434bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3c15be00 0x7fea3c0b22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:16.872 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea434bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea3c15c210 0x7fea3c0b2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea434bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c167b70 con 0x7fea28004800 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea434bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fea3c1679f0 con 0x7fea3c15be00 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea434bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fea3c167cf0 con 0x7fea3c15c210 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea41259640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea3c159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea41259640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea3c159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48744/0 (socket says 172.21.15.22:48744) 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea41259640 1 -- 172.21.15.22:0/1068648276 learned_addr learned my addr 172.21.15.22:0/1068648276 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea41a5a640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea3c15c210 0x7fea3c0b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.869+0000 7fea40a58640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3c15be00 0x7fea3c0b22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea41259640 1 -- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea3c15c210 msgr2=0x7fea3c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea41259640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea3c15c210 0x7fea3c0b2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea41259640 1 -- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3c15be00 msgr2=0x7fea3c0b22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea41259640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3c15be00 0x7fea3c0b22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea41259640 1 -- 172.21.15.22:0/1068648276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3c1fa0b0 con 0x7fea28004800 2024-09-21T05:48:16.873 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea41259640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea3c159b70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fea38006680 tx=0x7fea380066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:16.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38015070 con 0x7fea28004800 2024-09-21T05:48:16.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea38011840 con 0x7fea28004800 2024-09-21T05:48:16.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38011c90 con 0x7fea28004800 2024-09-21T05:48:16.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3c1f92f0 con 0x7fea28004800 2024-09-21T05:48:16.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.870+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea3c1f98b0 con 0x7fea28004800 2024-09-21T05:48:16.874 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.871+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea3c0cf3d0 con 0x7fea28004800 2024-09-21T05:48:16.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.872+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fea38018070 con 0x7fea28004800 2024-09-21T05:48:16.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.873+0000 7fea31ffb640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fea20078bc0 0x7fea2007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:16.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.873+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fea3809bb00 con 0x7fea28004800 2024-09-21T05:48:16.875 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.873+0000 7fea40a58640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fea20078bc0 0x7fea2007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:16.877 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.875+0000 7fea40a58640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fea20078bc0 0x7fea2007b0d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fea2c032000 tx=0x7fea2c02c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:16.879 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:16.877+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fea3c0d3910 con 0x7fea28004800 2024-09-21T05:48:17.080 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.078+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea3c15f590 con 0x7fea28004800 2024-09-21T05:48:17.081 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.079+0000 7fea31ffb640 1 -- 172.21.15.22:0/1068648276 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v112) ==== 80+0+124 (secure 0 0 0) 0x7fea38063f40 con 0x7fea28004800 2024-09-21T05:48:17.081 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6837/4249862141 2024-09-22T05:48:09.072398+0000 2024-09-21T05:48:17.081 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6835/4249862141 2024-09-22T05:48:09.072398+0000 2024-09-21T05:48:17.081 INFO:teuthology.orchestra.run.smithi022.stderr:listed 2 entries 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fea20078bc0 msgr2=0x7fea2007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fea20078bc0 0x7fea2007b0d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fea2c032000 tx=0x7fea2c02c000 comp rx=0 tx=0).stop 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 msgr2=0x7fea3c159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea3c159b70 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fea38006680 tx=0x7fea380066b0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 shutdown_connections 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fea20078bc0 0x7fea2007b0d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fea3c15c210 0x7fea3c0b2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fea3c15be00 0x7fea3c0b22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 --2- 172.21.15.22:0/1068648276 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fea28004800 0x7fea3c159b70 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.081+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 >> 172.21.15.22:0/1068648276 conn(0x7fea3c0d1220 msgr2=0x7fea3c0b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.082+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 shutdown_connections 2024-09-21T05:48:17.085 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.082+0000 7fea434bd640 1 -- 172.21.15.22:0/1068648276 wait complete. 2024-09-21T05:48:17.095 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:48:17.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:17.095 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:17.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:48:17.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:17.124 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:48:17.179 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.336+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2466245554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 msgr2=0x7fcafc003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.336+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2466245554 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcafc003e70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400c260 tx=0x7fcafc005830 comp rx=0 tx=0).stop 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.336+0000 7fcb1887b640 1 -- 172.21.15.22:0/2466245554 reap_dead start 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.336+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2466245554 shutdown_connections 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.336+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2466245554 >> 172.21.15.22:0/2466245554 conn(0x7fcb1406de80 msgr2=0x7fcb1406e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.337+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2466245554 shutdown_connections 2024-09-21T05:48:17.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.337+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2466245554 wait complete. 2024-09-21T05:48:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.338+0000 7fcb1a2de640 1 Processor -- start 2024-09-21T05:48:17.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.339+0000 7fcb1a2de640 1 -- start start 2024-09-21T05:48:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1a2de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcb140d1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1a2de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1415bd20 0x7fcb141f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1a2de640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb1415c130 0x7fcb141f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1a2de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1406fae0 con 0x7fcafc003a00 2024-09-21T05:48:17.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1a2de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14073e60 con 0x7fcb1415bd20 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1a2de640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1406fc60 con 0x7fcb1415c130 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcb140d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcb140d1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48762/0 (socket says 172.21.15.22:48762) 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 -- 172.21.15.22:0/2427901371 learned_addr learned my addr 172.21.15.22:0/2427901371 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb1887b640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb1415c130 0x7fcb141f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 -- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb1415c130 msgr2=0x7fcb141f6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb137fe640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1415bd20 0x7fcb141f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb1415c130 0x7fcb141f6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 -- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1415bd20 msgr2=0x7fcb141f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1415bd20 0x7fcb141f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb13fff640 1 -- 172.21.15.22:0/2427901371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb141e2fa0 con 0x7fcafc003a00 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.340+0000 7fcb137fe640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1415bd20 0x7fcb141f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.341+0000 7fcb13fff640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcb140d1390 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fcb0402b110 tx=0x7fcb04006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.341+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04031040 con 0x7fcafc003a00 2024-09-21T05:48:17.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.341+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb040065e0 con 0x7fcafc003a00 2024-09-21T05:48:17.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.341+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb141e3280 con 0x7fcafc003a00 2024-09-21T05:48:17.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.341+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb04003a60 con 0x7fcafc003a00 2024-09-21T05:48:17.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.341+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb141e3840 con 0x7fcafc003a00 2024-09-21T05:48:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.342+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb140b4c30 con 0x7fcafc003a00 2024-09-21T05:48:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.343+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcb04034070 con 0x7fcafc003a00 2024-09-21T05:48:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.343+0000 7fcb10ff9640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb00078b90 0x7fcb0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.343+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fcb040bb000 con 0x7fcafc003a00 2024-09-21T05:48:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.344+0000 7fcb137fe640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb00078b90 0x7fcb0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.345+0000 7fcb137fe640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb00078b90 0x7fcb0007b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcb1414cc10 tx=0x7fcb08005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.348+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb04046060 con 0x7fcafc003a00 2024-09-21T05:48:17.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.573+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb140b2ba0 con 0x7fcafc003a00 2024-09-21T05:48:17.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.573+0000 7fcb10ff9640 1 -- 172.21.15.22:0/2427901371 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5234 (secure 0 0 0) 0x7fcb04083490 con 0x7fcafc003a00 2024-09-21T05:48:17.575 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:17.575 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":107,"btime":"2024-09-21T05:48:13:088019+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:13.088009+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8049},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8049":{"gid":8049,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":9,"addr":"172.21.15.22:6835/3772214991","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":3772214991},{"type":"v1","addr":"172.21.15.22:6835","nonce":3772214991}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8049,"qdb_cluster":[8049]},"id":8}]} 2024-09-21T05:48:17.576 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 107 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.575+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb00078b90 msgr2=0x7fcb0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.575+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb00078b90 0x7fcb0007b0a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcb1414cc10 tx=0x7fcb08005f90 comp rx=0 tx=0).stop 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.575+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 msgr2=0x7fcb140d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.575+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcb140d1390 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fcb0402b110 tx=0x7fcb04006d00 comp rx=0 tx=0).stop 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 shutdown_connections 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fcb00078b90 0x7fcb0007b0a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcb1415c130 0x7fcb141f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcb1415bd20 0x7fcb141f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 --2- 172.21.15.22:0/2427901371 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcafc003a00 0x7fcb140d1390 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 >> 172.21.15.22:0/2427901371 conn(0x7fcb1406de80 msgr2=0x7fcb1406cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 shutdown_connections 2024-09-21T05:48:17.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.576+0000 7fcb1a2de640 1 -- 172.21.15.22:0/2427901371 wait complete. 2024-09-21T05:48:17.589 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-09-21T05:48:17.722 INFO:teuthology.orchestra.run.smithi022.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-09-21T05:48:17.722 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-09-21T05:48:17.722 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-09-21T05:48:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.777+0000 7f6b0ab1df40 1 Processor -- start 2024-09-21T05:48:17.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.777+0000 7f6b0ab1df40 1 -- start start 2024-09-21T05:48:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b0ab1df40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e2c480 0x557d43dbd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b0ab1df40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43d76590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b0ab1df40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e239d0 0x557d43db5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b0ab1df40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557d43d22430 con 0x557d43e4ca20 2024-09-21T05:48:17.781 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b0ab1df40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d43d2f1f0 con 0x557d43e2c480 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b0ab1df40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557d43cc3d60 con 0x557d43e239d0 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b03361640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43d76590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b03361640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43d76590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48778/0 (socket says 172.21.15.22:48778) 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.778+0000 7f6b03361640 1 -- 172.21.15.22:0/2692004390 learned_addr learned my addr 172.21.15.22:0/2692004390 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03361640 1 -- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e239d0 msgr2=0x557d43db5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03361640 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e239d0 0x557d43db5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03361640 1 -- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e2c480 msgr2=0x557d43dbd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03b62640 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e2c480 0x557d43dbd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03361640 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e2c480 0x557d43dbd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03361640 1 -- 172.21.15.22:0/2692004390 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d43dbef30 con 0x557d43e4ca20 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03b62640 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e2c480 0x557d43dbd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b03361640 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43d76590 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6aec001de0 tx=0x7f6aec00b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e06b1a3bad7bcdd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.782 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b0235f640 1 -- 172.21.15.22:0/2692004390 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec016070 con 0x557d43e4ca20 2024-09-21T05:48:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b0235f640 1 -- 172.21.15.22:0/2692004390 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6aec00f6b0 con 0x557d43e4ca20 2024-09-21T05:48:17.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.779+0000 7f6b0235f640 1 -- 172.21.15.22:0/2692004390 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00fb40 con 0x557d43e4ca20 2024-09-21T05:48:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/2692004390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 msgr2=0x557d43d76590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43d76590 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6aec001de0 tx=0x7f6aec00b590 comp rx=0 tx=0).stop 2024-09-21T05:48:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/2692004390 shutdown_connections 2024-09-21T05:48:17.785 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e239d0 0x557d43db5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43d76590 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/2692004390 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e2c480 0x557d43dbd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/2692004390 >> 172.21.15.22:0/2692004390 conn(0x557d43c94ac0 msgr2=0x557d43db69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/2692004390 shutdown_connections 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.780+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/2692004390 wait complete. 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.781+0000 7f6b0ab1df40 1 Processor -- start 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.781+0000 7f6b0ab1df40 1 -- start start 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b0ab1df40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e239d0 0x557d43ee12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b0ab1df40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e2c480 0x557d43e48180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b0ab1df40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43ee1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b0ab1df40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x557d43ee2630 con 0x557d43e4ca20 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b0ab1df40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x557d43db5420 con 0x557d43e239d0 2024-09-21T05:48:17.786 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b0ab1df40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x557d43ee0e10 con 0x557d43e2c480 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43ee1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43ee1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48786/0 (socket says 172.21.15.22:48786) 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 -- 172.21.15.22:0/407115480 learned_addr learned my addr 172.21.15.22:0/407115480 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b03b62640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e239d0 0x557d43ee12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b03361640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e2c480 0x557d43e48180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 -- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e2c480 msgr2=0x557d43e48180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x557d43e2c480 0x557d43e48180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 -- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e239d0 msgr2=0x557d43ee12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x557d43e239d0 0x557d43ee12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.782+0000 7f6b04363640 1 -- 172.21.15.22:0/407115480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d43ee07f0 con 0x557d43e4ca20 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.783+0000 7f6b04363640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43ee1910 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6af4046570 tx=0x7f6af404d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.783+0000 7f6b0135d640 1 -- 172.21.15.22:0/407115480 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af4063070 con 0x557d43e4ca20 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.783+0000 7f6b0135d640 1 -- 172.21.15.22:0/407115480 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6af405f620 con 0x557d43e4ca20 2024-09-21T05:48:17.787 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.783+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557d43db5670 con 0x557d43e4ca20 2024-09-21T05:48:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.783+0000 7f6b0135d640 1 -- 172.21.15.22:0/407115480 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af405fa70 con 0x557d43e4ca20 2024-09-21T05:48:17.788 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.783+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d43ee0aa0 con 0x557d43e4ca20 2024-09-21T05:48:17.789 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.784+0000 7f6b0135d640 1 -- 172.21.15.22:0/407115480 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6af4066070 con 0x557d43e4ca20 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.785+0000 7f6b0135d640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ae40775a0 0x7f6ae4079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.785+0000 7f6b0135d640 1 -- 172.21.15.22:0/407115480 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f6af40ddf50 con 0x557d43e4ca20 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.785+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x557d43e215a0 0x557d43e23310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.785+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x557d43eecf60 con 0x557d43e215a0 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.785+0000 7f6b03b62640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ae40775a0 0x7f6ae4079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.785+0000 7f6b03361640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x557d43e215a0 0x557d43e23310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b03361640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x557d43e215a0 0x557d43e23310 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b03b62640 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ae40775a0 0x7f6ae4079ab0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005fd0 tx=0x7f6afc00d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b03361640 1 -- 172.21.15.22:0/407115480 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f6aec00f820 con 0x557d43e215a0 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x557d43e215a0 msgr2=0x557d43e23310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x557d43e215a0 0x557d43e23310 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.790 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ae40775a0 msgr2=0x7f6ae4079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ae40775a0 0x7f6ae4079ab0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005fd0 tx=0x7f6afc00d040 comp rx=0 tx=0).stop 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 msgr2=0x557d43ee1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 --2- 172.21.15.22:0/407115480 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x557d43e4ca20 0x557d43ee1910 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f6af4046570 tx=0x7f6af404d830 comp rx=0 tx=0).stop 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b04363640 1 -- 172.21.15.22:0/407115480 reap_dead start 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 shutdown_connections 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 >> 172.21.15.22:0/407115480 conn(0x557d43c94ac0 msgr2=0x557d43e4acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 shutdown_connections 2024-09-21T05:48:17.791 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.786+0000 7f6b0ab1df40 1 -- 172.21.15.22:0/407115480 wait complete. 2024-09-21T05:48:17.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:48:17.792 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-09-21T05:48:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.847+0000 7f8473d5cf40 1 Processor -- start 2024-09-21T05:48:17.849 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.847+0000 7f8473d5cf40 1 -- start start 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f8473d5cf40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd12b5ce0 0x562fd1355050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f8473d5cf40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd13ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f8473d5cf40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd135ef40 0x562fd12883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f8473d5cf40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562fd1235430 con 0x562fd12b6630 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f8473d5cf40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562fd12421f0 con 0x562fd135ef40 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f8473d5cf40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562fd11d6d60 con 0x562fd12b5ce0 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f846d015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd13ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.852 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f846d015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd13ee6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48802/0 (socket says 172.21.15.22:48802) 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f846d015640 1 -- 172.21.15.22:0/2932813572 learned_addr learned my addr 172.21.15.22:0/2932813572 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f846d816640 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd12b5ce0 0x562fd1355050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.848+0000 7f846e017640 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd135ef40 0x562fd12883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846d015640 1 -- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd12b5ce0 msgr2=0x562fd1355050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846d015640 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd12b5ce0 0x562fd1355050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846d015640 1 -- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd135ef40 msgr2=0x562fd12883c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846d015640 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd135ef40 0x562fd12883c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846d015640 1 -- 172.21.15.22:0/2932813572 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd12b26f0 con 0x562fd12b6630 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846e017640 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd135ef40 0x562fd12883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f846d015640 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd13ee6d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8464046410 tx=0x7f8464002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f251f30ffb385f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f845ffff640 1 -- 172.21.15.22:0/2932813572 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846404ea10 con 0x562fd12b6630 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f845ffff640 1 -- 172.21.15.22:0/2932813572 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f846404eca0 con 0x562fd12b6630 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f845ffff640 1 -- 172.21.15.22:0/2932813572 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84640566d0 con 0x562fd12b6630 2024-09-21T05:48:17.853 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f8473d5cf40 1 -- 172.21.15.22:0/2932813572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 msgr2=0x562fd13ee6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.849+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd13ee6d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8464046410 tx=0x7f8464002fb0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 -- 172.21.15.22:0/2932813572 shutdown_connections 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd135ef40 0x562fd12883c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd13ee6d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/2932813572 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd12b5ce0 0x562fd1355050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 -- 172.21.15.22:0/2932813572 >> 172.21.15.22:0/2932813572 conn(0x562fd11a7ac0 msgr2=0x562fd12d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 -- 172.21.15.22:0/2932813572 shutdown_connections 2024-09-21T05:48:17.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 -- 172.21.15.22:0/2932813572 wait complete. 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.850+0000 7f8473d5cf40 1 Processor -- start 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.851+0000 7f8473d5cf40 1 -- start start 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f8473d5cf40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd12b5ce0 0x562fd12b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f8473d5cf40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd1359f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f8473d5cf40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd135ef40 0x562fd135a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f8473d5cf40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562fd12b29d0 con 0x562fd12b6630 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f8473d5cf40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562fd12b2cb0 con 0x562fd12b5ce0 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f8473d5cf40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562fd12b2f90 con 0x562fd135ef40 2024-09-21T05:48:17.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd1359f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd1359f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48814/0 (socket says 172.21.15.22:48814) 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 -- 172.21.15.22:0/1593884524 learned_addr learned my addr 172.21.15.22:0/1593884524 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846e017640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd135ef40 0x562fd135a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d816640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd12b5ce0 0x562fd12b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 -- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd135ef40 msgr2=0x562fd135a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd135ef40 0x562fd135a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 -- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd12b5ce0 msgr2=0x562fd12b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd12b5ce0 0x562fd12b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 -- 172.21.15.22:0/1593884524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd1350570 con 0x562fd12b6630 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846e017640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562fd135ef40 0x562fd135a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d816640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562fd12b5ce0 0x562fd12b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f846d015640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd1359f00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8464000c00 tx=0x7f846404e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.852+0000 7f845effd640 1 -- 172.21.15.22:0/1593884524 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846406f020 con 0x562fd12b6630 2024-09-21T05:48:17.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.853+0000 7f845effd640 1 -- 172.21.15.22:0/1593884524 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8464056760 con 0x562fd12b6630 2024-09-21T05:48:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.853+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562fd1350850 con 0x562fd12b6630 2024-09-21T05:48:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.853+0000 7f845effd640 1 -- 172.21.15.22:0/1593884524 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464056bb0 con 0x562fd12b6630 2024-09-21T05:48:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.853+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd1353b70 con 0x562fd12b6630 2024-09-21T05:48:17.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.854+0000 7f845effd640 1 -- 172.21.15.22:0/1593884524 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8464072020 con 0x562fd12b6630 2024-09-21T05:48:17.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f845effd640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84500775a0 0x7f8450079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f845effd640 1 -- 172.21.15.22:0/1593884524 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f84640df150 con 0x562fd12b6630 2024-09-21T05:48:17.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f846d816640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84500775a0 0x7f8450079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562fd1353000 0x562fd1353410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x562fd1402080 con 0x562fd1353000 2024-09-21T05:48:17.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f846e017640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562fd1353000 0x562fd1353410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f846e017640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562fd1353000 0x562fd1353410 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.855+0000 7f846d816640 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84500775a0 0x7f8450079ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8460002fa0 tx=0x7f8460002780 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f846e017640 1 -- 172.21.15.22:0/1593884524 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f8468001660 con 0x562fd1353000 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562fd1353000 msgr2=0x562fd1353410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562fd1353000 0x562fd1353410 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84500775a0 msgr2=0x7f8450079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f84500775a0 0x7f8450079ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8460002fa0 tx=0x7f8460002780 comp rx=0 tx=0).stop 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 msgr2=0x562fd1359f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 --2- 172.21.15.22:0/1593884524 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562fd12b6630 0x562fd1359f00 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8464000c00 tx=0x7f846404e890 comp rx=0 tx=0).stop 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f846e017640 1 -- 172.21.15.22:0/1593884524 reap_dead start 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 shutdown_connections 2024-09-21T05:48:17.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 >> 172.21.15.22:0/1593884524 conn(0x562fd11a7ac0 msgr2=0x562fd1355010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 shutdown_connections 2024-09-21T05:48:17.863 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.856+0000 7f8473d5cf40 1 -- 172.21.15.22:0/1593884524 wait complete. 2024-09-21T05:48:17.863 INFO:teuthology.orchestra.run.smithi022.stdout:mytag123 2024-09-21T05:48:17.864 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-09-21T05:48:17.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.917+0000 7fc33782df40 1 Processor -- start 2024-09-21T05:48:17.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.918+0000 7fc33782df40 1 -- start start 2024-09-21T05:48:17.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc33782df40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 0x55970e722a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc33782df40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e72c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc33782df40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e658510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.922 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc33782df40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55970e605430 con 0x55970e658100 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc33782df40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55970e6121f0 con 0x55970e69dc30 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc33782df40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55970e5a6d60 con 0x55970e686630 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e658510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e658510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48820/0 (socket says 172.21.15.22:48820) 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 -- 172.21.15.22:0/1909930448 learned_addr learned my addr 172.21.15.22:0/1909930448 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 -- 172.21.15.22:0/1909930448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 msgr2=0x55970e722a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 0x55970e722a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 -- 172.21.15.22:0/1909930448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 msgr2=0x55970e72c9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e72c9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 -- 172.21.15.22:0/1909930448 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55970e685a90 con 0x55970e658100 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc331ae8640 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e658510 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00a510 tx=0x7fc32c00a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a4efc3bc620858 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.919+0000 7fc32b7fe640 1 -- 172.21.15.22:0/1909930448 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c015070 con 0x55970e658100 2024-09-21T05:48:17.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc32b7fe640 1 -- 172.21.15.22:0/1909930448 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc32c00e710 con 0x55970e658100 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc32b7fe640 1 -- 172.21.15.22:0/1909930448 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00eb60 con 0x55970e658100 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 -- 172.21.15.22:0/1909930448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 msgr2=0x55970e658510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e658510 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00a510 tx=0x7fc32c00a9d0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 -- 172.21.15.22:0/1909930448 shutdown_connections 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e658510 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e72c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 --2- 172.21.15.22:0/1909930448 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 0x55970e722a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 -- 172.21.15.22:0/1909930448 >> 172.21.15.22:0/1909930448 conn(0x55970e577ac0 msgr2=0x55970e725240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 -- 172.21.15.22:0/1909930448 shutdown_connections 2024-09-21T05:48:17.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.920+0000 7fc33782df40 1 -- 172.21.15.22:0/1909930448 wait complete. 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.921+0000 7fc33782df40 1 Processor -- start 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.921+0000 7fc33782df40 1 -- start start 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.922+0000 7fc33782df40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e7c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.922+0000 7fc33782df40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 0x55970e6557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.922+0000 7fc33782df40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e728ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.922+0000 7fc33782df40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55970e683b60 con 0x55970e658100 2024-09-21T05:48:17.926 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.922+0000 7fc33782df40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55970e683e40 con 0x55970e69dc30 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.922+0000 7fc33782df40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55970e72d3a0 con 0x55970e686630 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e7c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e7c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48828/0 (socket says 172.21.15.22:48828) 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 -- 172.21.15.22:0/2666974762 learned_addr learned my addr 172.21.15.22:0/2666974762 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc331ae8640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e728ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc32bfff640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 0x55970e6557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 -- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 msgr2=0x55970e6557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55970e686630 0x55970e6557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 -- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 msgr2=0x55970e728ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e728ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 -- 172.21.15.22:0/2666974762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55970e722ee0 con 0x55970e658100 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc331ae8640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55970e69dc30 0x55970e728ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc3312e7640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e7c4380 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fc324025ec0 tx=0x7fc32404d830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc32a7fc640 1 -- 172.21.15.22:0/2666974762 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324066070 con 0x55970e658100 2024-09-21T05:48:17.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc32a7fc640 1 -- 172.21.15.22:0/2666974762 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc324056780 con 0x55970e658100 2024-09-21T05:48:17.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55970e7c4cb0 con 0x55970e658100 2024-09-21T05:48:17.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc32a7fc640 1 -- 172.21.15.22:0/2666974762 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324056bd0 con 0x55970e658100 2024-09-21T05:48:17.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.923+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970e7c5270 con 0x55970e658100 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.925+0000 7fc32a7fc640 1 -- 172.21.15.22:0/2666974762 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc324054370 con 0x55970e658100 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.926+0000 7fc32a7fc640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc314077570 0x7fc314079a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.926+0000 7fc32a7fc640 1 -- 172.21.15.22:0/2666974762 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7fc3240dd3f0 con 0x55970e658100 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.926+0000 7fc33782df40 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55970e7c5ca0 0x55970e7c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.926+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x55970e655d20 con 0x55970e7c5ca0 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.926+0000 7fc32bfff640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc314077570 0x7fc314079a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.926+0000 7fc331ae8640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55970e7c5ca0 0x55970e7c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc331ae8640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55970e7c5ca0 0x55970e7c60b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc32bfff640 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc314077570 0x7fc314079a80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55970e725670 tx=0x7fc31c003490 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc331ae8640 1 -- 172.21.15.22:0/2666974762 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fc32c00e070 con 0x55970e7c5ca0 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55970e7c5ca0 msgr2=0x55970e7c60b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x55970e7c5ca0 0x55970e7c60b0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc314077570 msgr2=0x7fc314079a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc314077570 0x7fc314079a80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55970e725670 tx=0x7fc31c003490 comp rx=0 tx=0).stop 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 msgr2=0x55970e7c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 --2- 172.21.15.22:0/2666974762 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55970e658100 0x55970e7c4380 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fc324025ec0 tx=0x7fc32404d830 comp rx=0 tx=0).stop 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc331ae8640 1 -- 172.21.15.22:0/2666974762 reap_dead start 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 shutdown_connections 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 >> 172.21.15.22:0/2666974762 conn(0x55970e577ac0 msgr2=0x55970e680eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 shutdown_connections 2024-09-21T05:48:17.931 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.927+0000 7fc33782df40 1 -- 172.21.15.22:0/2666974762 wait complete. 2024-09-21T05:48:17.932 INFO:teuthology.orchestra.run.smithi022.stdout:mytag123 2024-09-21T05:48:17.932 DEBUG:teuthology.orchestra.run.smithi022:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-09-21T05:48:17.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.986+0000 7f854c753f40 1 Processor -- start 2024-09-21T05:48:17.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.987+0000 7f854c753f40 1 -- start start 2024-09-21T05:48:17.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854c753f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec8fa630 0x5608ec916100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854c753f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 0x5608ec99de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854c753f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec9a2a00 0x5608ec9a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854c753f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608ec81ad60 con 0x5608ec911c30 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854c753f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608ec879430 con 0x5608ec8fa630 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854c753f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5608ec8fb3d0 con 0x5608ec9a2a00 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 0x5608ec99de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 0x5608ec99de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48842/0 (socket says 172.21.15.22:48842) 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 -- 172.21.15.22:0/3335660105 learned_addr learned my addr 172.21.15.22:0/3335660105 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854620d640 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec8fa630 0x5608ec916100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f8546a0e640 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec9a2a00 0x5608ec9a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 -- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec9a2a00 msgr2=0x5608ec9a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec9a2a00 0x5608ec9a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 -- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec8fa630 msgr2=0x5608ec916100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec8fa630 0x5608ec916100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 -- 172.21.15.22:0/3335660105 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ec99e390 con 0x5608ec911c30 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f854620d640 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec8fa630 0x5608ec916100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.988+0000 7f853da0c640 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 0x5608ec99de50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f853000b4f0 tx=0x7f853000b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4f2e41319162cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854520b640 1 -- 172.21.15.22:0/3335660105 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85300099d0 con 0x5608ec911c30 2024-09-21T05:48:17.992 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854520b640 1 -- 172.21.15.22:0/3335660105 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8530009b70 con 0x5608ec911c30 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854520b640 1 -- 172.21.15.22:0/3335660105 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530014440 con 0x5608ec911c30 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 -- 172.21.15.22:0/3335660105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 msgr2=0x5608ec99de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 0x5608ec99de50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f853000b4f0 tx=0x7f853000b9b0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 -- 172.21.15.22:0/3335660105 shutdown_connections 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec9a2a00 0x5608ec9a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec911c30 0x5608ec99de50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 --2- 172.21.15.22:0/3335660105 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec8fa630 0x5608ec916100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 -- 172.21.15.22:0/3335660105 >> 172.21.15.22:0/3335660105 conn(0x5608ec7ebac0 msgr2=0x5608ec8f8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 -- 172.21.15.22:0/3335660105 shutdown_connections 2024-09-21T05:48:17.993 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.989+0000 7f854c753f40 1 -- 172.21.15.22:0/3335660105 wait complete. 2024-09-21T05:48:17.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.990+0000 7f854c753f40 1 Processor -- start 2024-09-21T05:48:17.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.990+0000 7f854c753f40 1 -- start start 2024-09-21T05:48:17.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.991+0000 7f854c753f40 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec8fa630 0x5608ec9a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.991+0000 7f854c753f40 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec911c30 0x5608ec9a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.991+0000 7f854c753f40 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec9a2a00 0x5608ec8f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.991+0000 7f854c753f40 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5608ec914a80 con 0x5608ec8fa630 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.991+0000 7f854c753f40 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5608ec99e5c0 con 0x5608ec9a2a00 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.991+0000 7f854c753f40 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5608ec995260 con 0x5608ec911c30 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec8fa630 0x5608ec9a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec8fa630 0x5608ec9a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48856/0 (socket says 172.21.15.22:48856) 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 -- 172.21.15.22:0/2324449423 learned_addr learned my addr 172.21.15.22:0/2324449423 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f8546a0e640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec9a2a00 0x5608ec8f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f853da0c640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec911c30 0x5608ec9a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 -- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec911c30 msgr2=0x5608ec9a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec911c30 0x5608ec9a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 -- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec9a2a00 msgr2=0x5608ec8f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5608ec9a2a00 0x5608ec8f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 -- 172.21.15.22:0/2324449423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608ec9a32c0 con 0x5608ec8fa630 2024-09-21T05:48:17.996 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f853da0c640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5608ec911c30 0x5608ec9a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854620d640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec8fa630 0x5608ec9a1f90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f8538025f00 tx=0x7f853804d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f853ffff640 1 -- 172.21.15.22:0/2324449423 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380566f0 con 0x5608ec8fa630 2024-09-21T05:48:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f853ffff640 1 -- 172.21.15.22:0/2324449423 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8538066070 con 0x5608ec8fa630 2024-09-21T05:48:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f853ffff640 1 -- 172.21.15.22:0/2324449423 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538056c00 con 0x5608ec8fa630 2024-09-21T05:48:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5608ec915030 con 0x5608ec8fa630 2024-09-21T05:48:17.997 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.992+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608ec9a42f0 con 0x5608ec8fa630 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.994+0000 7f853ffff640 1 -- 172.21.15.22:0/2324449423 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f853806b460 con 0x5608ec8fa630 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.995+0000 7f853ffff640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85280775a0 0x7f8528079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.995+0000 7f853ffff640 1 -- 172.21.15.22:0/2324449423 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f85380de0d0 con 0x5608ec8fa630 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.995+0000 7f854c753f40 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x5608eca3d350 0x5608eca3d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.995+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112) -- 0x5608eca3dca0 con 0x5608eca3d350 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.995+0000 7f853da0c640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85280775a0 0x7f8528079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.995+0000 7f8546a0e640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x5608eca3d350 0x5608eca3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f853da0c640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85280775a0 0x7f8528079ab0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5608ec916490 tx=0x7f853000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:17.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f8546a0e640 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x5608eca3d350 0x5608eca3d760 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f8546a0e640 1 -- 172.21.15.22:0/2324449423 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f8540011040 con 0x5608eca3d350 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x5608eca3d350 msgr2=0x5608eca3d760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x5608eca3d350 0x5608eca3d760 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85280775a0 msgr2=0x7f8528079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f85280775a0 0x7f8528079ab0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5608ec916490 tx=0x7f853000d040 comp rx=0 tx=0).stop 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec8fa630 msgr2=0x5608ec9a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 --2- 172.21.15.22:0/2324449423 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5608ec8fa630 0x5608ec9a1f90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f8538025f00 tx=0x7f853804d950 comp rx=0 tx=0).stop 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f8546a0e640 1 -- 172.21.15.22:0/2324449423 reap_dead start 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 shutdown_connections 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 >> 172.21.15.22:0/2324449423 conn(0x5608ec7ebac0 msgr2=0x5608ec913cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 shutdown_connections 2024-09-21T05:48:18.000 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:17.996+0000 7f854c753f40 1 -- 172.21.15.22:0/2324449423 wait complete. 2024-09-21T05:48:18.001 INFO:teuthology.orchestra.run.smithi022.stdout:mytag123 2024-09-21T05:48:18.001 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1260521769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d3d0 msgr2=0x7faa28068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1260521769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d3d0 0x7faa28068d00 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7faa10009830 tx=0x7faa1002a530 comp rx=0 tx=0).stop 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1260521769 shutdown_connections 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1260521769 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d3d0 0x7faa28068d00 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1260521769 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa2815ebd0 0x7faa280d3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1260521769 >> 172.21.15.22:0/1260521769 conn(0x7faa280d4820 msgr2=0x7faa280d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1260521769 shutdown_connections 2024-09-21T05:48:18.121 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.118+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1260521769 wait complete. 2024-09-21T05:48:18.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.120+0000 7faa2c9b6640 1 Processor -- start 2024-09-21T05:48:18.122 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.120+0000 7faa2c9b6640 1 -- start start 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa2c9b6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa2808d3d0 0x7faa2814dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa2c9b6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 0x7faa281f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa2c9b6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa2815ebd0 0x7faa281e8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa2c9b6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa280cf910 con 0x7faa2808d7e0 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa2c9b6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa280cf790 con 0x7faa2808d3d0 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa2c9b6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa280cfa90 con 0x7faa2815ebd0 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 0x7faa281f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 0x7faa281f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48882/0 (socket says 172.21.15.22:48882) 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 -- 172.21.15.22:0/1762276064 learned_addr learned my addr 172.21.15.22:0/1762276064 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 -- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa2815ebd0 msgr2=0x7faa281e8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa2815ebd0 0x7faa281e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 -- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa2808d3d0 msgr2=0x7faa2814dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa2808d3d0 0x7faa2814dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 -- 172.21.15.22:0/1762276064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa281e7670 con 0x7faa2808d7e0 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa25d9b640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 0x7faa281f5c70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7faa1002ab20 tx=0x7faa10033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.122+0000 7faa1effd640 1 -- 172.21.15.22:0/1762276064 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10031d60 con 0x7faa2808d7e0 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.123+0000 7faa1effd640 1 -- 172.21.15.22:0/1762276064 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa10033e10 con 0x7faa2808d7e0 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.123+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa281e7950 con 0x7faa2808d7e0 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.123+0000 7faa1effd640 1 -- 172.21.15.22:0/1762276064 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10005280 con 0x7faa2808d7e0 2024-09-21T05:48:18.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.123+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa281e7f10 con 0x7faa2808d7e0 2024-09-21T05:48:18.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.125+0000 7faa1effd640 1 -- 172.21.15.22:0/1762276064 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa10005420 con 0x7faa2808d7e0 2024-09-21T05:48:18.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.125+0000 7faa1effd640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa08078b90 0x7faa0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.125+0000 7faa1effd640 1 -- 172.21.15.22:0/1762276064 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7faa100bc0d0 con 0x7faa2808d7e0 2024-09-21T05:48:18.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.125+0000 7faa2659c640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa08078b90 0x7faa0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.126+0000 7faa2659c640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa08078b90 0x7faa0807b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faa20009650 tx=0x7faa20006ba0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.126+0000 7faa339ff740 1 Processor -- start 2024-09-21T05:48:18.128 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.126+0000 7faa339ff740 1 -- start start 2024-09-21T05:48:18.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.127+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 0x55a2ebf0ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.127+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebaf64d0 0x55a2ebee0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa339ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebee0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa339ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a2ebe39a60 con 0x55a2ebafe6b0 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2ebe28e90 con 0x55a2ebb1f390 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55a2ebe2cca0 con 0x55a2ebaf64d0 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebee0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebee0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48892/0 (socket says 172.21.15.22:48892) 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 -- 172.21.15.22:0/2971745696 learned_addr learned my addr 172.21.15.22:0/2971745696 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 -- 172.21.15.22:0/2971745696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebaf64d0 msgr2=0x55a2ebee0860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebaf64d0 0x55a2ebee0860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 -- 172.21.15.22:0/2971745696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 msgr2=0x55a2ebf0ba10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 0x55a2ebf0ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 -- 172.21.15.22:0/2971745696 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2ebc017d0 con 0x55a2ebafe6b0 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa26d9d640 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebee0da0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7faa18001c40 tx=0x7faa18002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cbae4d27127353 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa07fff640 1 -- 172.21.15.22:0/2971745696 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18017070 con 0x55a2ebafe6b0 2024-09-21T05:48:18.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa07fff640 1 -- 172.21.15.22:0/2971745696 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa18003da0 con 0x55a2ebafe6b0 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.128+0000 7faa07fff640 1 -- 172.21.15.22:0/2971745696 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1800ca10 con 0x55a2ebafe6b0 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 -- 172.21.15.22:0/2971745696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 msgr2=0x55a2ebee0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebee0da0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7faa18001c40 tx=0x7faa18002100 comp rx=0 tx=0).stop 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 -- 172.21.15.22:0/2971745696 shutdown_connections 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebee0da0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebaf64d0 0x55a2ebee0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 --2- 172.21.15.22:0/2971745696 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 0x55a2ebf0ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 -- 172.21.15.22:0/2971745696 >> 172.21.15.22:0/2971745696 conn(0x55a2ebb134e0 msgr2=0x55a2ebec6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 -- 172.21.15.22:0/2971745696 shutdown_connections 2024-09-21T05:48:18.133 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.129+0000 7faa339ff740 1 -- 172.21.15.22:0/2971745696 wait complete. 2024-09-21T05:48:18.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.130+0000 7faa339ff740 1 -- start start 2024-09-21T05:48:18.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.130+0000 7faa339ff740 1 Processor -- start 2024-09-21T05:48:18.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf1ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebf08570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebb1f390 0x55a2ebed3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.137 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a2ebb99740 con 0x55a2ebafe6b0 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x55a2ebdce970 con 0x55a2ebaf64d0 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55a2ebf1e300 con 0x55a2ebb1f390 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa339ff740 10 client.0 _ll_register_callbacks cb 0x55a2ebddd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebf08570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebf08570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48908/0 (socket says 172.21.15.22:48908) 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 -- 172.21.15.22:0/1805963527 learned_addr learned my addr 172.21.15.22:0/1805963527 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa26d9d640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebb1f390 0x55a2ebed3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa2659c640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf1ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 -- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebb1f390 msgr2=0x55a2ebed3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebb1f390 0x55a2ebed3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 -- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 msgr2=0x55a2ebf1ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf1ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.138 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 -- 172.21.15.22:0/1805963527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ebb8a970 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa25d9b640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebf08570 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7faa10083a80 tx=0x7faa10084350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b14b55fcd089ae server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa057fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa057fa640 1 -- 172.21.15.22:0/1805963527 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10048070 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.132+0000 7faa057fa640 1 -- 172.21.15.22:0/1805963527 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7faa100aa0d0 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa18003da0 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa057fa640 1 -- 172.21.15.22:0/1805963527 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7faa10009450 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 10 client.8367 fetch_fsmap learned FSMap version 107 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55a2ebb686a0 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa057fa640 1 -- 172.21.15.22:0/1805963527 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa18003da0 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa057fa640 1 -- 172.21.15.22:0/1805963527 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7faa1006f170 con 0x55a2ebafe6b0 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 10 client.8367 fetch_fsmap finished waiting for FSMap version 107 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 10 client.8367 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 20 client.8367 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:18.139 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebaddf60 0x55a2ebed4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 4 client.8367 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 --> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2ebe3b9b0 con 0x55a2ebaddf60 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.133+0000 7faa2659c640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebaddf60 0x55a2ebed4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa057fa640 10 client.8367 ms_handle_connect on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa2659c640 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebaddf60 0x55a2ebed4240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4013646df2005879 server_cookie=d629fe61e5531a7f in_seq=0 out_seq=0 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa057fa640 1 -- 172.21.15.22:0/1805963527 <== mds.0 v2:172.21.15.22:6833/3772214991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a2ebe3b9b0 con 0x55a2ebaddf60 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa057fa640 10 client.8367 handle_command_reply: tid=0 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa339ff740 1 client.8367 shutdown 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 msgr2=0x55a2ebf08570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa339ff740 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebf08570 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7faa10083a80 tx=0x7faa10084350 comp rx=0 tx=0).stop 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 shutdown_connections 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.134+0000 7faa339ff740 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebaddf60 0x55a2ebed4240 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebb1f390 0x55a2ebed3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.140 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebafe6b0 0x55a2ebf08570 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 1 --2- 172.21.15.22:0/1805963527 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf1ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa057fa640 0 client.8367 ms_handle_reset on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:18.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 >> 172.21.15.22:0/1805963527 conn(0x55a2ebb134e0 msgr2=0x55a2ebec1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 shutdown_connections 2024-09-21T05:48:18.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 1 -- 172.21.15.22:0/1805963527 wait complete. 2024-09-21T05:48:18.141 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.135+0000 7faa339ff740 20 client.8367 trim_cache size 0 max 16384 2024-09-21T05:48:18.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.150+0000 7faa339ff740 1 Processor -- start 2024-09-21T05:48:18.152 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.150+0000 7faa339ff740 1 -- start start 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf08370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 0x55a2ebec1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa339ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 0x55a2ebec1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa339ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faa28202370 con 0x55a2ebb1f390 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa28201490 con 0x55a2ebaf64d0 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faa2815ac00 con 0x55a2ebafe6b0 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 0x55a2ebec1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 0x55a2ebec1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48920/0 (socket says 172.21.15.22:48920) 2024-09-21T05:48:18.154 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 -- 172.21.15.22:0/3184914307 learned_addr learned my addr 172.21.15.22:0/3184914307 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 -- 172.21.15.22:0/3184914307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 msgr2=0x55a2ebec1a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 0x55a2ebec1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 -- 172.21.15.22:0/3184914307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 msgr2=0x55a2ebf08370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf08370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 -- 172.21.15.22:0/3184914307 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2ebec3860 con 0x55a2ebb1f390 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.151+0000 7faa26d9d640 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 0x55a2ebec1f60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7faa18001c10 tx=0x7faa18001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a629d48b7288f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.152+0000 7faa077fe640 1 -- 172.21.15.22:0/3184914307 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18022070 con 0x55a2ebb1f390 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.152+0000 7faa077fe640 1 -- 172.21.15.22:0/3184914307 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa18006b30 con 0x55a2ebb1f390 2024-09-21T05:48:18.155 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.152+0000 7faa077fe640 1 -- 172.21.15.22:0/3184914307 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18017040 con 0x55a2ebb1f390 2024-09-21T05:48:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 -- 172.21.15.22:0/3184914307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 msgr2=0x55a2ebec1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 0x55a2ebec1f60 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7faa18001c10 tx=0x7faa18001250 comp rx=0 tx=0).stop 2024-09-21T05:48:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 -- 172.21.15.22:0/3184914307 shutdown_connections 2024-09-21T05:48:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebb1f390 0x55a2ebec1f60 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 0x55a2ebec1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 --2- 172.21.15.22:0/3184914307 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf08370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 -- 172.21.15.22:0/3184914307 >> 172.21.15.22:0/3184914307 conn(0x55a2ebb134e0 msgr2=0x55a2ebf1ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 -- 172.21.15.22:0/3184914307 shutdown_connections 2024-09-21T05:48:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.153+0000 7faa339ff740 1 -- 172.21.15.22:0/3184914307 wait complete. 2024-09-21T05:48:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.154+0000 7faa339ff740 1 -- start start 2024-09-21T05:48:18.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.154+0000 7faa339ff740 1 Processor -- start 2024-09-21T05:48:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf8f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.159 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 0x55a2ebf744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 0x55a2ebed3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x55a2ebedf130 con 0x55a2ebaf64d0 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faa2815b250 con 0x55a2ebb1f390 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x55a2ebef0690 con 0x55a2ebafe6b0 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa339ff740 10 client.0 _ll_register_callbacks cb 0x7faa10005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf8f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf8f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48932/0 (socket says 172.21.15.22:48932) 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 -- 172.21.15.22:0/2227030969 learned_addr learned my addr 172.21.15.22:0/2227030969 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 -- 172.21.15.22:0/2227030969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 msgr2=0x55a2ebf744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 0x55a2ebf744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 -- 172.21.15.22:0/2227030969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 msgr2=0x55a2ebed3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 0x55a2ebed3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.155+0000 7faa2659c640 1 -- 172.21.15.22:0/2227030969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ebef1630 con 0x55a2ebaf64d0 2024-09-21T05:48:18.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa2659c640 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf8f360 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x55a2ebed49d0 tx=0x7faa200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad52167d1b3f361b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa057fa640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa057fa640 1 -- 172.21.15.22:0/2227030969 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2002c020 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa057fa640 1 -- 172.21.15.22:0/2227030969 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7faa20020a10 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa18006b30 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa057fa640 1 -- 172.21.15.22:0/2227030969 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7faa20036050 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 10 client.8373 fetch_fsmap learned FSMap version 107 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55a2ebed2ad0 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa057fa640 1 -- 172.21.15.22:0/2227030969 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa18006b30 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa057fa640 1 -- 172.21.15.22:0/2227030969 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7faa20039020 con 0x55a2ebaf64d0 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 10 client.8373 fetch_fsmap finished waiting for FSMap version 107 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 10 client.8373 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 20 client.8373 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebed4100 0x55a2ebf8b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 4 client.8373 mds_command: new command op to 8049 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-21T05:48:18.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.156+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 --> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2ebebca90 con 0x55a2ebed4100 2024-09-21T05:48:18.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.157+0000 7faa25d9b640 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebed4100 0x55a2ebf8b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.157+0000 7faa057fa640 10 client.8373 ms_handle_connect on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:18.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.157+0000 7faa25d9b640 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebed4100 0x55a2ebf8b6b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b25bac4677c50dd server_cookie=5df9a957ebd4626e in_seq=0 out_seq=0 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.166+0000 7faa057fa640 1 -- 172.21.15.22:0/2227030969 <== mds.0 v2:172.21.15.22:6833/3772214991 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2ebebca90 con 0x55a2ebed4100 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.166+0000 7faa057fa640 10 client.8373 handle_command_reply: tid=0 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.166+0000 7faa339ff740 1 client.8373 shutdown 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.166+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 msgr2=0x55a2ebf8f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.166+0000 7faa339ff740 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf8f360 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x55a2ebed49d0 tx=0x7faa200046b0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.166+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 shutdown_connections 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:6833/3772214991,v1:172.21.15.22:6835/3772214991] conn(0x55a2ebed4100 0x55a2ebf8b6b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x55a2ebb1f390 0x55a2ebed3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x55a2ebafe6b0 0x55a2ebf744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa057fa640 0 client.8373 ms_handle_reset on v2:172.21.15.22:6833/3772214991 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 --2- 172.21.15.22:0/2227030969 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x55a2ebaf64d0 0x55a2ebf8f360 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 >> 172.21.15.22:0/2227030969 conn(0x55a2ebb134e0 msgr2=0x55a2ebec1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 shutdown_connections 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 1 -- 172.21.15.22:0/2227030969 wait complete. 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa339ff740 20 client.8373 trim_cache size 0 max 16384 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stdout: "message": "", 2024-09-21T05:48:18.170 INFO:teuthology.orchestra.run.smithi022.stdout: "return_code": 0 2024-09-21T05:48:18.171 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:18.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa08078b90 msgr2=0x7faa0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa08078b90 0x7faa0807b0a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faa20009650 tx=0x7faa20006ba0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.171 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 msgr2=0x7faa281f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 0x7faa281f5c70 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7faa1002ab20 tx=0x7faa10033c20 comp rx=0 tx=0).stop 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 shutdown_connections 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7faa08078b90 0x7faa0807b0a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faa2815ebd0 0x7faa281e8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faa2808d7e0 0x7faa281f5c70 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 --2- 172.21.15.22:0/1762276064 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faa2808d3d0 0x7faa2814dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.167+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 >> 172.21.15.22:0/1762276064 conn(0x7faa280d4820 msgr2=0x7faa28161110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.168+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 shutdown_connections 2024-09-21T05:48:18.172 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.168+0000 7faa2c9b6640 1 -- 172.21.15.22:0/1762276064 wait complete. 2024-09-21T05:48:18.181 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 -- 172.21.15.22:0/1017447054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 msgr2=0x7f88640cfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 --2- 172.21.15.22:0/1017447054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88640cfba0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f884c009830 tx=0x7f884c02a530 comp rx=0 tx=0).stop 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 -- 172.21.15.22:0/1017447054 shutdown_connections 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 --2- 172.21.15.22:0/1017447054 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88640cfba0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 -- 172.21.15.22:0/1017447054 >> 172.21.15.22:0/1017447054 conn(0x7f88640d1850 msgr2=0x7f88640d47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 -- 172.21.15.22:0/1017447054 shutdown_connections 2024-09-21T05:48:18.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.295+0000 7f886975e640 1 -- 172.21.15.22:0/1017447054 wait complete. 2024-09-21T05:48:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.297+0000 7f886975e640 1 Processor -- start 2024-09-21T05:48:18.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.297+0000 7f886975e640 1 -- start start 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f886975e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f886406ea20 0x7f88640c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f886975e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88640b34c0 0x7f88641f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f886975e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88641f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f886975e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f88640d33b0 con 0x7f88640b3a10 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f886975e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f88640d3230 con 0x7f88640b34c0 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f886975e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f88640d3530 con 0x7f886406ea20 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88641f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88641f9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48948/0 (socket says 172.21.15.22:48948) 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 -- 172.21.15.22:0/3741772990 learned_addr learned my addr 172.21.15.22:0/3741772990 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f8862ffd640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f886406ea20 0x7f88640c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 -- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f886406ea20 msgr2=0x7f88640c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.302 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f886406ea20 0x7f88640c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 -- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88640b34c0 msgr2=0x7f88641f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88640b34c0 0x7f88641f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 -- 172.21.15.22:0/3741772990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88641f5eb0 con 0x7f88640b3a10 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f88637fe640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88641f9d30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f885c00c760 tx=0x7f885c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c003b70 con 0x7f88640b3a10 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f885c003d30 con 0x7f88640b3a10 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.299+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c0052d0 con 0x7f88640b3a10 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.300+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88641f6190 con 0x7f88640b3a10 2024-09-21T05:48:18.303 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.300+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88641f6750 con 0x7f88640b3a10 2024-09-21T05:48:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.301+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88640cfba0 con 0x7f88640b3a10 2024-09-21T05:48:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.301+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f885c005470 con 0x7f88640b3a10 2024-09-21T05:48:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.302+0000 7f885b7fe640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8844078b90 0x7f884407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.302+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5853+0+0 (secure 0 0 0) 0x7f885c09a9e0 con 0x7f88640b3a10 2024-09-21T05:48:18.304 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.303+0000 7f8862ffd640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8844078b90 0x7f884407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.305 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.303+0000 7f8862ffd640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8844078b90 0x7f884407b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8854000fb0 tx=0x7f8854009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.307+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f885c062e20 con 0x7f88640b3a10 2024-09-21T05:48:18.534 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.532+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f88640b4510 con 0x7f88640b3a10 2024-09-21T05:48:18.534 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:48:18.533+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:48:18.536 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.534+0000 7f885b7fe640 1 -- 172.21.15.22:0/3741772990 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7f885c062720 con 0x7f88640b3a10 2024-09-21T05:48:18.536 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:48:18.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8844078b90 msgr2=0x7f884407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8844078b90 0x7f884407b0a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8854000fb0 tx=0x7f8854009040 comp rx=0 tx=0).stop 2024-09-21T05:48:18.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 msgr2=0x7f88641f9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88641f9d30 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f885c00c760 tx=0x7f885c00cc20 comp rx=0 tx=0).stop 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 shutdown_connections 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f88640b3a10 0x7f88641f9d30 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f88640b34c0 0x7f88641f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8844078b90 0x7f884407b0a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 --2- 172.21.15.22:0/3741772990 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f886406ea20 0x7f88640c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 >> 172.21.15.22:0/3741772990 conn(0x7f88640d1850 msgr2=0x7f8864075d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 shutdown_connections 2024-09-21T05:48:18.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.537+0000 7f886975e640 1 -- 172.21.15.22:0/3741772990 wait complete. 2024-09-21T05:48:18.543 INFO:tasks.ceph.mds.a.smithi022.stderr: -1995> 2024-09-21T05:48:15.282+0000 7f35bdae3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:48:18.554 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force --yes-i-really-really-mean-it 2024-09-21T05:48:18.600 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:48:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.609+0000 7f12f49fe280 1 -- start start 2024-09-21T05:48:18.611 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.609+0000 7f12f49fe280 1 Processor -- start 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510975680 0x560511696b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 0x560511696580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976400 0x560511696000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f49fe280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5605109c2180 con 0x560510975b00 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x560510923800 con 0x560510975680 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x560510923380 con 0x560510976400 2024-09-21T05:48:18.614 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 0x560511696580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 0x560511696580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48980/0 (socket says 172.21.15.22:48980) 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 -- 172.21.15.22:0/1725312616 learned_addr learned my addr 172.21.15.22:0/1725312616 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f31a3640 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510975680 0x560511696b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f39a4640 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976400 0x560511696000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 -- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976400 msgr2=0x560511696000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976400 0x560511696000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 -- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510975680 msgr2=0x560511696b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510975680 0x560511696b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f29a2640 1 -- 172.21.15.22:0/1725312616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605108fe780 con 0x560510975b00 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.610+0000 7f12f31a3640 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510975680 0x560511696b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f29a2640 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 0x560511696580 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x5605116e9d10 tx=0x5605109b4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bf0080347c645 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f099e640 1 -- 172.21.15.22:0/1725312616 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b6ea0 con 0x560510975b00 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f099e640 1 -- 172.21.15.22:0/1725312616 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56050c8bc540 con 0x560510975b00 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f099e640 1 -- 172.21.15.22:0/1725312616 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b7380 con 0x560510975b00 2024-09-21T05:48:18.615 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605108ff4a0 con 0x560510975b00 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5605108fe000 con 0x560510975b00 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f49fe280 4 waiting for MDS map... 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f099e640 1 -- 172.21.15.22:0/1725312616 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x56051179e600 con 0x560510975b00 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f099e640 1 -- 172.21.15.22:0/1725312616 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x56051091b180 con 0x560510975b00 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f49fe280 4 Got MDS map 108 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f49fe280 10 main: JournalTool::main 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.611+0000 7f12f49fe280 4 main: JournalTool: connecting to RADOS... 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.612+0000 7f12f49fe280 1 Processor -- start 2024-09-21T05:48:18.616 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.612+0000 7f12f49fe280 1 -- start start 2024-09-21T05:48:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 0x560511697600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f49fe280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x560510923980 con 0x560510974900 2024-09-21T05:48:18.620 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605109c2d80 con 0x560510974480 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5605109c2f00 con 0x560510974d80 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f31a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 0x560511697600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.613+0000 7f12f31a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 0x560511697600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48982/0 (socket says 172.21.15.22:48982) 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 -- 172.21.15.22:0/230154820 learned_addr learned my addr 172.21.15.22:0/230154820 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f29a2640 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f39a4640 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 -- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974d80 msgr2=0x560511697080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 -- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974480 msgr2=0x560511697b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 -- 172.21.15.22:0/230154820 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605108fe3c0 con 0x560510974900 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f29a2640 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12f31a3640 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 0x560511697600 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5605116e9350 tx=0x5605117bccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bffbbbdd516649 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12ee99a640 1 -- 172.21.15.22:0/230154820 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b6340 con 0x560510974900 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12ee99a640 1 -- 172.21.15.22:0/230154820 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56050c8bd180 con 0x560510974900 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.614+0000 7f12ee99a640 1 -- 172.21.15.22:0/230154820 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b6ea0 con 0x560510974900 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 -- 172.21.15.22:0/230154820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 msgr2=0x560511697600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 0x560511697600 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x5605116e9350 tx=0x5605117bccc0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 -- 172.21.15.22:0/230154820 shutdown_connections 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974900 0x560511697600 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 --2- 172.21.15.22:0/230154820 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 -- 172.21.15.22:0/230154820 >> 172.21.15.22:0/230154820 conn(0x560510975200 msgr2=0x5605109e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 -- 172.21.15.22:0/230154820 shutdown_connections 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.615+0000 7f12f49fe280 1 -- 172.21.15.22:0/230154820 wait complete. 2024-09-21T05:48:18.622 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.616+0000 7f12f49fe280 1 Processor -- start 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.616+0000 7f12f49fe280 1 -- start start 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974900 0x560511697600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f39a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f39a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48988/0 (socket says 172.21.15.22:48988) 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f49fe280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5605109c3200 con 0x560510974480 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605109c3080 con 0x560510974d80 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f39a4640 1 -- 172.21.15.22:0/831522742 learned_addr learned my addr 172.21.15.22:0/831522742 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5605109c3380 con 0x560510974900 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.617+0000 7f12f31a3640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974900 0x560511697600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f39a4640 1 -- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974900 msgr2=0x560511697600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.623 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f39a4640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974900 0x560511697600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f39a4640 1 -- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974d80 msgr2=0x560511697080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f39a4640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f39a4640 1 -- 172.21.15.22:0/831522742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605108fe780 con 0x560510974480 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f39a4640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 0x560511697b80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x5605117b3e60 tx=0x5605117bcf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12ed197640 1 -- 172.21.15.22:0/831522742 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b6340 con 0x560510974480 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12ed197640 1 -- 172.21.15.22:0/831522742 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56050c8bc380 con 0x560510974480 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5605108ff4a0 con 0x560510974480 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12ed197640 1 -- 172.21.15.22:0/831522742 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b6ea0 con 0x560510974480 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605108fe960 con 0x560510974480 2024-09-21T05:48:18.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.618+0000 7f12f49fe280 4 main: JournalTool: resolving pool 16 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.619+0000 7f12ed197640 1 -- 172.21.15.22:0/831522742 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x5605116a4c00 con 0x560510974480 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.620+0000 7f12ed197640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x560510977180 0x560511698100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.620+0000 7f12ed197640 1 -- 172.21.15.22:0/831522742 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x560511810a00 con 0x560510974480 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.620+0000 7f12f49fe280 4 main: JournalTool: creating IoCtx.. 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.620+0000 7f12f49fe280 4 main: Executing for rank 0 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.620+0000 7f12f31a3640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x560510977180 0x560511698100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.621+0000 7f12f49fe280 1 -- start start 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.621+0000 7f12f49fe280 1 Processor -- start 2024-09-21T05:48:18.628 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.621+0000 7f12f31a3640 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x560510977180 0x560511698100 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5605116e9350 tx=0x56051099c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976880 0x560511698680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 0x560511698c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f49fe280 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510977600 0x560511699180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f49fe280 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5605109c3980 con 0x560510976d00 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5605109c3800 con 0x560510977600 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f49fe280 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5605109c2f00 con 0x560510976880 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f31a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 0x560511698c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f31a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 0x560511698c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:48996/0 (socket says 172.21.15.22:48996) 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f31a3640 1 -- 172.21.15.22:0/3526147995 learned_addr learned my addr 172.21.15.22:0/3526147995 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f39a4640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976880 0x560511698680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.622+0000 7f12f29a2640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510977600 0x560511699180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.629 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f31a3640 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976880 msgr2=0x560511698680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f31a3640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976880 0x560511698680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f31a3640 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510977600 msgr2=0x560511699180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f31a3640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510977600 0x560511699180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f31a3640 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605108ff680 con 0x560510976d00 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f31a3640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 0x560511698c00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x5605117b3290 tx=0x560510c959e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7b76a7b368a475 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12ea191640 1 -- 172.21.15.22:0/3526147995 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b7380 con 0x560510976d00 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12ea191640 1 -- 172.21.15.22:0/3526147995 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56050c8bcfc0 con 0x560510976d00 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12ea191640 1 -- 172.21.15.22:0/3526147995 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56050c8b7040 con 0x560510976d00 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605108fe780 con 0x560510976d00 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5605108ff4a0 con 0x560510976d00 2024-09-21T05:48:18.630 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f49fe280 4 waiting for MDS map... 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12ea191640 1 -- 172.21.15.22:0/3526147995 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x560510d34200 con 0x560510976d00 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12ea191640 1 -- 172.21.15.22:0/3526147995 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x56051091b500 con 0x560510976d00 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f49fe280 4 Got MDS map 108 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.623+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x560510977a80 0x560511699700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.624+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3a000 con 0x560510977a80 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.624+0000 7f12f39a4640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x560510977a80 0x560511699700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.631 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.624+0000 7f12f39a4640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x560510977a80 0x560511699700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=a79cfc0ac85cf3a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.628+0000 7f12f39a4640 1 -- 172.21.15.22:0/3526147995 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v113'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x560510a04000 con 0x560510977a80 2024-09-21T05:48:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.628+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3a400 con 0x560510977a80 2024-09-21T05:48:18.632 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.629+0000 7f12f39a4640 1 -- 172.21.15.22:0/3526147995 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v113'36 uv36 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x560510a04000 con 0x560510977a80 2024-09-21T05:48:18.632 INFO:teuthology.orchestra.run.smithi022.stdout:writing EResetJournal entry 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3a800 con 0x560510977a80 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x560510d40000 0x560510d48000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3ac00 con 0x560510d40000 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x560510d40480 0x560510d48580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3b000 con 0x560510d40480 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3b400 con 0x560510d40000 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x560510d40900 0x560510d48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3b800 con 0x560510d40900 2024-09-21T05:48:18.634 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x560510d40d80 0x560510d49080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x560510d3bc00 con 0x560510d40d80 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f39a4640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x560510d40480 0x560510d48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f29a2640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x560510d40000 0x560510d48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f31a3640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x560510d40900 0x560510d48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f29a2640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x560510d40d80 0x560510d49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.630+0000 7f12f39a4640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x560510d40480 0x560510d48580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=6a6f281aa5a8dc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.631+0000 7f12f31a3640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x560510d40900 0x560510d48b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=e1ab266e50271f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.631+0000 7f12f29a2640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x560510d40000 0x560510d48000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=9420e9dca931d2b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.631+0000 7f12f29a2640 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x560510d40d80 0x560510d49080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=95d78e899e41f57d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.635 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.631+0000 7f12f39a4640 1 -- 172.21.15.22:0/3526147995 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(3 200.00000001 [delete] v113'37 uv37 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x560510a04000 con 0x560510977a80 2024-09-21T05:48:18.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.633+0000 7f12f31a3640 1 -- 172.21.15.22:0/3526147995 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(7 200.00000005 [delete] v113'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560510a04500 con 0x560510d40900 2024-09-21T05:48:18.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.633+0000 7f12f39a4640 1 -- 172.21.15.22:0/3526147995 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(5 200.00000003 [delete] v113'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560510a04000 con 0x560510d40480 2024-09-21T05:48:18.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.634+0000 7f12f29a2640 1 -- 172.21.15.22:0/3526147995 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(4 200.00000002 [delete] v113'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560510a04780 con 0x560510d40000 2024-09-21T05:48:18.636 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.634+0000 7f12e898e640 1 -- 172.21.15.22:0/3526147995 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x56051179d400 con 0x560510977a80 2024-09-21T05:48:18.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.635+0000 7f12f39a4640 1 -- 172.21.15.22:0/3526147995 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v113'38 uv38 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x560510a04000 con 0x560510977a80 2024-09-21T05:48:18.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.636+0000 7f12f29a2640 1 -- 172.21.15.22:0/3526147995 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(6 200.00000004 [delete] v113'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560510a04780 con 0x560510d40000 2024-09-21T05:48:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f29a2640 1 -- 172.21.15.22:0/3526147995 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(8 200.00000006 [delete] v113'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560510a04780 con 0x560510d40d80 2024-09-21T05:48:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-09-21T05:48:18.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x560510977a80 msgr2=0x560511699700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x560510977a80 0x560511699700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x560510d40480 msgr2=0x560510d48580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x560510d40480 0x560510d48580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x560510d40900 msgr2=0x560510d48b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x560510d40900 0x560510d48b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x560510d40000 msgr2=0x560510d48000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x560510d40000 0x560510d48000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f29a2640 1 -- 172.21.15.22:0/3526147995 reap_dead start 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x560510d40d80 msgr2=0x560510d49080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x560510d40d80 0x560510d49080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f29a2640 1 -- 172.21.15.22:0/3526147995 reap_dead start 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 msgr2=0x560511698c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 0x560511698c00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x5605117b3290 tx=0x560510c959e0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 shutdown_connections 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 --2- 172.21.15.22:0/3526147995 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510976d00 0x560511698c00 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 >> 172.21.15.22:0/3526147995 conn(0x560510974000 msgr2=0x560510cd6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 shutdown_connections 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.639+0000 7f12f49fe280 1 -- 172.21.15.22:0/3526147995 wait complete. 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x560510977180 msgr2=0x560511698100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x560510977180 0x560511698100 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5605116e9350 tx=0x56051099c000 comp rx=0 tx=0).stop 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 msgr2=0x560511697b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 0x560511697b80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x5605117b3e60 tx=0x5605117bcf60 comp rx=0 tx=0).stop 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 shutdown_connections 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510974d80 0x560511697080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x560510977180 0x560511698100 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510974900 0x560511697600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/831522742 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510974480 0x560511697b80 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 >> 172.21.15.22:0/831522742 conn(0x560510975200 msgr2=0x5605109e9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 shutdown_connections 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/831522742 wait complete. 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 msgr2=0x560511696580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 0x560511696580 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x5605116e9d10 tx=0x5605109b4d20 comp rx=0 tx=0).stop 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 shutdown_connections 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x560510976400 0x560511696000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x560510975b00 0x560511696580 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 --2- 172.21.15.22:0/1725312616 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x560510975680 0x560511696b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.646 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 >> 172.21.15.22:0/1725312616 conn(0x560510975f80 msgr2=0x5605109e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 shutdown_connections 2024-09-21T05:48:18.647 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.640+0000 7f12f49fe280 1 -- 172.21.15.22:0/1725312616 wait complete. 2024-09-21T05:48:18.647 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force', '--yes-i-really-really-mean-it'] in time 0:00:00.093867, result: writing EResetJournal entry 2024-09-21T05:48:18.648 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-09-21T05:48:18.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.698+0000 7f47f622c740 1 -- start start 2024-09-21T05:48:18.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.698+0000 7f47f622c740 1 Processor -- start 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f622c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 0x559c0b2f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c1b00 0x559c0b2f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f622c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c1f80 0x559c0b2f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f622c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559c0b31a180 con 0x559c0a6c1680 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c0a57b800 con 0x559c0a6c1b00 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f622c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559c0a57b380 con 0x559c0a6c1f80 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f3b9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 0x559c0b2f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f3b9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 0x559c0b2f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49018/0 (socket says 172.21.15.22:49018) 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f3b9f640 1 -- 172.21.15.22:0/1490113825 learned_addr learned my addr 172.21.15.22:0/1490113825 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f339e640 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c1b00 0x559c0b2f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.699+0000 7f47f43a0640 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c1f80 0x559c0b2f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f3b9f640 1 -- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c1f80 msgr2=0x559c0b2f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c1f80 0x559c0b2f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f3b9f640 1 -- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c1b00 msgr2=0x559c0b2f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c1b00 0x559c0b2f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f3b9f640 1 -- 172.21.15.22:0/1490113825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c06515680 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 0x559c0b2f6b00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x559c0b4020c0 tx=0x559c0a606600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b142ab8ddb120410 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f139a640 1 -- 172.21.15.22:0/1490113825 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559c0650eea0 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f139a640 1 -- 172.21.15.22:0/1490113825 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559c0a56e8c0 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f139a640 1 -- 172.21.15.22:0/1490113825 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559c0650f380 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c06515860 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x559c065152c0 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f622c740 4 waiting for MDS map... 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f139a640 1 -- 172.21.15.22:0/1490113825 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x559c0b410600 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f139a640 1 -- 172.21.15.22:0/1490113825 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x559c0a573180 con 0x559c0a6c1680 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.700+0000 7f47f622c740 4 Got MDS map 108 2024-09-21T05:48:18.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.701+0000 7f47f622c740 4 datascan.main: Using metadata pool output 2024-09-21T05:48:18.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.701+0000 7f47f622c740 4 datascan.main: connecting to RADOS... 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.701+0000 7f47f622c740 1 Processor -- start 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.701+0000 7f47f622c740 1 -- start start 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.702+0000 7f47f622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.702+0000 7f47f622c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c0900 0x559c0b2f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.702+0000 7f47f622c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 0x559c0b2f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.702+0000 7f47f622c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559c0a57b980 con 0x559c0a6c2880 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.702+0000 7f47f622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c0b31ad80 con 0x559c0a6c0d80 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.702+0000 7f47f622c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559c0b31af00 con 0x559c0a6c0900 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 0x559c0b2f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.707 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 0x559c0b2f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49034/0 (socket says 172.21.15.22:49034) 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 -- 172.21.15.22:0/3537460072 learned_addr learned my addr 172.21.15.22:0/3537460072 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f43a0640 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 -- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c0900 msgr2=0x559c0b2f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c0900 0x559c0b2f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 -- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0d80 msgr2=0x559c0b2f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 -- 172.21.15.22:0/3537460072 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c065154a0 con 0x559c0a6c2880 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f43a0640 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47f339e640 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 0x559c0b2f7b80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x559c0b403530 tx=0x559c0b422c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4249c196e8fd50 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47ef396640 1 -- 172.21.15.22:0/3537460072 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559c0650e340 con 0x559c0a6c2880 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47ef396640 1 -- 172.21.15.22:0/3537460072 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559c0a56efc0 con 0x559c0a6c2880 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.703+0000 7f47ef396640 1 -- 172.21.15.22:0/3537460072 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559c0650eea0 con 0x559c0a6c2880 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 -- 172.21.15.22:0/3537460072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 msgr2=0x559c0b2f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 0x559c0b2f7b80 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x559c0b403530 tx=0x559c0b422c00 comp rx=0 tx=0).stop 2024-09-21T05:48:18.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 -- 172.21.15.22:0/3537460072 shutdown_connections 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c2880 0x559c0b2f7b80 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c0900 0x559c0b2f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 --2- 172.21.15.22:0/3537460072 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 -- 172.21.15.22:0/3537460072 >> 172.21.15.22:0/3537460072 conn(0x559c0a6c1200 msgr2=0x559c0a63b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 -- 172.21.15.22:0/3537460072 shutdown_connections 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.704+0000 7f47f622c740 1 -- 172.21.15.22:0/3537460072 wait complete. 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.705+0000 7f47f622c740 1 Processor -- start 2024-09-21T05:48:18.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.705+0000 7f47f622c740 1 -- start start 2024-09-21T05:48:18.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f622c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0900 0x559c0b2f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f622c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f622c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c2880 0x559c0b2f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f622c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x559c0b31b200 con 0x559c0a6c0d80 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f622c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x559c0b31b080 con 0x559c0a6c0900 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f622c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x559c0b31b380 con 0x559c0a6c2880 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49050/0 (socket says 172.21.15.22:49050) 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 learned_addr learned my addr 172.21.15.22:0/2408500857 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0900 0x559c0b2f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c2880 msgr2=0x559c0b2f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c2880 0x559c0b2f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0900 msgr2=0x559c0b2f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c0900 0x559c0b2f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c06515680 con 0x559c0a6c0d80 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.706+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x559c0b3326c0 tx=0x559c0a606900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.707+0000 7f47edb93640 1 -- 172.21.15.22:0/2408500857 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559c0650f1e0 con 0x559c0a6c0d80 2024-09-21T05:48:18.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.707+0000 7f47edb93640 1 -- 172.21.15.22:0/2408500857 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559c0a56e540 con 0x559c0a6c0d80 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.707+0000 7f47edb93640 1 -- 172.21.15.22:0/2408500857 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x559c0650e340 con 0x559c0a6c0d80 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.707+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559c06515860 con 0x559c0a6c0d80 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.707+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c065150e0 con 0x559c0a6c0d80 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.707+0000 7f47f622c740 4 datascan.init: resolving metadata pool 16 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.708+0000 7f47edb93640 1 -- 172.21.15.22:0/2408500857 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x559c0b330c00 con 0x559c0a6c0d80 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.708+0000 7f47edb93640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x559c0a6c3a80 0x559c0b2f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47edb93640 1 -- 172.21.15.22:0/2408500857 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x559c0b47a200 con 0x559c0a6c0d80 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f622c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f622c740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f622c740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x559c0a6c2d00 0x559c0b2f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924000 con 0x559c0a6c2d00 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x559c0a6c3a80 0x559c0b2f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.713 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f339e640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x559c0a6c2d00 0x559c0b2f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f339e640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x559c0a6c2d00 0x559c0b2f8680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.714 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.709+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x559c0a6c3a80 0x559c0b2f8100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x559c0a60d620 tx=0x559c0b422fc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.710+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(1 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.710+0000 7f47eeb95640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x559c0a92a000 0x559c0a932000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.710+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925400 con 0x559c0a92a000 2024-09-21T05:48:18.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.710+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x559c0a92a000 0x559c0a932000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.717 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.710+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x559c0a92a000 0x559c0a932000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.711+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(2 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.711+0000 7f47ef396640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x559c0a92a480 0x559c0a932580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.711+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925800 con 0x559c0a92a480 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.711+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x559c0a92a480 0x559c0a932580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.711+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x559c0a92a480 0x559c0a932580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(3 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47eeb95640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x559c0a92a900 0x559c0a932b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925c00 con 0x559c0a92a900 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47f339e640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x559c0a92a900 0x559c0a932b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47f339e640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x559c0a92a900 0x559c0a932b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(4 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.712+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946000 con 0x559c0a6c2d00 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.713+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== osd_op_reply(5 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.713+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924000 con 0x559c0a92a480 2024-09-21T05:48:18.718 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.713+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== osd_op_reply(6 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.713+0000 7f47ef396640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x559c0a92b200 0x559c0a933600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.713+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946400 con 0x559c0a92b200 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.713+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x559c0a92b200 0x559c0a933600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.714+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x559c0a92b200 0x559c0a933600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.714+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(7 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b200 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.714+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9c00 con 0x559c0a6c2d00 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.714+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 3 ==== osd_op_reply(8 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.714+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946000 con 0x559c0a92a900 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(9 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9800 con 0x559c0a92a480 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 3 ==== osd_op_reply(10 [pgnls start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946800 con 0x559c0a92a480 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 4 ==== osd_op_reply(11 [pgnls start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.719 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946c00 con 0x559c0a92a900 2024-09-21T05:48:18.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(12 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.720 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.715+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925c00 con 0x559c0a6c2d00 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.716+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 4 ==== osd_op_reply(13 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.716+0000 7f47eeb95640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x559c0a92ad80 0x559c0a933080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.716+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947000 con 0x559c0a92ad80 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.716+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x559c0a92ad80 0x559c0a933080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.716+0000 7f47f3b9f640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x559c0a92ad80 0x559c0a933080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.717+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(14 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.717+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947c00 con 0x559c0a92a000 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.717+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== osd_op_reply(15 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.717+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947400 con 0x559c0a92a900 2024-09-21T05:48:18.722 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.717+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(16 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.717+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946400 con 0x559c0a92ad80 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.718+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(17 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.718+0000 7f47eeb95640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x559c0a92bb00 0x559c0a934100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.718+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947800 con 0x559c0a92bb00 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.718+0000 7f47f339e640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x559c0a92bb00 0x559c0a934100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.718+0000 7f47f339e640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x559c0a92bb00 0x559c0a934100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(18 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947c00 con 0x559c0a92a480 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 5 ==== osd_op_reply(19 [pgnls start_epoch 113 out=81b] v112'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924400 con 0x559c0a92a480 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47f622c740 10 datascan.operator(): handling object 1099511627778.0 2024-09-21T05:48:18.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.719+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925c00 con 0x559c0a92a480 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.721+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 7 ==== osd_op_reply(21 10000000002.00000000 [call] v113'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.721+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925800 con 0x559c0a6c2d00 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.722+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 5 ==== osd_op_reply(22 [pgnls start_epoch 113 out=49b] v111'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.722+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925000 con 0x559c0a6c2d00 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.722+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 6 ==== osd_op_reply(23 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.722+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946000 con 0x559c0a92a480 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.722+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 8 ==== osd_op_reply(24 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.722+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925400 con 0x559c0a92a900 2024-09-21T05:48:18.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.723+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 5 ==== osd_op_reply(25 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.723+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946c00 con 0x559c0a92a000 2024-09-21T05:48:18.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.723+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 3 ==== osd_op_reply(26 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.723+0000 7f47ef396640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559c0a92b680 0x559c0a933b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.723+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947800 con 0x559c0a92b680 2024-09-21T05:48:18.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.723+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559c0a92b680 0x559c0a933b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.724+0000 7f47f43a0640 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559c0a92b680 0x559c0a933b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.724+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(27 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.724+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947400 con 0x559c0a92a900 2024-09-21T05:48:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.724+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 6 ==== osd_op_reply(28 [pgnls start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.724+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946800 con 0x559c0a92bb00 2024-09-21T05:48:18.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(29 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9c00 con 0x559c0a92a900 2024-09-21T05:48:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 7 ==== osd_op_reply(30 [pgnls start_epoch 113 out=81b] v111'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924000 con 0x559c0a92a900 2024-09-21T05:48:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 8 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47f622c740 10 datascan.operator(): handling object 1099511628279.0 2024-09-21T05:48:18.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.725+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925800 con 0x559c0a92a900 2024-09-21T05:48:18.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.727+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 9 ==== osd_op_reply(32 100000001f7.00000000 [call] v113'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.727+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9c00 con 0x559c0a92b200 2024-09-21T05:48:18.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.727+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== osd_op_reply(33 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b200 2024-09-21T05:48:18.729 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.727+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9800 con 0x559c0a92b680 2024-09-21T05:48:18.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.728+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(34 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.728+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947000 con 0x559c0a92a000 2024-09-21T05:48:18.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.728+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 4 ==== osd_op_reply(35 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.730 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.728+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925000 con 0x559c0a92ad80 2024-09-21T05:48:18.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.728+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(36 [pgnls start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.728+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946400 con 0x559c0a92a900 2024-09-21T05:48:18.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 10 ==== osd_op_reply(37 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.731 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946c00 con 0x559c0a92a900 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 11 ==== osd_op_reply(38 [pgnls start_epoch 113 out=49b] v111'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925400 con 0x559c0a92a480 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 9 ==== osd_op_reply(39 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947800 con 0x559c0a92a000 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 5 ==== osd_op_reply(40 [pgnls start_epoch 113 out=81b] v111'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.729+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947400 con 0x559c0a92a000 2024-09-21T05:48:18.732 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.730+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.730+0000 7f47f622c740 10 datascan.operator(): handling object 1099511627780.1 2024-09-21T05:48:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.730+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946800 con 0x559c0a92b680 2024-09-21T05:48:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.731+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 3 ==== osd_op_reply(42 10000000004.00000000 [call] v113'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.731+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964000 con 0x559c0a92a480 2024-09-21T05:48:18.733 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.732+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 10 ==== osd_op_reply(43 [pgnls start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.732+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947000 con 0x559c0a92ad80 2024-09-21T05:48:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.732+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(44 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.732+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924000 con 0x559c0a92bb00 2024-09-21T05:48:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.732+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(45 [pgnls start_epoch 113 out=81b] v112'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.734 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.732+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964400 con 0x559c0a92bb00 2024-09-21T05:48:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.733+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.733+0000 7f47f622c740 10 datascan.operator(): handling object 1099511628280.0 2024-09-21T05:48:18.735 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.733+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964800 con 0x559c0a92bb00 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.734+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v113'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.734+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964c00 con 0x559c0a92bb00 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 6 ==== osd_op_reply(48 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964000 con 0x559c0a92b680 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 4 ==== osd_op_reply(49 [pgnls start_epoch 113 out=81b] v113'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x559c0a965000 con 0x559c0a92b680 2024-09-21T05:48:18.737 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47f622c740 10 datascan.operator(): handling object 1099511627780.0 2024-09-21T05:48:18.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.735+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x559c0a965800 con 0x559c0a92b680 2024-09-21T05:48:18.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.737+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 6 ==== osd_op_reply(51 10000000004.00000000 [call] v113'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.737+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a965c00 con 0x559c0a92a000 2024-09-21T05:48:18.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.737+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 7 ==== osd_op_reply(52 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.737+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a965400 con 0x559c0a92bb00 2024-09-21T05:48:18.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.737+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 7 ==== osd_op_reply(53 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.737+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9c00 con 0x559c0a92ad80 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.738+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 5 ==== osd_op_reply(54 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.738+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924400 con 0x559c0a6c2d00 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.738+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 7 ==== osd_op_reply(55 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.738+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c064e9800 con 0x559c0a6c2d00 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.739+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 8 ==== osd_op_reply(56 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.741 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.739+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925c00 con 0x559c0a6c2d00 2024-09-21T05:48:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.739+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 9 ==== osd_op_reply(57 [pgnls start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.739+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946400 con 0x559c0a6c2d00 2024-09-21T05:48:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.739+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 10 ==== osd_op_reply(58 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.742 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.739+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925000 con 0x559c0a92ad80 2024-09-21T05:48:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.740+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 6 ==== osd_op_reply(59 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.740+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946c00 con 0x559c0a92ad80 2024-09-21T05:48:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.740+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 7 ==== osd_op_reply(60 [pgnls start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.740+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925400 con 0x559c0a92a480 2024-09-21T05:48:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.740+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.3 v2:172.21.15.22:6808/3633643032 11 ==== osd_op_reply(61 [pgnls start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92a480 2024-09-21T05:48:18.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.741+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947800 con 0x559c0a6c2d00 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.741+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.7 v2:172.21.15.134:6818/1266427391 11 ==== osd_op_reply(62 [pgnls start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a6c2d00 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.741+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947c00 con 0x559c0a92bb00 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.741+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.5 v2:172.21.15.134:6802/1222371224 8 ==== osd_op_reply(63 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92bb00 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.741+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a947400 con 0x559c0a92ad80 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.742+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 8 ==== osd_op_reply(64 [pgnls start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.742+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946000 con 0x559c0a92ad80 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.742+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 9 ==== osd_op_reply(65 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.742+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a946800 con 0x559c0a92b200 2024-09-21T05:48:18.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.742+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.2 v2:172.21.15.22:6816/3460873616 3 ==== osd_op_reply(66 [pgnls start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b200 2024-09-21T05:48:18.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.742+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964400 con 0x559c0a92a900 2024-09-21T05:48:18.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.743+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 12 ==== osd_op_reply(67 [pgnls start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.743+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925800 con 0x559c0a92ad80 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.743+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 10 ==== osd_op_reply(68 [pgnls start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.743+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964800 con 0x559c0a92b680 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.743+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 7 ==== osd_op_reply(69 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.743+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a924000 con 0x559c0a92a900 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.744+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 13 ==== osd_op_reply(70 [pgnls start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.744+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a964c00 con 0x559c0a92a000 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.744+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.1 v2:172.21.15.22:6824/48261475 8 ==== osd_op_reply(71 [pgnls start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92a000 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.744+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925000 con 0x559c0a92ad80 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.744+0000 7f47f3b9f640 1 -- 172.21.15.22:0/2408500857 <== osd.4 v2:172.21.15.134:6826/3950009634 11 ==== osd_op_reply(72 [pgnls start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a780 con 0x559c0a92ad80 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.744+0000 7f47ef396640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a965400 con 0x559c0a92b680 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f43a0640 1 -- 172.21.15.22:0/2408500857 <== osd.6 v2:172.21.15.134:6810/640262620 8 ==== osd_op_reply(73 [pgnls start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a500 con 0x559c0a92b680 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47eeb95640 1 -- 172.21.15.22:0/2408500857 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x559c0a925400 con 0x559c0a92a900 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 <== osd.0 v2:172.21.15.22:6800/3448237758 14 ==== osd_op_reply(74 [pgnls start_epoch 113 out=49b] v111'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x559c0a61a000 con 0x559c0a92a900 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x559c0a92a900 msgr2=0x559c0a932b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x559c0a92a900 0x559c0a932b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x559c0a92a000 msgr2=0x559c0a932000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x559c0a92a000 0x559c0a932000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x559c0a92b200 msgr2=0x559c0a933600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x559c0a92b200 0x559c0a933600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x559c0a92a480 msgr2=0x559c0a932580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x559c0a92a480 0x559c0a932580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x559c0a92ad80 msgr2=0x559c0a933080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x559c0a92ad80 0x559c0a933080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x559c0a92bb00 msgr2=0x559c0a934100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x559c0a92bb00 0x559c0a934100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559c0a92b680 msgr2=0x559c0a933b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x559c0a92b680 0x559c0a933b80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.750 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x559c0a6c2d00 msgr2=0x559c0b2f8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x559c0a6c2d00 0x559c0b2f8680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x559c0a6c3a80 msgr2=0x559c0b2f8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x559c0a6c3a80 0x559c0b2f8100 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x559c0a60d620 tx=0x559c0b422fc0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c0d80 msgr2=0x559c0b2f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 --2- 172.21.15.22:0/2408500857 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c0d80 0x559c0b2f7080 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x559c0b3326c0 tx=0x559c0a606900 comp rx=0 tx=0).stop 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 reap_dead start 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 shutdown_connections 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 reap_dead start 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 >> 172.21.15.22:0/2408500857 conn(0x559c0a6c1200 msgr2=0x559c0a63b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 reap_dead start 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 reap_dead start 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f339e640 1 -- 172.21.15.22:0/2408500857 reap_dead start 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 shutdown_connections 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.745+0000 7f47f622c740 1 -- 172.21.15.22:0/2408500857 wait complete. 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 msgr2=0x559c0b2f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 0x559c0b2f6b00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x559c0b4020c0 tx=0x559c0a606600 comp rx=0 tx=0).stop 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 shutdown_connections 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x559c0a6c1f80 0x559c0b2f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x559c0a6c1b00 0x559c0b2f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 --2- 172.21.15.22:0/1490113825 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x559c0a6c1680 0x559c0b2f6b00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 >> 172.21.15.22:0/1490113825 conn(0x559c0a6c2400 msgr2=0x559c0a63b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 shutdown_connections 2024-09-21T05:48:18.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.746+0000 7f47f622c740 1 -- 172.21.15.22:0/1490113825 wait complete. 2024-09-21T05:48:18.753 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.105168, result: 2024-09-21T05:48:18.753 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-09-21T05:48:18.807 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.805+0000 7f2c59a0f740 1 -- start start 2024-09-21T05:48:18.808 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.805+0000 7f2c59a0f740 1 Processor -- start 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc9680 0x563fe78fe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cc9b00 0x563fe78fe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 0x563fe78feb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563fe79fe180 con 0x563fe6cc9f80 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563fe79fe300 con 0x563fe6cc9b00 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563fe79fe480 con 0x563fe6cc9680 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 0x563fe78feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 0x563fe78feb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49066/0 (socket says 172.21.15.22:49066) 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 -- 172.21.15.22:0/2267469895 learned_addr learned my addr 172.21.15.22:0/2267469895 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c577db640 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc9680 0x563fe78fe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.811 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c56fda640 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cc9b00 0x563fe78fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 -- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc9680 msgr2=0x563fe78fe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc9680 0x563fe78fe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 -- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cc9b00 msgr2=0x563fe78fe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cc9b00 0x563fe78fe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 -- 172.21.15.22:0/2267469895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe2b1d680 con 0x563fe6cc9f80 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c577db640 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc9680 0x563fe78fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 0x563fe78feb00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x563fe7a060f0 tx=0x563fe6c0e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98fecd8c9d04d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c54fd6640 1 -- 172.21.15.22:0/2267469895 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fe2b16ea0 con 0x563fe6cc9f80 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c54fd6640 1 -- 172.21.15.22:0/2267469895 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563fe6b76540 con 0x563fe6cc9f80 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.807+0000 7f2c54fd6640 1 -- 172.21.15.22:0/2267469895 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fe2b17380 con 0x563fe6cc9f80 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe2b1d4a0 con 0x563fe6cc9f80 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563fe2b1da40 con 0x563fe6cc9f80 2024-09-21T05:48:18.812 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 4 waiting for MDS map... 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c54fd6640 1 -- 172.21.15.22:0/2267469895 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x563fe7a0e600 con 0x563fe6cc9f80 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c54fd6640 1 -- 172.21.15.22:0/2267469895 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x563fe6b7b180 con 0x563fe6cc9f80 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 4 Got MDS map 108 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 4 datascan.main: Using metadata pool output 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.808+0000 7f2c59a0f740 4 datascan.main: connecting to RADOS... 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.809+0000 7f2c59a0f740 1 Processor -- start 2024-09-21T05:48:18.813 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.809+0000 7f2c59a0f740 1 -- start start 2024-09-21T05:48:18.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.817 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563fe79fe600 con 0x563fe6cc8d80 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563fe79fe780 con 0x563fe6cca880 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563fe79fe900 con 0x563fe6cc8900 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49076/0 (socket says 172.21.15.22:49076) 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 -- 172.21.15.22:0/3294606244 learned_addr learned my addr 172.21.15.22:0/3294606244 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c577db640 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 -- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 msgr2=0x563fe78ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 -- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 msgr2=0x563fe78ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.818 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.810+0000 7f2c56fda640 1 -- 172.21.15.22:0/3294606244 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe2b1d860 con 0x563fe6cc8d80 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c56fda640 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x563fe7a07560 tx=0x563fe7a2ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded667d72120bfdc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c52fd2640 1 -- 172.21.15.22:0/3294606244 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fe2b16340 con 0x563fe6cc8d80 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c52fd2640 1 -- 172.21.15.22:0/3294606244 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563fe6b76fc0 con 0x563fe6cc8d80 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c52fd2640 1 -- 172.21.15.22:0/3294606244 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fe2b16ea0 con 0x563fe6cc8d80 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 -- 172.21.15.22:0/3294606244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 msgr2=0x563fe78ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x563fe7a07560 tx=0x563fe7a2ac60 comp rx=0 tx=0).stop 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 -- 172.21.15.22:0/3294606244 shutdown_connections 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.819 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/3294606244 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 -- 172.21.15.22:0/3294606244 >> 172.21.15.22:0/3294606244 conn(0x563fe6cc9200 msgr2=0x563fe6c43800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 -- 172.21.15.22:0/3294606244 shutdown_connections 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.811+0000 7f2c59a0f740 1 -- 172.21.15.22:0/3294606244 wait complete. 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.812+0000 7f2c59a0f740 1 Processor -- start 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.812+0000 7f2c59a0f740 1 -- start start 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c59a0f740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x563fe79ffb00 con 0x563fe6cc8d80 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x563fe79ff980 con 0x563fe6cca880 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c59a0f740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x563fe79ff800 con 0x563fe6cc8900 2024-09-21T05:48:18.820 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.813+0000 7f2c577db640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49090/0 (socket says 172.21.15.22:49090) 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 learned_addr learned my addr 172.21.15.22:0/1032868707 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 msgr2=0x563fe78ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc8900 0x563fe78ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 msgr2=0x563fe78ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe2b1d680 con 0x563fe6cc8d80 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cca880 0x563fe78ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x563fe7a826c0 tx=0x563fe7a2aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c517cf640 1 -- 172.21.15.22:0/1032868707 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fe2b16340 con 0x563fe6cc8d80 2024-09-21T05:48:18.821 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c517cf640 1 -- 172.21.15.22:0/1032868707 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563fe6b76c40 con 0x563fe6cc8d80 2024-09-21T05:48:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563fe2b1d4a0 con 0x563fe6cc8d80 2024-09-21T05:48:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c517cf640 1 -- 172.21.15.22:0/1032868707 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563fe2b16ea0 con 0x563fe6cc8d80 2024-09-21T05:48:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe2b1da40 con 0x563fe6cc8d80 2024-09-21T05:48:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.814+0000 7f2c59a0f740 4 datascan.init: resolving metadata pool 16 2024-09-21T05:48:18.822 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c517cf640 1 -- 172.21.15.22:0/1032868707 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x563fe7a80c00 con 0x563fe6cc8d80 2024-09-21T05:48:18.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c517cf640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x563fe6ccba80 0x563fe7900100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c517cf640 1 -- 172.21.15.22:0/1032868707 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x563fe793c600 con 0x563fe6cc8d80 2024-09-21T05:48:18.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c59a0f740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-21T05:48:18.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c59a0f740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-09-21T05:48:18.828 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c59a0f740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x563fe6ccad00 0x563fe7900680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84800 con 0x563fe6ccad00 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x563fe6ccba80 0x563fe7900100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.816+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x563fe6ccad00 0x563fe7900680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.817+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x563fe6ccad00 0x563fe7900680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.817+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x563fe6ccba80 0x563fe7900100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x563fe7a077d0 tx=0x563fe6ef03c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.817+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv32 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.817+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x563fe6ccb180 0x563fe7900c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.817+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6ccb180 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.817+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x563fe6ccb180 0x563fe7900c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.818+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x563fe6ccb180 0x563fe7900c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.818+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.818+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x563fe6ccb600 0x563fe7901180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.818+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85000 con 0x563fe6ccb600 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.818+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x563fe6ccb600 0x563fe7901180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.819+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x563fe6ccb600 0x563fe7901180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.819+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.819+0000 7f2c527d1640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x563fe6f4a000 0x563fe6f52000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.819+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85c00 con 0x563fe6f4a000 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.819+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x563fe6f4a000 0x563fe6f52000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.820+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x563fe6f4a000 0x563fe6f52000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.820+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.820+0000 7f2c52fd2640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x563fe6f4a480 0x563fe6f52580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.820+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56000 con 0x563fe6f4a480 2024-09-21T05:48:18.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.820+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x563fe6f4a480 0x563fe6f52580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.820+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x563fe6f4a480 0x563fe6f52580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.821+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.821+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56400 con 0x563fe6ccad00 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.821+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.821+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56c00 con 0x563fe6ccb600 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56800 con 0x563fe6f4a480 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c52fd2640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x563fe6f4a900 0x563fe6f52b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57000 con 0x563fe6f4a900 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x563fe6f4a900 0x563fe6f52b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.822+0000 7f2c577db640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x563fe6f4a900 0x563fe6f52b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.823+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6f4a900 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.823+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6ccb600 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.823+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.823+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56c00 con 0x563fe6ccad00 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.824+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.824+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d400 con 0x563fe6f4a480 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.824+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.824+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85000 con 0x563fe6f4a480 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.824+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.824+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d000 con 0x563fe6ccad00 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.825+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.825+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6ccb600 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.825+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.825+0000 7f2c527d1640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x563fe6f4b200 0x563fe6f53600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.825+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d000 con 0x563fe6f4b200 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.825+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x563fe6f4b200 0x563fe6f53600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c56fda640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x563fe6f4b200 0x563fe6f53600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56000 con 0x563fe6f4a000 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85c00 con 0x563fe6ccad00 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.833 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.826+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56400 con 0x563fe6f4b200 2024-09-21T05:48:18.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.827+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.839 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.827+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe2af1c00 con 0x563fe6ccb180 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.827+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.827+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 113 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56800 con 0x563fe6f4a480 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.828+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 113 out=81b] v113'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.828+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85400 con 0x563fe6ccb600 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56c00 con 0x563fe6f4a000 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57000 con 0x563fe6f4a480 2024-09-21T05:48:18.840 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68000 con 0x563fe6ccad00 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 6 ==== osd_op_reply(25 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.829+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57400 con 0x563fe6ccb600 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.830+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.830+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68400 con 0x563fe6f4a480 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.830+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.830+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57800 con 0x563fe6f4a900 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.831+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== osd_op_reply(28 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6f4a900 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.831+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68800 con 0x563fe6ccb600 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.831+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.831+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57c00 con 0x563fe6ccad00 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.832+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.841 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.832+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68c00 con 0x563fe6f4a480 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.832+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.832+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6f4a480 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.832+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.832+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69000 con 0x563fe6ccad00 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.833+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 8 ==== osd_op_reply(33 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.833+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85400 con 0x563fe6ccb600 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.833+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.833+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57400 con 0x563fe6f4b200 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.833+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.833+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85000 con 0x563fe6f4a000 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57800 con 0x563fe6ccad00 2024-09-21T05:48:18.842 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 9 ==== osd_op_reply(37 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d400 con 0x563fe6f4b200 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56800 con 0x563fe6ccb180 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.834+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.835+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d000 con 0x563fe6f4a480 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.835+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 113 out=49b] v113'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.835+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57000 con 0x563fe6ccb600 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.835+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 113 out=49b] v111'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.835+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe2af1c00 con 0x563fe6ccb600 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56000 con 0x563fe6f4a480 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68800 con 0x563fe6ccad00 2024-09-21T05:48:18.843 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 10 ==== osd_op_reply(44 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85c00 con 0x563fe6f4a000 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c527d1640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x563fe6f4b680 0x563fe6f53b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.836+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68400 con 0x563fe6f4b680 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.837+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x563fe6f4b680 0x563fe6f53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.837+0000 7f2c57fdc640 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x563fe6f4b680 0x563fe6f53b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.837+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4b680 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.837+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69000 con 0x563fe6ccad00 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 11 ==== osd_op_reply(47 [pgnls-filter start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.844 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68000 con 0x563fe6ccb180 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68800 con 0x563fe6ccad00 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 12 ==== osd_op_reply(49 [pgnls-filter start_epoch 113 out=81b] v113'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c59a0f740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c59a0f740 10 datascan.operator(): handling object 100000001f7.0 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.838+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56800 con 0x563fe6ccad00 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 13 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56000 con 0x563fe6f4a480 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv7 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d000 con 0x563fe6f4a900 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.2 v2:172.21.15.22:6816/3460873616 3 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6f4a900 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.839+0000 7f2c59a0f740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 20 datascan.get_frag_of: frag is * 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6f4a480 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 10 datascan.find_or_create_dirfrag: read_version = 7 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-09-21T05:48:18.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68c00 con 0x563fe6f4a480 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 20 datascan.read_dentry: bravo_head not found in result 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.840+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68400 con 0x563fe6f4a480 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.841+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v113'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.841+0000 7f2c59a0f740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.841+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68000 con 0x563fe6f4a900 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.842+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.2 v2:172.21.15.22:6816/3460873616 4 ==== osd_op_reply(56 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6f4a900 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.842+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d400 con 0x563fe6f4b680 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.842+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4b680 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.842+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69400 con 0x563fe6f4a000 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.842+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.842+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57c00 con 0x563fe6f4b200 2024-09-21T05:48:18.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69800 con 0x563fe6ccad00 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 14 ==== osd_op_reply(60 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57400 con 0x563fe6ccad00 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 15 ==== osd_op_reply(61 [pgnls-filter start_epoch 113 out=49b] v111'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56400 con 0x563fe6f4a480 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.843+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57800 con 0x563fe6f4a000 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 113 out=49b] v111'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56c00 con 0x563fe6f4a480 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85000 con 0x563fe6f4b200 2024-09-21T05:48:18.857 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.844+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69000 con 0x563fe6ccb180 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.845+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 113 out=49b] v113'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.845+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68c00 con 0x563fe6ccb180 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.845+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.845+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68800 con 0x563fe6f4b680 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.846+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.6 v2:172.21.15.134:6810/640262620 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 113 out=49b] v113'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4b680 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.846+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68400 con 0x563fe6f4a000 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.846+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.846+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69400 con 0x563fe6ccb180 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.846+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.846+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69000 con 0x563fe6f4b200 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.847+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.847+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69800 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.847+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.847+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56000 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.847+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.847+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56800 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56400 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57c00 con 0x563fe6f4b200 2024-09-21T05:48:18.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f56c00 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.848+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 113 out=49b] v111'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.849+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85c00 con 0x563fe6f4a480 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.849+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.3 v2:172.21.15.22:6808/3633643032 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 113 out=49b] v111'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4a480 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.849+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d400 con 0x563fe6ccb600 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.849+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.7 v2:172.21.15.134:6818/1266427391 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 113 out=49b] v111'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6ccb600 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.849+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85800 con 0x563fe6ccb180 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.849+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.5 v2:172.21.15.134:6802/1222371224 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6ccb180 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.850+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.850+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.850+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68000 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.850+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.850+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68400 con 0x563fe6f4a900 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.850+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 <== osd.2 v2:172.21.15.22:6816/3460873616 5 ==== osd_op_reply(83 [pgnls-filter start_epoch 113 out=49b] v111'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22780 con 0x563fe6f4a900 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.851+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69800 con 0x563fe6ccad00 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.851+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 16 ==== osd_op_reply(84 [pgnls-filter start_epoch 113 out=49b] v111'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.851+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f69000 con 0x563fe6f4b200 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.851+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 113 out=49b] v111'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.851+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a0d000 con 0x563fe6f4b680 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.851+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.6 v2:172.21.15.134:6810/640262620 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4b680 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f68000 con 0x563fe6ccad00 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 17 ==== osd_op_reply(87 [pgnls-filter start_epoch 113 out=49b] v111'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe2af1c00 con 0x563fe6f4a000 2024-09-21T05:48:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.1 v2:172.21.15.22:6824/48261475 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 113 out=49b] v111'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4a000 2024-09-21T05:48:18.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a85000 con 0x563fe6f4b200 2024-09-21T05:48:18.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.4 v2:172.21.15.134:6826/3950009634 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 113 out=49b] v111'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6f4b200 2024-09-21T05:48:18.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.852+0000 7f2c527d1640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe6f57000 con 0x563fe6f4b680 2024-09-21T05:48:18.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c57fdc640 1 -- 172.21.15.22:0/1032868707 <== osd.6 v2:172.21.15.134:6810/640262620 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 113 out=49b] v111'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22500 con 0x563fe6f4b680 2024-09-21T05:48:18.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c52fd2640 1 -- 172.21.15.22:0/1032868707 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 113 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x563fe7a84c00 con 0x563fe6ccad00 2024-09-21T05:48:18.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c56fda640 1 -- 172.21.15.22:0/1032868707 <== osd.0 v2:172.21.15.22:6800/3448237758 18 ==== osd_op_reply(91 [pgnls-filter start_epoch 113 out=49b] v111'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563fe6c22000 con 0x563fe6ccad00 2024-09-21T05:48:18.865 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x563fe6ccad00 msgr2=0x563fe7900680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x563fe6ccad00 0x563fe7900680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x563fe6f4a000 msgr2=0x563fe6f52000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x563fe6f4a000 0x563fe6f52000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x563fe6f4a900 msgr2=0x563fe6f52b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x563fe6f4a900 0x563fe6f52b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x563fe6f4a480 msgr2=0x563fe6f52580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x563fe6f4a480 0x563fe6f52580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 reap_dead start 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x563fe6f4b200 msgr2=0x563fe6f53600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x563fe6f4b200 0x563fe6f53600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x563fe6ccb180 msgr2=0x563fe7900c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x563fe6ccb180 0x563fe7900c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x563fe6f4b680 msgr2=0x563fe6f53b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x563fe6f4b680 0x563fe6f53b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x563fe6ccb600 msgr2=0x563fe7901180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x563fe6ccb600 0x563fe7901180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c577db640 1 -- 172.21.15.22:0/1032868707 reap_dead start 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x563fe6ccba80 msgr2=0x563fe7900100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x563fe6ccba80 0x563fe7900100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x563fe7a077d0 tx=0x563fe6ef03c0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 msgr2=0x563fe78ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x563fe7a826c0 tx=0x563fe7a2aea0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 shutdown_connections 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc8d80 0x563fe78ff080 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/1032868707 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x563fe6ccba80 0x563fe7900100 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 >> 172.21.15.22:0/1032868707 conn(0x563fe6cc9200 msgr2=0x563fe6c43800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 shutdown_connections 2024-09-21T05:48:18.867 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.853+0000 7f2c59a0f740 1 -- 172.21.15.22:0/1032868707 wait complete. 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 msgr2=0x563fe78feb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 0x563fe78feb00 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x563fe7a060f0 tx=0x563fe6c0e600 comp rx=0 tx=0).stop 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 shutdown_connections 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x563fe6cc9f80 0x563fe78feb00 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x563fe6cc9b00 0x563fe78fe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 --2- 172.21.15.22:0/2267469895 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x563fe6cc9680 0x563fe78fe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 >> 172.21.15.22:0/2267469895 conn(0x563fe6cca400 msgr2=0x563fe6c41800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 shutdown_connections 2024-09-21T05:48:18.868 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.854+0000 7f2c59a0f740 1 -- 172.21.15.22:0/2267469895 wait complete. 2024-09-21T05:48:18.868 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.115297, result: 2024-09-21T05:48:18.868 DEBUG:teuthology.orchestra.run.smithi022:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-09-21T05:48:18.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.918+0000 7fa6b482c740 1 -- start start 2024-09-21T05:48:18.920 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.918+0000 7fa6b482c740 1 Processor -- start 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e9680 0x562acee22000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1e9b00 0x562acee22580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 0x562acee22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b482c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562acee2e600 con 0x562ace1e9f80 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b482c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562acee2e480 con 0x562ace1e9b00 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b482c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562acee2e300 con 0x562ace1e9680 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b29a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 0x562acee22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b29a7640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 0x562acee22b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49094/0 (socket says 172.21.15.22:49094) 2024-09-21T05:48:18.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b29a7640 1 -- 172.21.15.22:0/969087098 learned_addr learned my addr 172.21.15.22:0/969087098 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e9680 0x562acee22000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.919+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1e9b00 0x562acee22580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b29a7640 1 -- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e9680 msgr2=0x562acee22000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e9680 0x562acee22000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b29a7640 1 -- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1e9b00 msgr2=0x562acee22580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1e9b00 0x562acee22580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b29a7640 1 -- 172.21.15.22:0/969087098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aca03d680 con 0x562ace1e9f80 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 0x562acee22b00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x562acef36000 tx=0x562ace12e600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442e5284e99ef455 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6af9a1640 1 -- 172.21.15.22:0/969087098 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562aca036340 con 0x562ace1e9f80 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6af9a1640 1 -- 172.21.15.22:0/969087098 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562ace096c40 con 0x562ace1e9f80 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6af9a1640 1 -- 172.21.15.22:0/969087098 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562aca036ea0 con 0x562ace1e9f80 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aca03d4a0 con 0x562ace1e9f80 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562aca03da40 con 0x562ace1e9f80 2024-09-21T05:48:18.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b482c740 4 waiting for MDS map... 2024-09-21T05:48:18.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6af9a1640 1 -- 172.21.15.22:0/969087098 <== mon.0 v2:172.21.15.22:3300/0 4 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x562acef20000 con 0x562ace1e9f80 2024-09-21T05:48:18.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6af9a1640 1 -- 172.21.15.22:0/969087098 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x562ace09b180 con 0x562ace1e9f80 2024-09-21T05:48:18.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.920+0000 7fa6b482c740 4 Got MDS map 108 2024-09-21T05:48:18.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.921+0000 7fa6b482c740 4 datascan.main: Using metadata pool output 2024-09-21T05:48:18.925 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.921+0000 7fa6b482c740 4 datascan.main: connecting to RADOS... 2024-09-21T05:48:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.921+0000 7fa6b482c740 1 Processor -- start 2024-09-21T05:48:18.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.921+0000 7fa6b482c740 1 -- start start 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 0x562acee23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b482c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562acee2e180 con 0x562ace1e8900 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b482c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562ace0a3380 con 0x562ace1ea880 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b482c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562ace0a3800 con 0x562ace1e8d80 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49100/0 (socket says 172.21.15.22:49100) 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 -- 172.21.15.22:0/876450573 learned_addr learned my addr 172.21.15.22:0/876450573 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 0x562acee23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 -- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 msgr2=0x562acee23080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 0x562acee23080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 -- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 msgr2=0x562acee23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 -- 172.21.15.22:0/876450573 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aca03d860 con 0x562ace1e8900 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x562acef37800 tx=0x562acef98120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f03974e0d77ac21 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6ad99d640 1 -- 172.21.15.22:0/876450573 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562aca036340 con 0x562ace1e8900 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6ad99d640 1 -- 172.21.15.22:0/876450573 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562ace0968c0 con 0x562ace1e8900 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.923+0000 7fa6ad99d640 1 -- 172.21.15.22:0/876450573 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562aca036ea0 con 0x562ace1e8900 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 -- 172.21.15.22:0/876450573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 msgr2=0x562acee23600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x562acef37800 tx=0x562acef98120 comp rx=0 tx=0).stop 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 -- 172.21.15.22:0/876450573 shutdown_connections 2024-09-21T05:48:18.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 --2- 172.21.15.22:0/876450573 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 0x562acee23080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 -- 172.21.15.22:0/876450573 >> 172.21.15.22:0/876450573 conn(0x562ace1e9200 msgr2=0x562ace163800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 -- 172.21.15.22:0/876450573 shutdown_connections 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.924+0000 7fa6b482c740 1 -- 172.21.15.22:0/876450573 wait complete. 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.925+0000 7fa6b482c740 1 Processor -- start 2024-09-21T05:48:18.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.925+0000 7fa6b482c740 1 -- start start 2024-09-21T05:48:18.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 0x562acee23080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b482c740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b482c740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x562acee2f380 con 0x562ace1e8900 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b482c740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x562acee2e180 con 0x562ace1ea880 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b482c740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x562acee2f500 con 0x562ace1e8d80 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49116/0 (socket says 172.21.15.22:49116) 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 learned_addr learned my addr 172.21.15.22:0/2193036035 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 msgr2=0x562acee23080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e8d80 0x562acee23080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 msgr2=0x562acee23b80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1ea880 0x562acee23b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.934 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aca03d2c0 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.926+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x562acef9c780 tx=0x562acef980c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.927+0000 7fa6ac19a640 1 -- 172.21.15.22:0/2193036035 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562aca036340 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.927+0000 7fa6ac19a640 1 -- 172.21.15.22:0/2193036035 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562ace096540 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.927+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562aca03d0e0 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.927+0000 7fa6ac19a640 1 -- 172.21.15.22:0/2193036035 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562aca036ea0 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.927+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aca03c3c0 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.927+0000 7fa6b482c740 4 datascan.init: resolving metadata pool 16 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.928+0000 7fa6ac19a640 1 -- 172.21.15.22:0/2193036035 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x562acef96c00 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6ac19a640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x562ace1eba80 0x562acee24100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6ac19a640 1 -- 172.21.15.22:0/2193036035 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x562acef95000 con 0x562ace1e8900 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6b482c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6b482c740 4 datascan.main: resolving metadata pool 16 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x562ace1ead00 0x562acee24680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.935 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acefa2c00 con 0x562ace1ead00 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x562ace1ead00 0x562acee24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.929+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x562ace1ead00 0x562acee24680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== osd_op_reply(1 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x562ace448000 0x562ace44c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acefa2c00 con 0x562ace448000 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x562ace448000 0x562ace44c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x562ace1eba80 0x562acee24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x562ace448000 0x562ace44c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== osd_op_reply(2 [pgnls start_epoch 113 out=177b] v112'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.930+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acefa2c00 con 0x562ace1ead00 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== osd_op_reply(3 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acefa2c00 con 0x562ace1ead00 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 3 ==== osd_op_reply(4 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562ace448480 0x562ace44c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acefa2c00 con 0x562ace448480 2024-09-21T05:48:18.936 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562ace448480 0x562ace44c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.931+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562ace448480 0x562ace44c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.932+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== osd_op_reply(5 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.937 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.932+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace1ead00 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.932+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 4 ==== osd_op_reply(6 [pgnls start_epoch 113 out=149b] v113'38 uv38 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.932+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x562ace448900 0x562ace44cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.932+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448900 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.933+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x562ace1eba80 0x562acee24100 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x562acef9d500 tx=0x562acef99aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.934+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x562ace448900 0x562ace44cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.934+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x562ace448900 0x562ace44cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.935+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== osd_op_reply(7 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.935+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448000 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.935+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== osd_op_reply(8 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.935+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x562ace448d80 0x562ace44d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.935+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448d80 2024-09-21T05:48:18.939 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.935+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x562ace448d80 0x562ace44d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.936+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x562ace448d80 0x562ace44d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.936+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== osd_op_reply(9 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448d80 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.936+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x562ace449200 0x562ace44d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.936+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449200 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.936+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x562ace449200 0x562ace44d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.937+0000 7fa6b21a6640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x562ace449200 0x562ace44d600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.937+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== osd_op_reply(10 [pgnls start_epoch 113 out=81b] v112'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.940 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.937+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448000 2024-09-21T05:48:18.942 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.937+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 3 ==== osd_op_reply(11 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.937+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448480 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== osd_op_reply(12 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449200 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== osd_op_reply(13 [pgnls start_epoch 113 out=77b] v112'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448480 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 3 ==== osd_op_reply(14 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448000 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.938+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 4 ==== osd_op_reply(15 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.939+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x562ace449680 0x562ace44db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.939+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449680 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.939+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x562ace449680 0x562ace44db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.939+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x562ace449680 0x562ace44db80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== osd_op_reply(16 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace449680 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448000 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 5 ==== osd_op_reply(17 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448900 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== osd_op_reply(18 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b19a5640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x562ace449b00 0x562ace44e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449b00 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.940+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x562ace449b00 0x562ace44e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:18.944 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.941+0000 7fa6b29a7640 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x562ace449b00 0x562ace44e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:18.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.941+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== osd_op_reply(19 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.941+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448d80 2024-09-21T05:48:18.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.941+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== osd_op_reply(20 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448d80 2024-09-21T05:48:18.946 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.941+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace1ead00 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.942+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 5 ==== osd_op_reply(21 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.942+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace1ead00 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.942+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 6 ==== osd_op_reply(22 [pgnls start_epoch 113 out=97b] v112'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.942+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449200 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.942+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 3 ==== osd_op_reply(23 [pgnls start_epoch 113 out=74b] v108'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.942+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448000 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.943+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 6 ==== osd_op_reply(24 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.943+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448900 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.943+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 3 ==== osd_op_reply(25 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.943+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449b00 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.943+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== osd_op_reply(26 [pgnls start_epoch 113 out=98b] v112'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.947 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.943+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace449b00 2024-09-21T05:48:18.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.944+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 3 ==== osd_op_reply(27 [pgnls start_epoch 113 out=76b] v113'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.944+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448900 2024-09-21T05:48:18.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.944+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 4 ==== osd_op_reply(28 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.944+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448480 2024-09-21T05:48:18.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.944+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 4 ==== osd_op_reply(29 [pgnls start_epoch 113 out=105b] v113'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.948 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.944+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448000 2024-09-21T05:48:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 7 ==== osd_op_reply(30 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace1ead00 2024-09-21T05:48:18.953 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 7 ==== osd_op_reply(31 [pgnls start_epoch 113 out=73b] v113'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace448900 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 5 ==== osd_op_reply(32 [pgnls start_epoch 113 out=101b] v113'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.945+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3000 con 0x562ace448000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3400 con 0x562ace448000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3800 con 0x562ace448000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.954 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.946+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3c00 con 0x562ace448000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3000 con 0x562ace448000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3400 con 0x562ace1ead00 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv34 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.955 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.947+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef38c00 con 0x562ace448000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef38800 con 0x562ace449200 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.948+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3800 con 0x562ace448000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3c00 con 0x562ace1ead00 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-09-21T05:48:18.956 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c000 con 0x562ace448000 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.949+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c400 con 0x562ace449b00 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c800 con 0x562ace448480 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.957 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47cc00 con 0x562ace448480 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 20 datascan.scan_links: injected ino detected 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.950+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d000 con 0x562ace448900 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.951+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.951+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.951+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 10 ==== osd_op_reply(48 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.951+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.958 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.951+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 16 ==== osd_op_reply(49 [pgnls start_epoch 113 out=177b] v112'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.959 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 11 ==== osd_op_reply(50 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 12 ==== osd_op_reply(51 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448480 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 7 ==== osd_op_reply(52 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.952+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.953+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 13 ==== osd_op_reply(53 [pgnls start_epoch 113 out=149b] v113'38 uv38 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.953+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448900 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.953+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 7 ==== osd_op_reply(54 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.953+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.953+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 17 ==== osd_op_reply(55 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.953+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448d80 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.954+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.4 v2:172.21.15.134:6826/3950009634 3 ==== osd_op_reply(56 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448d80 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.954+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449200 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.954+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 5 ==== osd_op_reply(57 [pgnls start_epoch 113 out=81b] v112'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.954+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.954+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 18 ==== osd_op_reply(58 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.954+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448480 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.955+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 8 ==== osd_op_reply(59 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.955+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449200 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.955+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 6 ==== osd_op_reply(60 [pgnls start_epoch 113 out=77b] v112'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.955+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448480 2024-09-21T05:48:18.970 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.955+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 9 ==== osd_op_reply(61 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.955+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.956+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 19 ==== osd_op_reply(62 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.956+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449680 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.956+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== osd_op_reply(63 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace449680 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.956+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.956+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 20 ==== osd_op_reply(64 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.956+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448900 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 8 ==== osd_op_reply(65 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449b00 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 5 ==== osd_op_reply(66 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448d80 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.4 v2:172.21.15.134:6826/3950009634 4 ==== osd_op_reply(67 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448d80 2024-09-21T05:48:18.971 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 14 ==== osd_op_reply(68 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.957+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.958+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 15 ==== osd_op_reply(69 [pgnls start_epoch 113 out=97b] v112'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.958+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449200 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.958+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 7 ==== osd_op_reply(70 [pgnls start_epoch 113 out=74b] v108'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.958+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.958+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 21 ==== osd_op_reply(71 [pgnls start_epoch 113 out=73b] v112'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.958+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448900 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.959+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 9 ==== osd_op_reply(72 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.959+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449b00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.959+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 6 ==== osd_op_reply(73 [pgnls start_epoch 113 out=98b] v112'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.959+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace449b00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.959+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 7 ==== osd_op_reply(74 [pgnls start_epoch 113 out=76b] v113'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.972 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.959+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448900 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 10 ==== osd_op_reply(75 [pgnls start_epoch 113 out=49b] v113'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448480 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 10 ==== osd_op_reply(76 [pgnls start_epoch 113 out=105b] v113'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448000 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 22 ==== osd_op_reply(77 [pgnls start_epoch 113 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace1ead00 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 16 ==== osd_op_reply(78 [pgnls start_epoch 113 out=73b] v113'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.960+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 113 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d400 con 0x562ace448900 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 11 ==== osd_op_reply(79 [pgnls start_epoch 113 out=101b] v113'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d000 con 0x562ace448000 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-09-21T05:48:18.973 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d800 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.961+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47dc00 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3400 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3c00 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-09-21T05:48:18.974 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef39400 con 0x562ace1ead00 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv34 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.962+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c000 con 0x562ace448000 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c400 con 0x562ace449200 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=584b] v0'0 uv3 ondisk = 0) ==== 164+0+584 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-09-21T05:48:18.975 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.963+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d000 con 0x562ace448000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47d800 con 0x562ace1ead00 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.0 v2:172.21.15.22:6800/3448237758 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142000 con 0x562ace1ead00 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3000 con 0x562ace448000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.2 v2:172.21.15.22:6816/3460873616 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142500 con 0x562ace448000 2024-09-21T05:48:18.976 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.964+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acefa3800 con 0x562ace449b00 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef38c00 con 0x562ace448480 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef39800 con 0x562ace448480 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1667b] v0'0 uv9 ondisk = 0) ==== 164+0+1667 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 20 datascan.scan_links: injected ino detected 2024-09-21T05:48:18.977 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.965+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef38800 con 0x562ace448900 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b19a5640 1 -- 172.21.15.22:0/2193036035 <== osd.5 v2:172.21.15.134:6802/1222371224 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x562ace142000 con 0x562ace448900 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 10 datascan.scan_links: processing 1 injected_inos 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-09-21T05:48:18.978 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562acef38c00 con 0x562ace448480 2024-09-21T05:48:18.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-09-21T05:48:18.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 20 datascan.scan_links: first is now 2 2024-09-21T05:48:18.979 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.966+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c800 con 0x562ace448480 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.967+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.3 v2:172.21.15.22:6808/3633643032 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v113'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562ace142780 con 0x562ace448480 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.967+0000 7fa6b482c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.967+0000 7fa6b482c740 10 datascan.scan_links: updating inotable 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.967+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47cc00 con 0x562ace449b00 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.968+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.968+0000 7fa6b482c740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-09-21T05:48:18.982 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.968+0000 7fa6b482c740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.968+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) -- 0x562ace47c400 con 0x562ace449b00 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b29a7640 1 -- 172.21.15.22:0/2193036035 <== osd.1 v2:172.21.15.22:6824/48261475 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v113'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x562ace142500 con 0x562ace449b00 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 10 datascan.scan_links: updating snaptable 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) -- 0x562ace47dc00 con 0x562ace449200 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 <== osd.6 v2:172.21.15.134:6810/640262620 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x562ace142780 con 0x562ace449200 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x562ace1ead00 msgr2=0x562acee24680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x562ace1ead00 0x562acee24680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x562ace449b00 msgr2=0x562ace44e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x562ace449b00 0x562ace44e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x562ace448000 msgr2=0x562ace44c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x562ace448000 0x562ace44c000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562ace448480 msgr2=0x562ace44c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x562ace448480 0x562ace44c580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 reap_dead start 2024-09-21T05:48:18.983 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x562ace448d80 msgr2=0x562ace44d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x562ace448d80 0x562ace44d080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 reap_dead start 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x562ace448900 msgr2=0x562ace44cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x562ace448900 0x562ace44cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x562ace449200 msgr2=0x562ace44d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.969+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x562ace449200 0x562ace44d600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x562ace449680 msgr2=0x562ace44db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x562ace449680 0x562ace44db80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x562ace1eba80 msgr2=0x562acee24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x562ace1eba80 0x562acee24100 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x562acef9d500 tx=0x562acef99aa0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 msgr2=0x562acee23600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/2193036035 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e8900 0x562acee23600 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x562acef9c780 tx=0x562acef980c0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b21a6640 1 -- 172.21.15.22:0/2193036035 reap_dead start 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 shutdown_connections 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 >> 172.21.15.22:0/2193036035 conn(0x562ace1e9200 msgr2=0x562ace163800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 shutdown_connections 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/2193036035 wait complete. 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 msgr2=0x562acee22b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 0x562acee22b00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x562acef36000 tx=0x562ace12e600 comp rx=0 tx=0).stop 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 shutdown_connections 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x562ace1e9f80 0x562acee22b00 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x562ace1e9b00 0x562acee22580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 --2- 172.21.15.22:0/969087098 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x562ace1e9680 0x562acee22000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 >> 172.21.15.22:0/969087098 conn(0x562ace1ea400 msgr2=0x562ace161800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 shutdown_connections 2024-09-21T05:48:18.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:18.970+0000 7fa6b482c740 1 -- 172.21.15.22:0/969087098 wait complete. 2024-09-21T05:48:18.985 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.116832, result: 2024-09-21T05:48:18.986 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:18.988 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-09-21T05:48:19.031 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-09-21T05:48:19.032 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-09-21T05:48:19.033 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on skipped message = False 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-09-21T05:48:19.034 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:19.035 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-09-21T05:48:19.036 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-09-21T05:48:19.037 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:19.038 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.039 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-09-21T05:48:19.040 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = False 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-21T05:48:19.041 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:19.042 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-21T05:48:19.043 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-21T05:48:19.044 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.045 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-09-21T05:48:19.046 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:19.047 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.048 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:19.049 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-21T05:48:19.050 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-21T05:48:19.051 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-21T05:48:19.051 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-21T05:48:19.051 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-09-21T05:48:19.051 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-09-21T05:48:19.051 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = False 2024-09-21T05:48:19.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:19.094 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: chdir = "" 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: auth supported = cephx 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: filestore xattr use omap = true 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.137 INFO:teuthology.orchestra.run.smithi022.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: auth debug = true 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on old message = true 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on bug = true 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default size = 2 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:19.138 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd allow pg remap = true 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon warn on too few osds = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: osd default data pool replay window = 5 2024-09-21T05:48:19.139 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: mon allow pool delete = true 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: mon cluster log file level = debug 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: debug asserts on shutdown = true 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: mon health detail to clog = false 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: ms die on skipped message = False 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout:[osd] 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd journal size = 100 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd scrub max interval = 600 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:19.140 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd recover clone overlap = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug shutdown = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug op order = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug trim objects = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd open classes on start = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug pg log writeout = true 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: osd map max advance = 10 2024-09-21T05:48:19.141 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: journal zero on create = true 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: bdev debug aio = true 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: osd debug misdirected ops = true 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: bdev async discard = True 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: bdev enable discard = True 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore allocator = bitmap 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore block size = 96636764160 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: bluestore fsck on mount = True 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluefs = 1/20 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug bluestore = 1/20 2024-09-21T05:48:19.142 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: debug osd = 20 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: debug rocksdb = 4/10 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: osd objectstore = bluestore 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout:[mgr] 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: debug mgr = 20 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.143 INFO:teuthology.orchestra.run.smithi022.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout:[mon] 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: debug ms = 1 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: debug mon = 20 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: debug paxos = 20 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: debug auth = 20 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon data avail warn = 5 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon osd prime pg temp = true 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.144 INFO:teuthology.orchestra.run.smithi022.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: # 1m isn't quite enough 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: mon down mkfs grace = 300 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:19.145 INFO:teuthology.orchestra.run.smithi022.stdout:[client] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: rgw cache enabled = true 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable ops log = true 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: rgw enable usage log = true 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: pid file = "" 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.a] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.b] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mon.c] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.a] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.c] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.b] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mds.d] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout:[mds] 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: mds verify scatter = True 2024-09-21T05:48:19.146 INFO:teuthology.orchestra.run.smithi022.stdout: mds debug scatterstat = True 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-21T05:48:19.147 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-21T05:48:19.148 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T05:48:19.149 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = "172.21.15.22,172.21.15.134,[v2:172.21.15.134:3301,v1:172.21.15.134:6790]" 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on skipped message = False 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = cdac58e0-7b6b-4531-9caa-dad309e0d36b 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-21T05:48:19.150 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug trim objects = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.151 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: bdev async discard = True 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: bdev enable discard = True 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore allocator = bitmap 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore block size = 96636764160 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore fsck on mount = True 2024-09-21T05:48:19.152 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluefs = 1/20 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug bluestore = 1/20 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug rocksdb = 4/10 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd full ratio = 0.9 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd nearfull ratio = 0.8 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: osd failsafe full ratio = 0.95 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: osd objectstore = bluestore 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:19.153 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-21T05:48:19.154 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: mon down mkfs grace = 300 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-21T05:48:19.155 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = "" 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.b] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.c] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.a] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.c] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.b] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mds.d] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout:[mds] 2024-09-21T05:48:19.156 INFO:teuthology.orchestra.run.smithi134.stdout: mds verify scatter = True 2024-09-21T05:48:19.157 INFO:teuthology.orchestra.run.smithi134.stdout: mds debug scatterstat = True 2024-09-21T05:48:19.158 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-21T05:48:19.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 -- 172.21.15.22:0/2060455527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c0d3ea0 msgr2=0x7f975c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:19.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 --2- 172.21.15.22:0/2060455527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c0d3ea0 0x7f975c068d00 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f975002a5f0 comp rx=0 tx=0).stop 2024-09-21T05:48:19.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 -- 172.21.15.22:0/2060455527 shutdown_connections 2024-09-21T05:48:19.318 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 --2- 172.21.15.22:0/2060455527 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c0d3ea0 0x7f975c068d00 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:19.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 -- 172.21.15.22:0/2060455527 >> 172.21.15.22:0/2060455527 conn(0x7f975c0d4820 msgr2=0x7f975c0d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:19.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 -- 172.21.15.22:0/2060455527 shutdown_connections 2024-09-21T05:48:19.319 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.316+0000 7f9761300640 1 -- 172.21.15.22:0/2060455527 wait complete. 2024-09-21T05:48:19.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.318+0000 7f9761300640 1 Processor -- start 2024-09-21T05:48:19.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.318+0000 7f9761300640 1 -- start start 2024-09-21T05:48:19.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f9761300640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 0x7f975c1f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:19.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f9761300640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f975c0d3ea0 0x7f975c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f9761300640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f975c15ebd0 0x7f975c0b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f9761300640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f975c0b85f0 con 0x7f975c069240 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f9761300640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f975c0b8470 con 0x7f975c15ebd0 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f9761300640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f975c0b8770 con 0x7f975c0d3ea0 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f975affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 0x7f975c1f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f975affd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 0x7f975c1f64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49130/0 (socket says 172.21.15.22:49130) 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f975affd640 1 -- 172.21.15.22:0/2272327034 learned_addr learned my addr 172.21.15.22:0/2272327034 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f975b7fe640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f975c15ebd0 0x7f975c0b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.319+0000 7f975a7fc640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f975c0d3ea0 0x7f975c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f975affd640 1 -- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f975c0d3ea0 msgr2=0x7f975c1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f975affd640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f975c0d3ea0 0x7f975c1579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f975affd640 1 -- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f975c15ebd0 msgr2=0x7f975c0b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f975affd640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f975c15ebd0 0x7f975c0b6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:19.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f975affd640 1 -- 172.21.15.22:0/2272327034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c1f6a20 con 0x7f975c069240 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f975affd640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 0x7f975c1f64e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c760 tx=0x7f974c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c003b70 con 0x7f975c069240 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f974c003d30 con 0x7f975c069240 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c0d00a0 con 0x7f975c069240 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c00aab0 con 0x7f975c069240 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.320+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c0cfdc0 con 0x7f975c069240 2024-09-21T05:48:19.324 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.321+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c068d00 con 0x7f975c069240 2024-09-21T05:48:19.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.322+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f974c00ac50 con 0x7f975c069240 2024-09-21T05:48:19.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.322+0000 7f97437fe640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9748078bc0 0x7f974807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:19.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.323+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f974c09ab90 con 0x7f975c069240 2024-09-21T05:48:19.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.323+0000 7f975a7fc640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9748078bc0 0x7f974807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:19.325 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.323+0000 7f975a7fc640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9748078bc0 0x7f974807b0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9750033000 tx=0x7f9750004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:19.329 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.327+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f975c1f6a20 con 0x7f975c069240 2024-09-21T05:48:19.560 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:19.558+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f975c15c3c0 con 0x7f975c069240 2024-09-21T05:48:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.544+0000 7f97437fe640 1 -- 172.21.15.22:0/2272327034 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v110) ==== 162+0+0 (secure 0 0 0) 0x7f974c063000 con 0x7f975c069240 2024-09-21T05:48:20.546 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9748078bc0 msgr2=0x7f974807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9748078bc0 0x7f974807b0d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9750033000 tx=0x7f9750004730 comp rx=0 tx=0).stop 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 msgr2=0x7f975c1f64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 0x7f975c1f64e0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c760 tx=0x7f974c00cc20 comp rx=0 tx=0).stop 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 shutdown_connections 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f975c15ebd0 0x7f975c0b6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9748078bc0 0x7f974807b0d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f975c0d3ea0 0x7f975c1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 --2- 172.21.15.22:0/2272327034 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f975c069240 0x7f975c1f64e0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 >> 172.21.15.22:0/2272327034 conn(0x7f975c0d4820 msgr2=0x7f975c15c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 shutdown_connections 2024-09-21T05:48:20.550 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.547+0000 7f9761300640 1 -- 172.21.15.22:0/2272327034 wait complete. 2024-09-21T05:48:20.566 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:20.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 -- 172.21.15.22:0/2941195611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc500069410 msgr2=0x7fc500169640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.689 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 --2- 172.21.15.22:0/2941195611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc500069410 0x7fc500169640 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8005fd0 tx=0x7fc4f802a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 -- 172.21.15.22:0/2941195611 shutdown_connections 2024-09-21T05:48:20.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 --2- 172.21.15.22:0/2941195611 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc500069410 0x7fc500169640 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 -- 172.21.15.22:0/2941195611 >> 172.21.15.22:0/2941195611 conn(0x7fc5000d4820 msgr2=0x7fc5000d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:20.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 -- 172.21.15.22:0/2941195611 shutdown_connections 2024-09-21T05:48:20.690 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.687+0000 7fc508162640 1 -- 172.21.15.22:0/2941195611 wait complete. 2024-09-21T05:48:20.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.689+0000 7fc508162640 1 Processor -- start 2024-09-21T05:48:20.691 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.689+0000 7fc508162640 1 -- start start 2024-09-21T05:48:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc508162640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5000d3ea0 0x7fc50015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc508162640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 0x7fc5001504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc508162640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc500153150 0x7fc500155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc508162640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc500169fc0 con 0x7fc50015ebd0 2024-09-21T05:48:20.693 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc508162640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc500169e40 con 0x7fc5000d3ea0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc508162640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc50016a140 con 0x7fc500153150 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.690+0000 7fc5056fd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 0x7fc5001504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 0x7fc5001504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49156/0 (socket says 172.21.15.22:49156) 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 -- 172.21.15.22:0/3785939376 learned_addr learned my addr 172.21.15.22:0/3785939376 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 -- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc500153150 msgr2=0x7fc500155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc500153150 0x7fc500155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 -- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5000d3ea0 msgr2=0x7fc50015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5000d3ea0 0x7fc50015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 -- 172.21.15.22:0/3785939376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc500151ed0 con 0x7fc50015ebd0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc5056fd640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 0x7fc5001504c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400b410 tx=0x7fc4f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4015020 con 0x7fc50015ebd0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4f4011920 con 0x7fc50015ebd0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4011d70 con 0x7fc50015ebd0 2024-09-21T05:48:20.694 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5001521b0 con 0x7fc50015ebd0 2024-09-21T05:48:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.691+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc500152770 con 0x7fc50015ebd0 2024-09-21T05:48:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.692+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5000b7260 con 0x7fc50015ebd0 2024-09-21T05:48:20.695 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.693+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4f4018020 con 0x7fc50015ebd0 2024-09-21T05:48:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.694+0000 7fc4fe7fc640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc4ec078b90 0x7fc4ec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:20.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.694+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fc4f409aef0 con 0x7fc50015ebd0 2024-09-21T05:48:20.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.696+0000 7fc505efe640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc4ec078b90 0x7fc4ec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:20.700 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.698+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5000b7260 con 0x7fc50015ebd0 2024-09-21T05:48:20.702 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.700+0000 7fc505efe640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc4ec078b90 0x7fc4ec07b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0001c10 tx=0x7fc4f0007cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:20.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.927+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc50008d3d0 con 0x7fc50015ebd0 2024-09-21T05:48:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.928+0000 7fc4fe7fc640 1 -- 172.21.15.22:0/3785939376 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5224 (secure 0 0 0) 0x7fc4f401f020 con 0x7fc50015ebd0 2024-09-21T05:48:20.930 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:20.930 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":111,"btime":"2024-09-21T05:48:20:548757+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:20.548736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:replay","state_seq":1,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-21T05:48:20.930 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 111 2024-09-21T05:48:20.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc4ec078b90 msgr2=0x7fc4ec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.932 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc4ec078b90 0x7fc4ec07b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0001c10 tx=0x7fc4f0007cb0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 msgr2=0x7fc5001504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 0x7fc5001504c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400b410 tx=0x7fc4f400b8d0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 shutdown_connections 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc4ec078b90 0x7fc4ec07b0a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc500153150 0x7fc500155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc50015ebd0 0x7fc5001504c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 --2- 172.21.15.22:0/3785939376 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc5000d3ea0 0x7fc50015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 >> 172.21.15.22:0/3785939376 conn(0x7fc5000d4820 msgr2=0x7fc50015a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 shutdown_connections 2024-09-21T05:48:20.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:20.930+0000 7fc508162640 1 -- 172.21.15.22:0/3785939376 wait complete. 2024-09-21T05:48:21.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:48:20.548736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8388': {'gid': 8388, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.22:6835/433765027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 433765027}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 433765027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:48:21.346 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8388:up:replay 2024-09-21T05:48:22.347 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:22.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473c607640 1 -- 172.21.15.22:0/865866299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 msgr2=0x7f471c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:22.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473c607640 1 --2- 172.21.15.22:0/865866299 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f471c004800 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4728001630 tx=0x7f471c005830 comp rx=0 tx=0).stop 2024-09-21T05:48:22.480 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473aba4640 1 -- 172.21.15.22:0/865866299 reap_dead start 2024-09-21T05:48:22.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473c607640 1 -- 172.21.15.22:0/865866299 shutdown_connections 2024-09-21T05:48:22.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473c607640 1 -- 172.21.15.22:0/865866299 >> 172.21.15.22:0/865866299 conn(0x7f47340b2320 msgr2=0x7f47340b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:22.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473c607640 1 -- 172.21.15.22:0/865866299 shutdown_connections 2024-09-21T05:48:22.481 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.478+0000 7f473c607640 1 -- 172.21.15.22:0/865866299 wait complete. 2024-09-21T05:48:22.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.480+0000 7f473c607640 1 Processor -- start 2024-09-21T05:48:22.482 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.480+0000 7f473c607640 1 -- start start 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.481+0000 7f473c607640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f4734142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.481+0000 7f473c607640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47341f5950 0x7f47340ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.481+0000 7f473c607640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47340b13f0 0x7f47340b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.481+0000 7f473c607640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f47340b43f0 con 0x7f471c004000 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.481+0000 7f473c607640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f47340d0270 con 0x7f47341f5950 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.481+0000 7f473c607640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f47340b4570 con 0x7f47340b13f0 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f4734142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f4734142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:49180/0 (socket says 172.21.15.22:49180) 2024-09-21T05:48:22.485 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 -- 172.21.15.22:0/2192569698 learned_addr learned my addr 172.21.15.22:0/2192569698 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f4739ba2640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47341f5950 0x7f47340ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473aba4640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47340b13f0 0x7f47340b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 -- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47340b13f0 msgr2=0x7f47340b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47340b13f0 0x7f47340b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 -- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47341f5950 msgr2=0x7f47340ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47341f5950 0x7f47340ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 -- 172.21.15.22:0/2192569698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47340b1d40 con 0x7f471c004000 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473a3a3640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f4734142ac0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4724011d60 tx=0x7f472400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724003d80 con 0x7f471c004000 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4724011e50 con 0x7f471c004000 2024-09-21T05:48:22.486 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473415be90 con 0x7f471c004000 2024-09-21T05:48:22.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724004d40 con 0x7f471c004000 2024-09-21T05:48:22.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.482+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473415c450 con 0x7f471c004000 2024-09-21T05:48:22.487 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.483+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47341605c0 con 0x7f471c004000 2024-09-21T05:48:22.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.484+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4724004050 con 0x7f471c004000 2024-09-21T05:48:22.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.485+0000 7f4732ffd640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4720078b90 0x7f472007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:22.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.485+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f472409bf00 con 0x7f471c004000 2024-09-21T05:48:22.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.485+0000 7f4739ba2640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4720078b90 0x7f472007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:22.488 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.486+0000 7f4739ba2640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4720078b90 0x7f472007b0a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4728032000 tx=0x7f472802c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:22.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.489+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47340b1d40 con 0x7f471c004000 2024-09-21T05:48:22.723 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.721+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47340b3150 con 0x7f471c004000 2024-09-21T05:48:22.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.722+0000 7f4732ffd640 1 -- 172.21.15.22:0/2192569698 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5224 (secure 0 0 0) 0x7f47240641b0 con 0x7f471c004000 2024-09-21T05:48:22.724 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:22.724 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":113,"btime":"2024-09-21T05:48:22:553152+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:21.555097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-21T05:48:22.724 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 113 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.724+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4720078b90 msgr2=0x7f472007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.724+0000 7f473c607640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4720078b90 0x7f472007b0a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4728032000 tx=0x7f472802c040 comp rx=0 tx=0).stop 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.724+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 msgr2=0x7f4734142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.724+0000 7f473c607640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f4734142ac0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4724011d60 tx=0x7f472400f790 comp rx=0 tx=0).stop 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 shutdown_connections 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4720078b90 0x7f472007b0a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:22.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f47340b13f0 0x7f47340b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f47341f5950 0x7f47340ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 --2- 172.21.15.22:0/2192569698 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f471c004000 0x7f4734142ac0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 >> 172.21.15.22:0/2192569698 conn(0x7f47340b2320 msgr2=0x7f47340b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 shutdown_connections 2024-09-21T05:48:22.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:22.725+0000 7f473c607640 1 -- 172.21.15.22:0/2192569698 wait complete. 2024-09-21T05:48:22.907 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:48:22.909 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:48:23.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:48:21.555097+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8388': {'gid': 8388, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.22:6835/433765027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 433765027}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 433765027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-21T05:48:23.141 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8388:up:rejoin 2024-09-21T05:48:24.142 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:24.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.271+0000 7f91e4733640 1 -- 172.21.15.22:0/2024963699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc08d3d0 msgr2=0x7f91dc068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:24.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.271+0000 7f91e4733640 1 --2- 172.21.15.22:0/2024963699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc08d3d0 0x7f91dc068d00 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009830 tx=0x7f91cc02a530 comp rx=0 tx=0).stop 2024-09-21T05:48:24.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.272+0000 7f91e4733640 1 -- 172.21.15.22:0/2024963699 shutdown_connections 2024-09-21T05:48:24.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.272+0000 7f91e4733640 1 --2- 172.21.15.22:0/2024963699 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc08d3d0 0x7f91dc068d00 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.274 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.272+0000 7f91e4733640 1 --2- 172.21.15.22:0/2024963699 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91dc15ebd0 0x7f91dc0d3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.272+0000 7f91e4733640 1 -- 172.21.15.22:0/2024963699 >> 172.21.15.22:0/2024963699 conn(0x7f91dc0d4820 msgr2=0x7f91dc0d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:24.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.272+0000 7f91e4733640 1 -- 172.21.15.22:0/2024963699 shutdown_connections 2024-09-21T05:48:24.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.272+0000 7f91e4733640 1 -- 172.21.15.22:0/2024963699 wait complete. 2024-09-21T05:48:24.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.274+0000 7f91e4733640 1 Processor -- start 2024-09-21T05:48:24.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.274+0000 7f91e4733640 1 -- start start 2024-09-21T05:48:24.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e4733640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91dc08d3d0 0x7f91dc0ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:24.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e4733640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91dc08d7e0 0x7f91dc14d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:24.278 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e4733640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 0x7f91dc0adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e4733640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0cf910 con 0x7f91dc15ebd0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e4733640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0cf790 con 0x7f91dc08d7e0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e4733640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0cfa90 con 0x7f91dc08d3d0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 0x7f91dc0adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 0x7f91dc0adf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55506/0 (socket says 172.21.15.22:55506) 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 -- 172.21.15.22:0/1894212549 learned_addr learned my addr 172.21.15.22:0/1894212549 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e1cce640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91dc08d7e0 0x7f91dc14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 -- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91dc08d3d0 msgr2=0x7f91dc0ab540 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91dc08d3d0 0x7f91dc0ab540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 -- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91dc08d7e0 msgr2=0x7f91dc14d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91dc08d7e0 0x7f91dc14d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.275+0000 7f91e2cd0640 1 -- 172.21.15.22:0/1894212549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc14d8a0 con 0x7f91dc15ebd0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.276+0000 7f91e2cd0640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 0x7f91dc0adf30 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f91d400cae0 tx=0x7f91d400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:24.279 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.276+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4016020 con 0x7f91dc15ebd0 2024-09-21T05:48:24.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.276+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91d400ad30 con 0x7f91dc15ebd0 2024-09-21T05:48:24.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.276+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc14fcc0 con 0x7f91dc15ebd0 2024-09-21T05:48:24.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.276+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4012680 con 0x7f91dc15ebd0 2024-09-21T05:48:24.280 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.276+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc150280 con 0x7f91dc15ebd0 2024-09-21T05:48:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.277+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91dc068d00 con 0x7f91dc15ebd0 2024-09-21T05:48:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.278+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f91d4019020 con 0x7f91dc15ebd0 2024-09-21T05:48:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.278+0000 7f91d2ffd640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91c0078bc0 0x7f91c007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.278+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f91d409b0c0 con 0x7f91dc15ebd0 2024-09-21T05:48:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.278+0000 7f91e24cf640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91c0078bc0 0x7f91c007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:24.281 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.279+0000 7f91e24cf640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91c0078bc0 0x7f91c007b0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f91d80095d0 tx=0x7f91d8006b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:24.285 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.283+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91d4020030 con 0x7f91dc15ebd0 2024-09-21T05:48:24.514 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.512+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91dc0b9b10 con 0x7f91dc15ebd0 2024-09-21T05:48:24.515 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.513+0000 7f91d2ffd640 1 -- 172.21.15.22:0/1894212549 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5231 (secure 0 0 0) 0x7f91d4063410 con 0x7f91dc15ebd0 2024-09-21T05:48:24.515 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:24.515 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":114,"btime":"2024-09-21T05:48:23:563169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:23.563160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8388,"qdb_cluster":[8388]},"id":8}]} 2024-09-21T05:48:24.515 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 114 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.515+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91c0078bc0 msgr2=0x7f91c007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.515+0000 7f91e4733640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91c0078bc0 0x7f91c007b0d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f91d80095d0 tx=0x7f91d8006b30 comp rx=0 tx=0).stop 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.515+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 msgr2=0x7f91dc0adf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.515+0000 7f91e4733640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 0x7f91dc0adf30 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f91d400cae0 tx=0x7f91d400a060 comp rx=0 tx=0).stop 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 shutdown_connections 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f91dc15ebd0 0x7f91dc0adf30 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f91dc08d7e0 0x7f91dc14d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f91c0078bc0 0x7f91c007b0d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 --2- 172.21.15.22:0/1894212549 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f91dc08d3d0 0x7f91dc0ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:24.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 >> 172.21.15.22:0/1894212549 conn(0x7f91dc0d4820 msgr2=0x7f91dc15e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:24.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 shutdown_connections 2024-09-21T05:48:24.519 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:24.516+0000 7f91e4733640 1 -- 172.21.15.22:0/1894212549 wait complete. 2024-09-21T05:48:24.543 INFO:tasks.ceph.mds.a.smithi022.stderr:2024-09-21T05:48:24.542+0000 7f4bf804f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:48:24.931 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T05:47:52.696945+0000', 'modified': '2024-09-21T05:48:23.563160+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8388}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8388': {'gid': 8388, 'name': 'a', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.22:6835/433765027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.22:6833', 'nonce': 433765027}, {'type': 'v1', 'addr': '172.21.15.22:6835', 'nonce': 433765027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8388, 'qdb_cluster': [8388]} 2024-09-21T05:48:24.932 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T05:48:24.932 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T05:48:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 -- 172.21.15.22:0/2574881709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488073020 msgr2=0x7fc4880b56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.054 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2574881709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488073020 0x7fc4880b56f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fc470009830 tx=0x7fc47002a510 comp rx=0 tx=0).stop 2024-09-21T05:48:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 -- 172.21.15.22:0/2574881709 shutdown_connections 2024-09-21T05:48:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2574881709 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc48815b760 0x7fc4880b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2574881709 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488073020 0x7fc4880b56f0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 -- 172.21.15.22:0/2574881709 >> 172.21.15.22:0/2574881709 conn(0x7fc4880b25a0 msgr2=0x7fc4880b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 -- 172.21.15.22:0/2574881709 shutdown_connections 2024-09-21T05:48:25.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.052+0000 7fc48d40e640 1 -- 172.21.15.22:0/2574881709 wait complete. 2024-09-21T05:48:25.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.054+0000 7fc48d40e640 1 Processor -- start 2024-09-21T05:48:25.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.054+0000 7fc48d40e640 1 -- start start 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.055+0000 7fc48d40e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4880af8b0 0x7fc488146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.055+0000 7fc48d40e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc48815b760 0x7fc488148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.055+0000 7fc48d40e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 0x7fc4881674b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.055+0000 7fc48d40e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880b42d0 con 0x7fc488147f60 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.055+0000 7fc48d40e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880b4150 con 0x7fc48815b760 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.055+0000 7fc48d40e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc4880b4450 con 0x7fc4880af8b0 2024-09-21T05:48:25.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 0x7fc4881674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 0x7fc4881674b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55528/0 (socket says 172.21.15.22:55528) 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 -- 172.21.15.22:0/2214880806 learned_addr learned my addr 172.21.15.22:0/2214880806 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4867fc640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc48815b760 0x7fc488148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4880af8b0 0x7fc488146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 -- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4880af8b0 msgr2=0x7fc488146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4880af8b0 0x7fc488146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 -- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc48815b760 msgr2=0x7fc488148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc48815b760 0x7fc488148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 -- 172.21.15.22:0/2214880806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc488072e10 con 0x7fc488147f60 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 0x7fc4881674b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c970 tx=0x7fc48000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc47f7fe640 1 -- 172.21.15.22:0/2214880806 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480004460 con 0x7fc488147f60 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc47f7fe640 1 -- 172.21.15.22:0/2214880806 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4800077a0 con 0x7fc488147f60 2024-09-21T05:48:25.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc47f7fe640 1 -- 172.21.15.22:0/2214880806 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480007bd0 con 0x7fc488147f60 2024-09-21T05:48:25.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4881f9310 con 0x7fc488147f60 2024-09-21T05:48:25.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.056+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc488073390 con 0x7fc488147f60 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.058+0000 7fc47f7fe640 1 -- 172.21.15.22:0/2214880806 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc4800035f0 con 0x7fc488147f60 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.059+0000 7fc47f7fe640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc468078b90 0x7fc46807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.059+0000 7fc47f7fe640 1 -- 172.21.15.22:0/2214880806 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fc48009b580 con 0x7fc488147f60 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.059+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc468078b90 0x7fc46807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.060+0000 7fc494456740 1 Processor -- start 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.060+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc468078b90 0x7fc46807b0a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc478007fd0 tx=0x7fc478009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.062 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.060+0000 7fc494456740 1 -- start start 2024-09-21T05:48:25.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc494456740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f7bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e8f7c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc494456740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586e8ec4820 con 0x5586e8baa270 2024-09-21T05:48:25.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586e8eb3c50 con 0x5586e8b813b0 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5586e8eb7a60 con 0x5586e8b89590 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f7bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f7bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55542/0 (socket says 172.21.15.22:55542) 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc486ffd640 1 -- 172.21.15.22:0/387017757 learned_addr learned my addr 172.21.15.22:0/387017757 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc4867fc640 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e8f7c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.061+0000 7fc4877fe640 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc486ffd640 1 -- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 msgr2=0x5586e8f993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc486ffd640 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc486ffd640 1 -- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 msgr2=0x5586e8f7c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc486ffd640 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e8f7c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc486ffd640 1 -- 172.21.15.22:0/387017757 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e8c15850 con 0x5586e8baa270 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc486ffd640 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f7bfa0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc478003b60 tx=0x7fc4780021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d984cab7d6c5a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc47cff9640 1 -- 172.21.15.22:0/387017757 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4780052e0 con 0x5586e8baa270 2024-09-21T05:48:25.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc47cff9640 1 -- 172.21.15.22:0/387017757 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc478005570 con 0x5586e8baa270 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.062+0000 7fc47cff9640 1 -- 172.21.15.22:0/387017757 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478021450 con 0x5586e8baa270 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 -- 172.21.15.22:0/387017757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 msgr2=0x5586e8f7bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f7bfa0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc478003b60 tx=0x7fc4780021f0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 -- 172.21.15.22:0/387017757 shutdown_connections 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e8f7c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 --2- 172.21.15.22:0/387017757 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f7bfa0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 -- 172.21.15.22:0/387017757 >> 172.21.15.22:0/387017757 conn(0x5586e8b9e3c0 msgr2=0x5586e8f52260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 -- 172.21.15.22:0/387017757 shutdown_connections 2024-09-21T05:48:25.067 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.063+0000 7fc494456740 1 -- 172.21.15.22:0/387017757 wait complete. 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.064+0000 7fc494456740 1 -- start start 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.064+0000 7fc494456740 1 Processor -- start 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 0x5586e8fa6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f8f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8baa270 0x5586e8f6a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586e8c24620 con 0x5586e8b813b0 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586e8e595f0 con 0x5586e8baa270 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5586e8f7ca20 con 0x5586e8b89590 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.065+0000 7fc494456740 10 client.0 _ll_register_callbacks cb 0x5586e8e5d0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:25.071 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 0x5586e8fa6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 0x5586e8fa6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55546/0 (socket says 172.21.15.22:55546) 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 -- 172.21.15.22:0/847632338 learned_addr learned my addr 172.21.15.22:0/847632338 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc4877fe640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8baa270 0x5586e8f6a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc4867fc640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f8f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 -- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 msgr2=0x5586e8f8f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f8f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 -- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8baa270 msgr2=0x5586e8f6a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8baa270 0x5586e8f6a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 -- 172.21.15.22:0/847632338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e8c8c6b0 con 0x5586e8b813b0 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc4877fe640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8baa270 0x5586e8f6a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc4867fc640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f8f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc486ffd640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 0x5586e8fa6130 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc478014000 tx=0x7fc478001500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff1e9a0c67ec436 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:25.072 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc465ffb640 1 -- 172.21.15.22:0/847632338 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478021920 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc465ffb640 1 -- 172.21.15.22:0/847632338 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fc478021ac0 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc478005570 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc465ffb640 1 -- 172.21.15.22:0/847632338 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7fc478021cc0 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc494456740 10 client.8493 fetch_fsmap learned FSMap version 114 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.066+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x5586e8f6aad0 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc465ffb640 1 -- 172.21.15.22:0/847632338 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc478005570 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc465ffb640 1 -- 172.21.15.22:0/847632338 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 114) ==== 3502+0+0 (secure 0 0 0) 0x7fc47802b480 con 0x5586e8b813b0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc494456740 10 client.8493 fetch_fsmap finished waiting for FSMap version 114 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc494456740 10 client.8493 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc494456740 20 client.8493 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc494456740 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8b68e40 0x5586e8f9a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc494456740 4 client.8493 mds_command: new command op to 8388 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 --> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586e8ec6770 con 0x5586e8b68e40 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc4867fc640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8b68e40 0x5586e8f9a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.073 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc4867fc640 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8b68e40 0x5586e8f9a330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72418b21912619a0 server_cookie=b5e9c50235660203 in_seq=0 out_seq=0 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.067+0000 7fc465ffb640 10 client.8493 ms_handle_connect on v2:172.21.15.22:6833/433765027 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc465ffb640 1 -- 172.21.15.22:0/847632338 <== mds.0 v2:172.21.15.22:6833/433765027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586e8ec6770 con 0x5586e8b68e40 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc465ffb640 10 client.8493 handle_command_reply: tid=0 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 client.8493 shutdown 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 msgr2=0x5586e8fa6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 0x5586e8fa6130 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc478014000 tx=0x7fc478001500 comp rx=0 tx=0).stop 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 shutdown_connections 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8b68e40 0x5586e8f9a330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8baa270 0x5586e8f6a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8f8f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 --2- 172.21.15.22:0/847632338 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8b813b0 0x5586e8fa6130 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc465ffb640 0 client.8493 ms_handle_reset on v2:172.21.15.22:6833/433765027 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 >> 172.21.15.22:0/847632338 conn(0x5586e8b9e3c0 msgr2=0x5586e8f989b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.074 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 shutdown_connections 2024-09-21T05:48:25.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 1 -- 172.21.15.22:0/847632338 wait complete. 2024-09-21T05:48:25.075 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.068+0000 7fc494456740 20 client.8493 trim_cache size 0 max 16384 2024-09-21T05:48:25.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.087+0000 7fc494456740 1 Processor -- start 2024-09-21T05:48:25.089 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.087+0000 7fc494456740 1 -- start start 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b813b0 0x5586e8f7c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b89590 0x5586e8f69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc494456740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc494456740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fc488201260 con 0x5586e8baa270 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880cf840 con 0x5586e8b89590 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fc488200c20 con 0x5586e8b813b0 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f6a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55552/0 (socket says 172.21.15.22:55552) 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.088+0000 7fc4877fe640 1 -- 172.21.15.22:0/2115599900 learned_addr learned my addr 172.21.15.22:0/2115599900 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:25.092 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4867fc640 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b89590 0x5586e8f69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b813b0 0x5586e8f7c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4877fe640 1 -- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b813b0 msgr2=0x5586e8f7c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b813b0 0x5586e8f7c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4877fe640 1 -- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b89590 msgr2=0x5586e8f69c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b89590 0x5586e8f69c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4877fe640 1 -- 172.21.15.22:0/2115599900 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586e8f538f0 con 0x5586e8baa270 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f6a1b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fc480062a30 tx=0x7fc48006a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a619c5663378f90d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc467fff640 1 -- 172.21.15.22:0/2115599900 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480067ca0 con 0x5586e8baa270 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc467fff640 1 -- 172.21.15.22:0/2115599900 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48006cb50 con 0x5586e8baa270 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.089+0000 7fc467fff640 1 -- 172.21.15.22:0/2115599900 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48005f430 con 0x5586e8baa270 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 -- 172.21.15.22:0/2115599900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 msgr2=0x5586e8f6a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f6a1b0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fc480062a30 tx=0x7fc48006a040 comp rx=0 tx=0).stop 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 -- 172.21.15.22:0/2115599900 shutdown_connections 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8f6a1b0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b89590 0x5586e8f69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.093 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 --2- 172.21.15.22:0/2115599900 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b813b0 0x5586e8f7c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 -- 172.21.15.22:0/2115599900 >> 172.21.15.22:0/2115599900 conn(0x5586e8b9e3c0 msgr2=0x5586e8f934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 -- 172.21.15.22:0/2115599900 shutdown_connections 2024-09-21T05:48:25.094 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.090+0000 7fc494456740 1 -- 172.21.15.22:0/2115599900 wait complete. 2024-09-21T05:48:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.091+0000 7fc494456740 1 -- start start 2024-09-21T05:48:25.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.091+0000 7fc494456740 1 Processor -- start 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e90132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8fa9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8faa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x5586e8f92250 con 0x5586e8baa270 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x5586e8c24620 con 0x5586e8b813b0 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x5586e8f92840 con 0x5586e8b89590 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc494456740 10 client.0 _ll_register_callbacks cb 0x7fc470009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8faa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8faa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55560/0 (socket says 172.21.15.22:55560) 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 -- 172.21.15.22:0/2860495149 learned_addr learned my addr 172.21.15.22:0/2860495149 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 -- 172.21.15.22:0/2860495149 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 msgr2=0x5586e8fa9c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8fa9c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 -- 172.21.15.22:0/2860495149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 msgr2=0x5586e90132e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T05:48:25.098 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e90132e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.092+0000 7fc4877fe640 1 -- 172.21.15.22:0/2860495149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586e8fb13f0 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc4877fe640 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8faa140 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fc480062e70 tx=0x7fc480062a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5319071953df75b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc465ffb640 10 client.0 ms_handle_connect on v2:172.21.15.22:3300/0 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc465ffb640 1 -- 172.21.15.22:0/2860495149 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480083070 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc465ffb640 1 -- 172.21.15.22:0/2860495149 <== mon.0 v2:172.21.15.22:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fc48009b0c0 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48006cb50 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc465ffb640 1 -- 172.21.15.22:0/2860495149 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) ==== 24+0+0 (secure 0 0 0) 0x7fc480067ca0 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 10 client.8499 fetch_fsmap learned FSMap version 114 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) -- 0x5586e9016f20 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc465ffb640 1 -- 172.21.15.22:0/2860495149 <== mon.0 v2:172.21.15.22:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc48006cb50 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc465ffb640 1 -- 172.21.15.22:0/2860495149 <== mon.0 v2:172.21.15.22:3300/0 5 ==== fsmap(e 114) ==== 3502+0+0 (secure 0 0 0) 0x7fc48001f030 con 0x5586e8baa270 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 10 client.8499 fetch_fsmap finished waiting for FSMap version 114 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 10 client.8499 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T05:48:25.099 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 20 client.8499 populate_metadata read hostname 'smithi022' 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8f99090 0x5586e8f8db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 4 client.8499 mds_command: new command op to 8388 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 --> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] -- command(tid 0: {"prefix": "status"}) -- 0x5586e8ec2220 con 0x5586e8f99090 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.093+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8f99090 0x5586e8f8db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc465ffb640 10 client.8499 ms_handle_connect on v2:172.21.15.22:6833/433765027 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc486ffd640 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8f99090 0x5586e8f8db90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47520d699ac07cc server_cookie=d4a23e1ffcb3bfd9 in_seq=0 out_seq=0 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc465ffb640 1 -- 172.21.15.22:0/2860495149 <== mds.0 v2:172.21.15.22:6833/433765027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5586e8ec2220 con 0x5586e8f99090 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc465ffb640 10 client.8499 handle_command_reply: tid=0 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 client.8499 shutdown 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 msgr2=0x5586e8faa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8faa140 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fc480062e70 tx=0x7fc480062a30 comp rx=0 tx=0).stop 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 shutdown_connections 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:6833/433765027,v1:172.21.15.22:6835/433765027] conn(0x5586e8f99090 0x5586e8f8db90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x5586e8baa270 0x5586e8faa140 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x5586e8b89590 0x5586e8fa9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc465ffb640 0 client.8499 ms_handle_reset on v2:172.21.15.22:6833/433765027 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 --2- 172.21.15.22:0/2860495149 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x5586e8b813b0 0x5586e90132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 >> 172.21.15.22:0/2860495149 conn(0x5586e8b9e3c0 msgr2=0x5586e8f47920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 shutdown_connections 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 1 -- 172.21.15.22:0/2860495149 wait complete. 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.094+0000 7fc494456740 20 client.8499 trim_cache size 0 max 16384 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "cluster_fsid": "cdac58e0-7b6b-4531-9caa-dad309e0d36b", 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "whoami": 0, 2024-09-21T05:48:25.101 INFO:teuthology.orchestra.run.smithi022.stdout: "id": 8388, 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "want_state": "up:active", 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "state": "up:active", 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "fs_name": "cephfs", 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "rank_uptime": 4.5448736949999997, 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "mdsmap_epoch": 114, 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch": 113, 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "osdmap_epoch_barrier": 113, 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout: "uptime": 6.494412649 2024-09-21T05:48:25.102 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc468078b90 msgr2=0x7fc46807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc468078b90 0x7fc46807b0a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc478007fd0 tx=0x7fc478009040 comp rx=0 tx=0).stop 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 msgr2=0x7fc4881674b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 0x7fc4881674b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c970 tx=0x7fc48000ce30 comp rx=0 tx=0).stop 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 shutdown_connections 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fc488147f60 0x7fc4881674b0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fc48815b760 0x7fc488148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fc468078b90 0x7fc46807b0a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 --2- 172.21.15.22:0/2214880806 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fc4880af8b0 0x7fc488146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 >> 172.21.15.22:0/2214880806 conn(0x7fc4880b25a0 msgr2=0x7fc488151b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 shutdown_connections 2024-09-21T05:48:25.104 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.095+0000 7fc48d40e640 1 -- 172.21.15.22:0/2214880806 wait complete. 2024-09-21T05:48:25.108 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T05:48:25.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 -- 172.21.15.22:0/1245831515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b56e0 msgr2=0x7f28d40b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 --2- 172.21.15.22:0/1245831515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b56e0 0x7f28d40b5af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009860 tx=0x7f28bc02a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 -- 172.21.15.22:0/1245831515 shutdown_connections 2024-09-21T05:48:25.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 --2- 172.21.15.22:0/1245831515 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b56e0 0x7f28d40b5af0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 --2- 172.21.15.22:0/1245831515 >> v2:172.21.15.134:3300/0 conn(0x7f28d40b4ed0 0x7f28d4142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 -- 172.21.15.22:0/1245831515 >> 172.21.15.22:0/1245831515 conn(0x7f28d409ba20 msgr2=0x7f28d40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 -- 172.21.15.22:0/1245831515 shutdown_connections 2024-09-21T05:48:25.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.222+0000 7f28d92bd640 1 -- 172.21.15.22:0/1245831515 wait complete. 2024-09-21T05:48:25.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.224+0000 7f28d92bd640 1 Processor -- start 2024-09-21T05:48:25.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.224+0000 7f28d92bd640 1 -- start start 2024-09-21T05:48:25.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d92bd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b4ed0 0x7f28d413eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d92bd640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d40b56e0 0x7f28d40d1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.228 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d92bd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 0x7f28d40d1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d92bd640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f28d40b1cd0 con 0x7f28d40b6030 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d92bd640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f28d40b1b50 con 0x7f28d40b56e0 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d92bd640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f28d4071f80 con 0x7f28d40b4ed0 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.225+0000 7f28d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 0x7f28d40d1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 0x7f28d40d1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55570/0 (socket says 172.21.15.22:55570) 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 -- 172.21.15.22:0/3680415159 learned_addr learned my addr 172.21.15.22:0/3680415159 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 -- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b4ed0 msgr2=0x7f28d413eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b4ed0 0x7f28d413eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 -- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d40b56e0 msgr2=0x7f28d40d1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d40b56e0 0x7f28d40d1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 -- 172.21.15.22:0/3680415159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d415d590 con 0x7f28d40b6030 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d37fe640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 0x7f28d40d1e70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c3f0 tx=0x7f28c400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011790 con 0x7f28d40b6030 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28c4017070 con 0x7f28d40b6030 2024-09-21T05:48:25.229 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d415d870 con 0x7f28d40b6030 2024-09-21T05:48:25.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011d00 con 0x7f28d40b6030 2024-09-21T05:48:25.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.226+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d415de30 con 0x7f28d40b6030 2024-09-21T05:48:25.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.227+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d415d300 con 0x7f28d40b6030 2024-09-21T05:48:25.230 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.228+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f28c401c460 con 0x7f28d40b6030 2024-09-21T05:48:25.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.229+0000 7f28cb7fe640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f28b4078bc0 0x7f28b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:25.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.229+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f28c409aaa0 con 0x7f28d40b6030 2024-09-21T05:48:25.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.229+0000 7f28d2ffd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f28b4078bc0 0x7f28b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:25.231 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.230+0000 7f28d2ffd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f28b4078bc0 0x7f28b407b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005c40 tx=0x7f28cc005bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:25.235 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.233+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28c4062e80 con 0x7f28d40b6030 2024-09-21T05:48:25.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.460+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f28d4072400 con 0x7f28d40b6030 2024-09-21T05:48:25.462 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.460+0000 7f28cb7fe640 1 -- 172.21.15.22:0/3680415159 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) ==== 53+0+73 (secure 0 0 0) 0x7f28c4062780 con 0x7f28d40b6030 2024-09-21T05:48:25.462 INFO:teuthology.orchestra.run.smithi022.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f28b4078bc0 msgr2=0x7f28b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f28b4078bc0 0x7f28b407b0d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f28cc005c40 tx=0x7f28cc005bd0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 msgr2=0x7f28d40d1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 0x7f28d40d1e70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c3f0 tx=0x7f28c400c7f0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 shutdown_connections 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f28d40b6030 0x7f28d40d1e70 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f28d40b56e0 0x7f28d40d1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f28b4078bc0 0x7f28b407b0d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 --2- 172.21.15.22:0/3680415159 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f28d40b4ed0 0x7f28d413eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:25.465 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 >> 172.21.15.22:0/3680415159 conn(0x7f28d409ba20 msgr2=0x7f28d4142140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 shutdown_connections 2024-09-21T05:48:25.466 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25.462+0000 7f28d92bd640 1 -- 172.21.15.22:0/3680415159 wait complete. 2024-09-21T05:48:25.475 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T05:48:25.475 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi022.front.sepia.ceph.com 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T05:48:25.476 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:25.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:25.476 DEBUG:teuthology.orchestra.run.smithi022:> ip addr 2024-09-21T05:48:25.492 INFO:teuthology.orchestra.run.smithi022.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T05:48:25.492 INFO:teuthology.orchestra.run.smithi022.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 ::1/128 scope host 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9a brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f0 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:6b:fd:9b brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp7s0f1 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e4 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f0 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: inet 172.21.15.22/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft 42238sec preferred_lft 42238sec 2024-09-21T05:48:25.493 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::ec4:7aff:fe59:5de4/64 scope link 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 0c:c4:7a:59:5d:e5 brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: altname enp1s0f1 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::9c20:a1ff:fe02:84d3/64 scope link 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether be:5e:d6:55:da:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::bc5e:d6ff:fe55:da2d/64 scope link 2024-09-21T05:48:25.494 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.495 INFO:teuthology.orchestra.run.smithi022.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T05:48:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: link/ether 22:d2:8d:46:36:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:48:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: inet6 fe80::20d2:8dff:fe46:365c/64 scope link 2024-09-21T05:48:25.495 INFO:teuthology.orchestra.run.smithi022.stdout: valid_lft forever preferred_lft forever 2024-09-21T05:48:25.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:25.495 DEBUG:teuthology.orchestra.run.smithi022:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T05:48:25.557 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:25.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:25.558 DEBUG:teuthology.orchestra.run.smithi022:> route 2024-09-21T05:48:25.616 INFO:teuthology.orchestra.run.smithi022.stdout:Kernel IP routing table 2024-09-21T05:48:25.616 INFO:teuthology.orchestra.run.smithi022.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T05:48:25.616 INFO:teuthology.orchestra.run.smithi022.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T05:48:25.616 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T05:48:25.616 INFO:teuthology.orchestra.run.smithi022.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T05:48:25.616 DEBUG:teuthology.orchestra.run.smithi022:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T05:48:25.616 DEBUG:teuthology.orchestra.run.smithi022:> set -e 2024-09-21T05:48:25.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T05:48:25.616 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T05:48:25.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T05:48:25.617 DEBUG:teuthology.orchestra.run.smithi022:> ') 2024-09-21T05:48:25.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T05:48:25.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:25.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:25.744 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:48:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:48:25.760 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:48:25.760 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:25.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:25.760 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link set brx.0 up 2024-09-21T05:48:25.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:25.826 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:25.882 INFO:teuthology.orchestra.run.smithi022.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T05:48:25.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T05:48:25.883 DEBUG:teuthology.orchestra.run.smithi022:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:25.938 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T05:48:25.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:25.938 DEBUG:teuthology.orchestra.run.smithi022:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:26.033 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:26.033 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:26.033 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:26.044 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:26.044 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:26.044 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:26.044 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:26.044 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:26.050 INFO:teuthology.orchestra.run.smithi022.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T05:48:26.050 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T05:48:26.050 INFO:teuthology.orchestra.run.smithi022.stdout:invalid new device string format 2024-09-21T05:48:26.050 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:3300,172.21.15.134:3300,172.21.15.134:3301" 2024-09-21T05:48:26.050 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: resolved to: "172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790" 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.22:6789,172.21.15.134:6789,172.21.15.134:6790:/ 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=cdac58e0-7b6b-4531-9caa-dad309e0d36b" will pass to kernel 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: You just mounted a file system that supports labels which does not 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T05:48:26.051 INFO:teuthology.orchestra.run.smithi022.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T05:48:26.053 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T05:48:26.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:26.053 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:26.078 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T05:48:26.168 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T05:48:26.169 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.175 INFO:teuthology.orchestra.run.smithi022.stdout:client7720 2024-09-21T05:48:26.175 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:26.175 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:26.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:26.192 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-21T05:48:26.252 INFO:teuthology.orchestra.run.smithi022.stdout:./ 2024-09-21T05:48:26.252 INFO:teuthology.orchestra.run.smithi022.stdout:./parent 2024-09-21T05:48:26.252 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed 2024-09-21T05:48:26.252 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/charlie 2024-09-21T05:48:26.253 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/alpha 2024-09-21T05:48:26.253 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/flushed/bravo 2024-09-21T05:48:26.253 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/unflushed 2024-09-21T05:48:26.253 INFO:teuthology.orchestra.run.smithi022.stdout:./parent/unflushed/jfile 2024-09-21T05:48:26.253 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.253 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.253 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.253 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.253 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-21T05:48:26.253 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:26.359 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-09-21T05:48:26.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.368 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.368 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.369 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.369 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.369 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-21T05:48:26.369 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:26.475 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627776 2024-09-21T05:48:26.478 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.484 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.484 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.484 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.485 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.485 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-21T05:48:26.485 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:26.591 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627777 2024-09-21T05:48:26.594 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.601 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.601 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.601 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.601 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.601 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-21T05:48:26.601 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:26.707 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628280 2024-09-21T05:48:26.710 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.717 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.717 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.717 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.717 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.717 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-21T05:48:26.717 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:26.823 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627778 2024-09-21T05:48:26.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.832 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.833 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.833 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.833 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.833 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-09-21T05:48:26.833 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:26.940 INFO:teuthology.orchestra.run.smithi022.stdout:1099511628279 2024-09-21T05:48:26.943 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:26.951 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:26.951 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:26.951 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:26.951 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:26.951 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-21T05:48:26.951 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:27.057 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627779 2024-09-21T05:48:27.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:27.066 DEBUG:teuthology.orchestra.run.smithi022:> stdin-killer -- python3 -c ' 2024-09-21T05:48:27.066 DEBUG:teuthology.orchestra.run.smithi022:> import os 2024-09-21T05:48:27.066 DEBUG:teuthology.orchestra.run.smithi022:> import stat 2024-09-21T05:48:27.066 DEBUG:teuthology.orchestra.run.smithi022:> 2024-09-21T05:48:27.066 DEBUG:teuthology.orchestra.run.smithi022:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-21T05:48:27.066 DEBUG:teuthology.orchestra.run.smithi022:> ' 2024-09-21T05:48:27.172 INFO:teuthology.orchestra.run.smithi022.stdout:1099511627780 2024-09-21T05:48:27.175 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T05:48:27.181 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T05:48:27.181 DEBUG:teuthology.orchestra.run.smithi022:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:48:27.266 DEBUG:teuthology.orchestra.run.smithi134:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T05:48:27.308 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T05:48:27.308 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:48:27.308 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:27.308 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:27.324 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:27.324 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:27.381 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 -- 172.21.15.22:0/3420178257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 msgr2=0x7f27740d3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 --2- 172.21.15.22:0/3420178257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27740d3ea0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f277406efa0 tx=0x7f276402a5a0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 -- 172.21.15.22:0/3420178257 shutdown_connections 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 --2- 172.21.15.22:0/3420178257 >> v2:172.21.15.134:3300/0 conn(0x7f277408d3d0 0x7f2774068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 --2- 172.21.15.22:0/3420178257 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d7e0 0x7f2774069240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 --2- 172.21.15.22:0/3420178257 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27740d3ea0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 -- 172.21.15.22:0/3420178257 >> 172.21.15.22:0/3420178257 conn(0x7f27740d4820 msgr2=0x7f27740d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 -- 172.21.15.22:0/3420178257 shutdown_connections 2024-09-21T05:48:27.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.536+0000 7f277b207640 1 -- 172.21.15.22:0/3420178257 wait complete. 2024-09-21T05:48:27.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.538+0000 7f277b207640 1 Processor -- start 2024-09-21T05:48:27.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.538+0000 7f277b207640 1 -- start start 2024-09-21T05:48:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f277b207640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d3d0 0x7f27741f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f277b207640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277408d7e0 0x7f27740cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f277b207640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27741f90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f277b207640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f27740cf6e0 con 0x7f277415ebd0 2024-09-21T05:48:27.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f277b207640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f27740cf560 con 0x7f277408d7e0 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f277b207640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f27740cf860 con 0x7f277408d3d0 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27741f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27741f90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55584/0 (socket says 172.21.15.22:55584) 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 -- 172.21.15.22:0/1621880599 learned_addr learned my addr 172.21.15.22:0/1621880599 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f2778fa3640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d3d0 0x7f27741f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f2773fff640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277408d7e0 0x7f27740cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 -- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d3d0 msgr2=0x7f27741f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d3d0 0x7f27741f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 -- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277408d7e0 msgr2=0x7f27740cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277408d7e0 0x7f27740cbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 -- 172.21.15.22:0/1621880599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2774158ec0 con 0x7f277415ebd0 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f2773fff640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277408d7e0 0x7f27740cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f2778fa3640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d3d0 0x7f27741f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:27.544 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27797a4640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27741f90b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f276c00b490 tx=0x7f276c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.540+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c015020 con 0x7f277415ebd0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.541+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f276c011940 con 0x7f277415ebd0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.541+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276c011d90 con 0x7f277415ebd0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.541+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27741594b0 con 0x7f277415ebd0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.541+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277415b170 con 0x7f277415ebd0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.542+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27740cfe00 con 0x7f277415ebd0 2024-09-21T05:48:27.545 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.542+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f276c018020 con 0x7f277415ebd0 2024-09-21T05:48:27.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.543+0000 7f27717fa640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2754078bc0 0x7f275407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:27.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.543+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f276c09ab30 con 0x7f277415ebd0 2024-09-21T05:48:27.546 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.544+0000 7f2778fa3640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2754078bc0 0x7f275407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:27.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.547+0000 7f2778fa3640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2754078bc0 0x7f275407b0d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2764033000 tx=0x7f2764004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:27.549 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.548+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f276c01f020 con 0x7f277415ebd0 2024-09-21T05:48:27.748 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.746+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f277406b160 con 0x7f277415ebd0 2024-09-21T05:48:27.749 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.747+0000 7f27717fa640 1 -- 172.21.15.22:0/1621880599 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v113) ==== 80+0+246 (secure 0 0 0) 0x7f276c062fa0 con 0x7f277415ebd0 2024-09-21T05:48:27.749 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6835/3772214991 2024-09-22T05:48:18.533817+0000 2024-09-21T05:48:27.749 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.22:6833/3772214991 2024-09-22T05:48:18.533817+0000 2024-09-21T05:48:27.749 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6837/4249862141 2024-09-22T05:48:09.072398+0000 2024-09-21T05:48:27.749 INFO:teuthology.orchestra.run.smithi022.stdout:172.21.15.134:6835/4249862141 2024-09-22T05:48:09.072398+0000 2024-09-21T05:48:27.750 INFO:teuthology.orchestra.run.smithi022.stderr:listed 4 entries 2024-09-21T05:48:27.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2754078bc0 msgr2=0x7f275407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:27.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2754078bc0 0x7f275407b0d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f2764033000 tx=0x7f2764004730 comp rx=0 tx=0).stop 2024-09-21T05:48:27.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 msgr2=0x7f27741f90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:27.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27741f90b0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f276c00b490 tx=0x7f276c00b950 comp rx=0 tx=0).stop 2024-09-21T05:48:27.751 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 shutdown_connections 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f277415ebd0 0x7f27741f90b0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f277408d7e0 0x7f27740cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f2754078bc0 0x7f275407b0d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 --2- 172.21.15.22:0/1621880599 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f277408d3d0 0x7f27741f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 >> 172.21.15.22:0/1621880599 conn(0x7f27740d4820 msgr2=0x7f27741618b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 shutdown_connections 2024-09-21T05:48:27.752 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:27.749+0000 7f277b207640 1 -- 172.21.15.22:0/1621880599 wait complete. 2024-09-21T05:48:27.762 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T05:48:27.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:27.762 DEBUG:teuthology.orchestra.run.smithi022:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T05:48:27.790 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:48:27.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:27.790 DEBUG:teuthology.orchestra.run.smithi022:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T05:48:27.847 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:48:27.847 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:48:27.847 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:27.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:48:27.866 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:48:27.867 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:27.867 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:27.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:48:27.903 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T05:48:27.903 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:48:27.903 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:27.925 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:48:27.925 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:28.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 -- 172.21.15.22:0/2806039506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815bd10 msgr2=0x7fad080cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/2806039506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815bd10 0x7fad080cfeb0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fad080b36b0 tx=0x7facf002a5f0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 -- 172.21.15.22:0/2806039506 shutdown_connections 2024-09-21T05:48:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/2806039506 >> v2:172.21.15.134:3300/0 conn(0x7fad0815f340 0x7fad0806f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/2806039506 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815bd10 0x7fad080cfeb0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 -- 172.21.15.22:0/2806039506 >> 172.21.15.22:0/2806039506 conn(0x7fad080b2320 msgr2=0x7fad080b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 -- 172.21.15.22:0/2806039506 shutdown_connections 2024-09-21T05:48:28.060 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.057+0000 7fad0d7ad640 1 -- 172.21.15.22:0/2806039506 wait complete. 2024-09-21T05:48:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.059+0000 7fad0d7ad640 1 Processor -- start 2024-09-21T05:48:28.061 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.059+0000 7fad0d7ad640 1 -- start start 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad0d7ad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad080cd130 0x7fad08153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad0d7ad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad0815bd10 0x7fad081f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad0d7ad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 0x7fad081f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad0d7ad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fad080b4fa0 con 0x7fad0815f340 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad0d7ad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fad080b4e20 con 0x7fad080cd130 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad0d7ad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fad080b5120 con 0x7fad0815bd10 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 0x7fad081f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad077fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 0x7fad081f7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55606/0 (socket says 172.21.15.22:55606) 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.060+0000 7fad077fe640 1 -- 172.21.15.22:0/3627190169 learned_addr learned my addr 172.21.15.22:0/3627190169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad06ffd640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad080cd130 0x7fad08153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad067fc640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad0815bd10 0x7fad081f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad077fe640 1 -- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad0815bd10 msgr2=0x7fad081f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad077fe640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad0815bd10 0x7fad081f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.064 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad077fe640 1 -- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad080cd130 msgr2=0x7fad08153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad077fe640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad080cd130 0x7fad08153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad077fe640 1 -- 172.21.15.22:0/3627190169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad081536c0 con 0x7fad0815f340 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad067fc640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad0815bd10 0x7fad081f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad077fe640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 0x7fad081f7440 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7facf800d620 tx=0x7facf800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8017070 con 0x7fad0815f340 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facf8005310 con 0x7fad0815f340 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad08155e70 con 0x7fad0815f340 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf80047a0 con 0x7fad0815f340 2024-09-21T05:48:28.065 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.061+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad08156430 con 0x7fad0815f340 2024-09-21T05:48:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.062+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad08155be0 con 0x7fad0815f340 2024-09-21T05:48:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.063+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7facf8004940 con 0x7fad0815f340 2024-09-21T05:48:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.063+0000 7facff7fe640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face8078bc0 0x7face807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.064+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7facf809ad60 con 0x7fad0815f340 2024-09-21T05:48:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.064+0000 7fad06ffd640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face8078bc0 0x7face807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.066 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.064+0000 7fad06ffd640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face8078bc0 0x7face807b0d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad00001e20 tx=0x7fad00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:28.070 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.068+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fad081536c0 con 0x7fad0815f340 2024-09-21T05:48:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.293+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad080b5640 con 0x7fad0815f340 2024-09-21T05:48:28.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.293+0000 7facff7fe640 1 -- 172.21.15.22:0/3627190169 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5231 (secure 0 0 0) 0x7facf8062a50 con 0x7fad0815f340 2024-09-21T05:48:28.295 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:28.296 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":114,"btime":"2024-09-21T05:48:23:563169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:23.563160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8388,"qdb_cluster":[8388]},"id":8}]} 2024-09-21T05:48:28.296 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 114 2024-09-21T05:48:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face8078bc0 msgr2=0x7face807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face8078bc0 0x7face807b0d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fad00001e20 tx=0x7fad00009040 comp rx=0 tx=0).stop 2024-09-21T05:48:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 msgr2=0x7fad081f7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 0x7fad081f7440 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7facf800d620 tx=0x7facf800dae0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 shutdown_connections 2024-09-21T05:48:28.298 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fad0815f340 0x7fad081f7440 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7face8078bc0 0x7face807b0d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fad0815bd10 0x7fad081f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 --2- 172.21.15.22:0/3627190169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fad080cd130 0x7fad08153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 >> 172.21.15.22:0/3627190169 conn(0x7fad080b2320 msgr2=0x7fad08072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 shutdown_connections 2024-09-21T05:48:28.299 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.296+0000 7fad0d7ad640 1 -- 172.21.15.22:0/3627190169 wait complete. 2024-09-21T05:48:28.310 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 -- 172.21.15.22:0/46555684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40b5d30 msgr2=0x7fb2c40cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 --2- 172.21.15.22:0/46555684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40cf3d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009830 tx=0x7fb2b402a4a0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 -- 172.21.15.22:0/46555684 shutdown_connections 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 --2- 172.21.15.22:0/46555684 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2c40cf910 0x7fb2c40cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 --2- 172.21.15.22:0/46555684 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40cf3d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 -- 172.21.15.22:0/46555684 >> 172.21.15.22:0/46555684 conn(0x7fb2c40d1220 msgr2=0x7fb2c40d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 -- 172.21.15.22:0/46555684 shutdown_connections 2024-09-21T05:48:28.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.425+0000 7fb2c8958640 1 -- 172.21.15.22:0/46555684 wait complete. 2024-09-21T05:48:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.427+0000 7fb2c8958640 1 Processor -- start 2024-09-21T05:48:28.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.427+0000 7fb2c8958640 1 -- start start 2024-09-21T05:48:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c8958640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2c40b4120 0x7fb2c4150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c8958640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c8958640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40cf910 0x7fb2c415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c8958640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40b4c40 con 0x7fb2c40b5d30 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c8958640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c40d3820 con 0x7fb2c40cf910 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c8958640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fb2c40b4dc0 con 0x7fb2c40b4120 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55614/0 (socket says 172.21.15.22:55614) 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.428+0000 7fb2c1d9b640 1 -- 172.21.15.22:0/65188259 learned_addr learned my addr 172.21.15.22:0/65188259 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c1d9b640 1 -- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2c40b4120 msgr2=0x7fb2c4150d00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c1d9b640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2c40b4120 0x7fb2c4150d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c1d9b640 1 -- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40cf910 msgr2=0x7fb2c415bcd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c1d9b640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40cf910 0x7fb2c415bcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c1d9b640 1 -- 172.21.15.22:0/65188259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c415ac30 con 0x7fb2c40b5d30 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c1d9b640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40c9260 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800e790 tx=0x7fb2b800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b800eea0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2b800cdb0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c415af10 con 0x7fb2c40b5d30 2024-09-21T05:48:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8010690 con 0x7fb2c40b5d30 2024-09-21T05:48:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.429+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c415b4d0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.430+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c40b5300 con 0x7fb2c40b5d30 2024-09-21T05:48:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.431+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb2b8010830 con 0x7fb2c40b5d30 2024-09-21T05:48:28.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.431+0000 7fb2b2ffd640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb2a4078b90 0x7fb2a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.431+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fb2b809b8e0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.432+0000 7fb2c259c640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb2a4078b90 0x7fb2a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.432+0000 7fb2c259c640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb2a4078b90 0x7fb2a407b0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4034000 tx=0x7fb2b4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:28.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.436+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2b8063bb0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.661 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.660+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2c4161fd0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.662 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.660+0000 7fb2b2ffd640 1 -- 172.21.15.22:0/65188259 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5231 (secure 0 0 0) 0x7fb2b80634b0 con 0x7fb2c40b5d30 2024-09-21T05:48:28.662 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:28.662 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":114,"btime":"2024-09-21T05:48:23:563169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:23.563160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8388,"qdb_cluster":[8388]},"id":8}]} 2024-09-21T05:48:28.663 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 114 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb2a4078b90 msgr2=0x7fb2a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb2a4078b90 0x7fb2a407b0a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4034000 tx=0x7fb2b4004730 comp rx=0 tx=0).stop 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 msgr2=0x7fb2c40c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40c9260 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800e790 tx=0x7fb2b800ec50 comp rx=0 tx=0).stop 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 shutdown_connections 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fb2c40cf910 0x7fb2c415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fb2c40b5d30 0x7fb2c40c9260 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fb2a4078b90 0x7fb2a407b0a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 --2- 172.21.15.22:0/65188259 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fb2c40b4120 0x7fb2c4150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.665 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.662+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 >> 172.21.15.22:0/65188259 conn(0x7fb2c40d1220 msgr2=0x7fb2c40b1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:28.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.663+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 shutdown_connections 2024-09-21T05:48:28.666 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.663+0000 7fb2c8958640 1 -- 172.21.15.22:0/65188259 wait complete. 2024-09-21T05:48:28.676 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:48:28.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 -- 172.21.15.22:0/4082893516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff120073c50 msgr2=0x7ff12006fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 --2- 172.21.15.22:0/4082893516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff120073c50 0x7ff12006fae0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7ff1200b6ab0 tx=0x7ff10802a560 comp rx=0 tx=0).stop 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 -- 172.21.15.22:0/4082893516 shutdown_connections 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 --2- 172.21.15.22:0/4082893516 >> v2:172.21.15.134:3300/0 conn(0x7ff120070020 0x7ff1200b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 --2- 172.21.15.22:0/4082893516 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff120073c50 0x7ff12006fae0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 -- 172.21.15.22:0/4082893516 >> 172.21.15.22:0/4082893516 conn(0x7ff12006de80 msgr2=0x7ff12006e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 -- 172.21.15.22:0/4082893516 shutdown_connections 2024-09-21T05:48:28.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.791+0000 7ff1253df640 1 -- 172.21.15.22:0/4082893516 wait complete. 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.793+0000 7ff1253df640 1 Processor -- start 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.793+0000 7ff1253df640 1 -- start start 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff1253df640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff120070020 0x7ff120141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff1253df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1201526f0 0x7ff1201f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff1253df640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 0x7ff1200d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff1253df640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff1200b5500 con 0x7ff1200d0d90 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff1253df640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff1200b5380 con 0x7ff1201526f0 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff1253df640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff1200b5680 con 0x7ff120070020 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 0x7ff1200d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff11f7fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 0x7ff1200d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55644/0 (socket says 172.21.15.22:55644) 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.794+0000 7ff11f7fe640 1 -- 172.21.15.22:0/2863741400 learned_addr learned my addr 172.21.15.22:0/2863741400 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11e7fc640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1201526f0 0x7ff1201f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11effd640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff120070020 0x7ff120141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11f7fe640 1 -- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff120070020 msgr2=0x7ff120141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11f7fe640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff120070020 0x7ff120141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11f7fe640 1 -- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1201526f0 msgr2=0x7ff1201f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11f7fe640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1201526f0 0x7ff1201f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11f7fe640 1 -- 172.21.15.22:0/2863741400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1201f7520 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11e7fc640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1201526f0 0x7ff1201f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff11f7fe640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 0x7ff1200d11a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7ff11800d690 tx=0x7ff11800db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118017070 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff118005310 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118009820 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1200d16e0 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.795+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff120145fe0 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.796+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff120145d50 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.797+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1180099c0 con 0x7ff1200d0d90 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.798+0000 7ff1177fe640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff100078bc0 0x7ff10007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:28.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.798+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7ff11809ae50 con 0x7ff1200d0d90 2024-09-21T05:48:28.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.798+0000 7ff11effd640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff100078bc0 0x7ff10007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:28.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.799+0000 7ff11effd640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff100078bc0 0x7ff10007b0d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff120074550 tx=0x7ff110009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:28.890 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.803+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1200d16e0 con 0x7ff1200d0d90 2024-09-21T05:48:28.994 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.992+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1200b5bd0 con 0x7ff1200d0d90 2024-09-21T05:48:28.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.993+0000 7ff1177fe640 1 -- 172.21.15.22:0/2863741400 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17029 (secure 0 0 0) 0x7ff1201f7520 con 0x7ff1200d0d90 2024-09-21T05:48:28.996 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:28.996 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":113,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:48:18.533848+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:47:50.363316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T05:47:51.649557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.b","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6835/3772214991":"2024-09-22T05:48:18.533817+0000","172.21.15.22:6833/3772214991":"2024-09-22T05:48:18.533817+0000","172.21.15.134:6837/4249862141":"2024-09-22T05:48:09.072398+0000","172.21.15.134:6835/4249862141":"2024-09-22T05:48:09.072398+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.995+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff100078bc0 msgr2=0x7ff10007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.995+0000 7ff1253df640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff100078bc0 0x7ff10007b0d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff120074550 tx=0x7ff110009040 comp rx=0 tx=0).stop 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 msgr2=0x7ff1200d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 0x7ff1200d11a0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7ff11800d690 tx=0x7ff11800db50 comp rx=0 tx=0).stop 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 shutdown_connections 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff1200d0d90 0x7ff1200d11a0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff1201526f0 0x7ff1201f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff100078bc0 0x7ff10007b0d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.998 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 --2- 172.21.15.22:0/2863741400 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff120070020 0x7ff120141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 >> 172.21.15.22:0/2863741400 conn(0x7ff12006de80 msgr2=0x7ff12006cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 shutdown_connections 2024-09-21T05:48:28.999 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:28.996+0000 7ff1253df640 1 -- 172.21.15.22:0/2863741400 wait complete. 2024-09-21T05:48:29.009 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:48:29.009 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 -- 172.21.15.22:0/2173226692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 msgr2=0x7fcaa00b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 --2- 172.21.15.22:0/2173226692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa00b0400 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00afbb0 tx=0x7fca9802a540 comp rx=0 tx=0).stop 2024-09-21T05:48:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 -- 172.21.15.22:0/2173226692 shutdown_connections 2024-09-21T05:48:29.125 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 --2- 172.21.15.22:0/2173226692 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa00b0400 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 --2- 172.21.15.22:0/2173226692 >> v2:172.21.15.134:3300/0 conn(0x7fcaa00b5df0 0x7fcaa00afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 -- 172.21.15.22:0/2173226692 >> 172.21.15.22:0/2173226692 conn(0x7fcaa009ba20 msgr2=0x7fcaa00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 -- 172.21.15.22:0/2173226692 shutdown_connections 2024-09-21T05:48:29.126 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.123+0000 7fcaa820e640 1 -- 172.21.15.22:0/2173226692 wait complete. 2024-09-21T05:48:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.125+0000 7fcaa820e640 1 Processor -- start 2024-09-21T05:48:29.127 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.125+0000 7fcaa820e640 1 -- start start 2024-09-21T05:48:29.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa820e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcaa009be30 0x7fcaa014fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa820e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa01f9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.129 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa820e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa0153a90 0x7fcaa0153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa820e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa00b0ac0 con 0x7fcaa015ecf0 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa820e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa00b0940 con 0x7fcaa0153a90 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa820e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa00b0c40 con 0x7fcaa009be30 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa57a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa01f9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.126+0000 7fcaa57a9640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa01f9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55666/0 (socket says 172.21.15.22:55666) 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 -- 172.21.15.22:0/3802385523 learned_addr learned my addr 172.21.15.22:0/3802385523 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa67ab640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa0153a90 0x7fcaa0153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 -- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcaa009be30 msgr2=0x7fcaa014fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcaa009be30 0x7fcaa014fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 -- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa0153a90 msgr2=0x7fcaa0153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa0153a90 0x7fcaa0153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.130 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 -- 172.21.15.22:0/3802385523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa01543e0 con 0x7fcaa015ecf0 2024-09-21T05:48:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa57a9640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa01f9aa0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d700 tx=0x7fca9400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94017070 con 0x7fcaa015ecf0 2024-09-21T05:48:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca94005310 con 0x7fcaa015ecf0 2024-09-21T05:48:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94009820 con 0x7fcaa015ecf0 2024-09-21T05:48:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa01f5f40 con 0x7fcaa015ecf0 2024-09-21T05:48:29.131 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.127+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01f6500 con 0x7fcaa015ecf0 2024-09-21T05:48:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.128+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa00b1160 con 0x7fcaa015ecf0 2024-09-21T05:48:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.129+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca940099c0 con 0x7fcaa015ecf0 2024-09-21T05:48:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.130+0000 7fca9e7fc640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca8c078b90 0x7fca8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.130+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7fca9409b380 con 0x7fcaa015ecf0 2024-09-21T05:48:29.132 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.130+0000 7fcaa5faa640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca8c078b90 0x7fca8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.134 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.132+0000 7fcaa5faa640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca8c078b90 0x7fca8c07b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca9000ab90 tx=0x7fca90009230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:29.136 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.134+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcaa01543e0 con 0x7fcaa015ecf0 2024-09-21T05:48:29.359 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.357+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa0069d20 con 0x7fcaa015ecf0 2024-09-21T05:48:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.358+0000 7fca9e7fc640 1 -- 172.21.15.22:0/3802385523 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5231 (secure 0 0 0) 0x7fca94063080 con 0x7fcaa015ecf0 2024-09-21T05:48:29.360 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:29.360 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":114,"btime":"2024-09-21T05:48:23:563169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:23.563160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8388,"qdb_cluster":[8388]},"id":8}]} 2024-09-21T05:48:29.360 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 114 2024-09-21T05:48:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca8c078b90 msgr2=0x7fca8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca8c078b90 0x7fca8c07b0a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fca9000ab90 tx=0x7fca90009230 comp rx=0 tx=0).stop 2024-09-21T05:48:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 msgr2=0x7fcaa01f9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.362 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa01f9aa0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d700 tx=0x7fca9400dbc0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 shutdown_connections 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fcaa0153a90 0x7fcaa0153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fcaa015ecf0 0x7fcaa01f9aa0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fca8c078b90 0x7fca8c07b0a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 --2- 172.21.15.22:0/3802385523 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fcaa009be30 0x7fcaa014fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 >> 172.21.15.22:0/3802385523 conn(0x7fcaa009ba20 msgr2=0x7fcaa015a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 shutdown_connections 2024-09-21T05:48:29.363 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.360+0000 7fcaa820e640 1 -- 172.21.15.22:0/3802385523 wait complete. 2024-09-21T05:48:29.373 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:48:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 -- 172.21.15.22:0/3674428844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 msgr2=0x7f53a40b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.489 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 --2- 172.21.15.22:0/3674428844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a40b5550 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f53a40cedb0 tx=0x7f539402aa30 comp rx=0 tx=0).stop 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 -- 172.21.15.22:0/3674428844 shutdown_connections 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 --2- 172.21.15.22:0/3674428844 >> v2:172.21.15.134:3300/0 conn(0x7f53a40b5fa0 0x7f53a40b63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 --2- 172.21.15.22:0/3674428844 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53a40d2fc0 0x7f53a40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 --2- 172.21.15.22:0/3674428844 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a40b5550 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 -- 172.21.15.22:0/3674428844 >> 172.21.15.22:0/3674428844 conn(0x7f53a40d4860 msgr2=0x7f53a40d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 -- 172.21.15.22:0/3674428844 shutdown_connections 2024-09-21T05:48:29.490 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.487+0000 7f53a99ba640 1 -- 172.21.15.22:0/3674428844 wait complete. 2024-09-21T05:48:29.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.489+0000 7f53a99ba640 1 Processor -- start 2024-09-21T05:48:29.491 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.489+0000 7f53a99ba640 1 -- start start 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.490+0000 7f53a99ba640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53a40b5fa0 0x7f53a40ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.490+0000 7f53a99ba640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a4151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.490+0000 7f53a99ba640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a40d2fc0 0x7f53a4160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.490+0000 7f53a99ba640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40d0f30 con 0x7f53a40d09a0 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.490+0000 7f53a99ba640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40d0db0 con 0x7f53a40d2fc0 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.490+0000 7f53a99ba640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53a40cf350 con 0x7f53a40b5fa0 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a4151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a4151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55690/0 (socket says 172.21.15.22:55690) 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 -- 172.21.15.22:0/2141664531 learned_addr learned my addr 172.21.15.22:0/2141664531 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a2ffd640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53a40b5fa0 0x7f53a40ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 -- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53a40b5fa0 msgr2=0x7f53a40ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.494 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53a40b5fa0 0x7f53a40ca920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a37fe640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a40d2fc0 0x7f53a4160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 -- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a40d2fc0 msgr2=0x7f53a4160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a40d2fc0 0x7f53a4160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 -- 172.21.15.22:0/2141664531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a40c9350 con 0x7f53a40d09a0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a37fe640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a40d2fc0 0x7f53a4160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a27fc640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a4151200 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f538c00b410 tx=0x7f538c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c015020 con 0x7f53a40d09a0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f538c011940 con 0x7f53a40d09a0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a4151740 con 0x7f53a40d09a0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c011d90 con 0x7f53a40d09a0 2024-09-21T05:48:29.495 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.491+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a40c9880 con 0x7f53a40d09a0 2024-09-21T05:48:29.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.492+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a40b5550 con 0x7f53a40d09a0 2024-09-21T05:48:29.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.493+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f538c018020 con 0x7f53a40d09a0 2024-09-21T05:48:29.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.494+0000 7f539bfff640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5384078b90 0x7f538407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.494+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f538c09ada0 con 0x7f53a40d09a0 2024-09-21T05:48:29.496 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.494+0000 7f53a2ffd640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5384078b90 0x7f538407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.497 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.495+0000 7f53a2ffd640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5384078b90 0x7f538407b0a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5394035000 tx=0x7f539403b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:29.500 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.499+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f538c01f020 con 0x7f53a40d09a0 2024-09-21T05:48:29.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.722+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53a40c9610 con 0x7f53a40d09a0 2024-09-21T05:48:29.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.722+0000 7f539bfff640 1 -- 172.21.15.22:0/2141664531 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5231 (secure 0 0 0) 0x7f538c0630e0 con 0x7f53a40d09a0 2024-09-21T05:48:29.725 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:29.725 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":114,"btime":"2024-09-21T05:48:23:563169+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T05:47:52.696945+0000","modified":"2024-09-21T05:48:23.563160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8388},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8388":{"gid":8388,"name":"a","rank":0,"incarnation":111,"state":"up:active","state_seq":4,"addr":"172.21.15.22:6835/433765027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":433765027},{"type":"v1","addr":"172.21.15.22:6835","nonce":433765027}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8388,"qdb_cluster":[8388]},"id":8}]} 2024-09-21T05:48:29.725 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 114 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5384078b90 msgr2=0x7f538407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5384078b90 0x7f538407b0a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5394035000 tx=0x7f539403b040 comp rx=0 tx=0).stop 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 msgr2=0x7f53a4151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a4151200 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f538c00b410 tx=0x7f538c00b8d0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 shutdown_connections 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53a40d2fc0 0x7f53a4160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53a40d09a0 0x7f53a4151200 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f5384078b90 0x7f538407b0a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 --2- 172.21.15.22:0/2141664531 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53a40b5fa0 0x7f53a40ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.724+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 >> 172.21.15.22:0/2141664531 conn(0x7f53a40d4860 msgr2=0x7f53a40b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.725+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 shutdown_connections 2024-09-21T05:48:29.728 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.725+0000 7f53a99ba640 1 -- 172.21.15.22:0/2141664531 wait complete. 2024-09-21T05:48:29.738 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 -- 172.21.15.22:0/369407472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 msgr2=0x7f98500b0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 --2- 172.21.15.22:0/369407472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f98500b0400 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9848005fd0 tx=0x7f984802aa90 comp rx=0 tx=0).stop 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 -- 172.21.15.22:0/369407472 shutdown_connections 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 --2- 172.21.15.22:0/369407472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f98500b0400 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 --2- 172.21.15.22:0/369407472 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f98500b5df0 0x7f98500afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 -- 172.21.15.22:0/369407472 >> 172.21.15.22:0/369407472 conn(0x7f985009ba20 msgr2=0x7f98500b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 -- 172.21.15.22:0/369407472 shutdown_connections 2024-09-21T05:48:29.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.853+0000 7f98579df640 1 -- 172.21.15.22:0/369407472 wait complete. 2024-09-21T05:48:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.854+0000 7f98579df640 1 Processor -- start 2024-09-21T05:48:29.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.855+0000 7f98579df640 1 -- start start 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f98579df640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985009be30 0x7f985015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f98579df640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f9850152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f98579df640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850152cc0 0x7f985015cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f98579df640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f98500b0ac0 con 0x7f985015ecf0 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f98579df640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f98500b0940 con 0x7f9850152cc0 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f98579df640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f98500b0c40 con 0x7f985009be30 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f9850152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f9850152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55710/0 (socket says 172.21.15.22:55710) 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 -- 172.21.15.22:0/2841847169 learned_addr learned my addr 172.21.15.22:0/2841847169 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f985577b640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985009be30 0x7f985015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9855f7c640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850152cc0 0x7f985015cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 -- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985009be30 msgr2=0x7f985015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985009be30 0x7f985015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 -- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850152cc0 msgr2=0x7f985015cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850152cc0 0x7f985015cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f9854f7a640 1 -- 172.21.15.22:0/2841847169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98501588f0 con 0x7f985015ecf0 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.856+0000 7f985577b640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985009be30 0x7f985015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.857+0000 7f9854f7a640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f9850152430 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f984000c970 tx=0x7f984000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.857+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840007bb0 con 0x7f985015ecf0 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.857+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9840007e40 con 0x7f985015ecf0 2024-09-21T05:48:29.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.857+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840005280 con 0x7f985015ecf0 2024-09-21T05:48:29.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.857+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9850158610 con 0x7f985015ecf0 2024-09-21T05:48:29.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.857+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f985015b630 con 0x7f985015ecf0 2024-09-21T05:48:29.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.858+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9850158380 con 0x7f985015ecf0 2024-09-21T05:48:29.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.859+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9840005420 con 0x7f985015ecf0 2024-09-21T05:48:29.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.859+0000 7f9845ffb640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9834078bc0 0x7f983407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:29.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.859+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f984009b1f0 con 0x7f985015ecf0 2024-09-21T05:48:29.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.860+0000 7f985577b640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9834078bc0 0x7f983407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:29.862 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.860+0000 7f985577b640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9834078bc0 0x7f983407b0d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f984c000f60 tx=0x7f984c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:29.866 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:29.864+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9840014030 con 0x7f985015ecf0 2024-09-21T05:48:30.055 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.053+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98500aff50 con 0x7f985015ecf0 2024-09-21T05:48:30.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.054+0000 7f9845ffb640 1 -- 172.21.15.22:0/2841847169 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+17029 (secure 0 0 0) 0x7f98500aff50 con 0x7f985015ecf0 2024-09-21T05:48:30.056 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:30.056 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":113,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:48:18.533848+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T05:47:50.363316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T05:47:51.649557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.b","mappings":[{"from":0,"to":5}]},{"pgid":"17.18","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6835/3772214991":"2024-09-22T05:48:18.533817+0000","172.21.15.22:6833/3772214991":"2024-09-22T05:48:18.533817+0000","172.21.15.134:6837/4249862141":"2024-09-22T05:48:09.072398+0000","172.21.15.134:6835/4249862141":"2024-09-22T05:48:09.072398+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:48:30.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9834078bc0 msgr2=0x7f983407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.058 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9834078bc0 0x7f983407b0d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f984c000f60 tx=0x7f984c009040 comp rx=0 tx=0).stop 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 msgr2=0x7f9850152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f9850152430 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f984000c970 tx=0x7f984000ce30 comp rx=0 tx=0).stop 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 shutdown_connections 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f9850152cc0 0x7f985015cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f985015ecf0 0x7f9850152430 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f9834078bc0 0x7f983407b0d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 --2- 172.21.15.22:0/2841847169 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f985009be30 0x7f985015be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 >> 172.21.15.22:0/2841847169 conn(0x7f985009ba20 msgr2=0x7f9850154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 shutdown_connections 2024-09-21T05:48:30.059 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.056+0000 7f98579df640 1 -- 172.21.15.22:0/2841847169 wait complete. 2024-09-21T05:48:30.070 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.184+0000 7f179c620640 1 -- 172.21.15.22:0/2337875262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940b5d30 msgr2=0x7f17940cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.184+0000 7f179c620640 1 --2- 172.21.15.22:0/2337875262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940b5d30 0x7f17940cf3d0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f179408d900 tx=0x7f179002a540 comp rx=0 tx=0).stop 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.185+0000 7f179c620640 1 -- 172.21.15.22:0/2337875262 shutdown_connections 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.185+0000 7f179c620640 1 --2- 172.21.15.22:0/2337875262 >> v2:172.21.15.134:3300/0 conn(0x7f17940b5130 0x7f17940d24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.185+0000 7f179c620640 1 --2- 172.21.15.22:0/2337875262 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940b5d30 0x7f17940cf3d0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.185+0000 7f179c620640 1 -- 172.21.15.22:0/2337875262 >> 172.21.15.22:0/2337875262 conn(0x7f17940d1220 msgr2=0x7f17940d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.185+0000 7f179c620640 1 -- 172.21.15.22:0/2337875262 shutdown_connections 2024-09-21T05:48:30.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.185+0000 7f179c620640 1 -- 172.21.15.22:0/2337875262 wait complete. 2024-09-21T05:48:30.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.186+0000 7f179c620640 1 Processor -- start 2024-09-21T05:48:30.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.186+0000 7f179c620640 1 -- start start 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179c620640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940b5130 0x7f1794150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179c620640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940b5d30 0x7f17940ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179c620640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 0x7f17940cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179c620640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f17940d3030 con 0x7f17940cf910 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179c620640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f17940d2eb0 con 0x7f17940b5d30 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179c620640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f17940d31b0 con 0x7f17940b5130 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 0x7f17940cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 0x7f17940cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55732/0 (socket says 172.21.15.22:55732) 2024-09-21T05:48:30.191 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 -- 172.21.15.22:0/609131974 learned_addr learned my addr 172.21.15.22:0/609131974 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179a3bc640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940b5130 0x7f1794150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f1799bbb640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940b5d30 0x7f17940ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 -- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940b5130 msgr2=0x7f1794150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940b5130 0x7f1794150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 -- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940b5d30 msgr2=0x7f17940ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940b5d30 0x7f17940ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 -- 172.21.15.22:0/609131974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17940ca540 con 0x7f17940cf910 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.188+0000 7f179abbd640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 0x7f17940cd480 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f178c00caa0 tx=0x7f178c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.189+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00adf0 con 0x7f17940cf910 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.189+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f178c00f540 con 0x7f17940cf910 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.189+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17940ca820 con 0x7f17940cf910 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.189+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00f990 con 0x7f17940cf910 2024-09-21T05:48:30.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.189+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17940cca20 con 0x7f17940cf910 2024-09-21T05:48:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.190+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17940cad40 con 0x7f17940cf910 2024-09-21T05:48:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.190+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f178c00fb30 con 0x7f17940cf910 2024-09-21T05:48:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.191+0000 7f178affd640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1778078bc0 0x7f177807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.191+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5939+0+0 (secure 0 0 0) 0x7f178c09b740 con 0x7f17940cf910 2024-09-21T05:48:30.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.191+0000 7f179a3bc640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1778078bc0 0x7f177807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.192+0000 7f179a3bc640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1778078bc0 0x7f177807b0d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1790033000 tx=0x7f1790004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:30.197 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.195+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f17940cad40 con 0x7f17940cf910 2024-09-21T05:48:30.424 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.422+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f17940d3730 con 0x7f17940cf910 2024-09-21T05:48:30.424 INFO:tasks.ceph.mon.a.smithi022.stderr:2024-09-21T05:48:30.422+0000 7fbf5e882640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T05:48:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.424+0000 7f178affd640 1 -- 172.21.15.22:0/609131974 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) ==== 161+0+0 (secure 0 0 0) 0x7f178c00fe20 con 0x7f17940cf910 2024-09-21T05:48:30.426 INFO:teuthology.orchestra.run.smithi022.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T05:48:30.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1778078bc0 msgr2=0x7f177807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1778078bc0 0x7f177807b0d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1790033000 tx=0x7f1790004730 comp rx=0 tx=0).stop 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 msgr2=0x7f17940cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 0x7f17940cd480 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f178c00caa0 tx=0x7f178c00cf60 comp rx=0 tx=0).stop 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 shutdown_connections 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f17940cf910 0x7f17940cd480 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f17940b5d30 0x7f17940ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f1778078bc0 0x7f177807b0d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 --2- 172.21.15.22:0/609131974 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f17940b5130 0x7f1794150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.427+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 >> 172.21.15.22:0/609131974 conn(0x7f17940d1220 msgr2=0x7f17940b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.428+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 shutdown_connections 2024-09-21T05:48:30.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.428+0000 7f179c620640 1 -- 172.21.15.22:0/609131974 wait complete. 2024-09-21T05:48:30.431 INFO:tasks.ceph.mds.a.smithi022.stderr: -1243> 2024-09-21T05:48:24.542+0000 7f4bf804f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T05:48:30.442 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T05:48:30.487 INFO:tasks.ceph.mds.a.smithi022.stdout:starting mds.a at 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15d09ed640 1 -- 172.21.15.22:0/3332913192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15b8004d40 msgr2=0x7f15b8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15d09ed640 1 --2- 172.21.15.22:0/3332913192 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15b8004d40 0x7f15b8005150 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009b20 tx=0x7f15b8005830 comp rx=0 tx=0).stop 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15cad9d640 1 -- 172.21.15.22:0/3332913192 reap_dead start 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15d09ed640 1 -- 172.21.15.22:0/3332913192 shutdown_connections 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15d09ed640 1 -- 172.21.15.22:0/3332913192 >> 172.21.15.22:0/3332913192 conn(0x7f15cc0d1220 msgr2=0x7f15cc0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15d09ed640 1 -- 172.21.15.22:0/3332913192 shutdown_connections 2024-09-21T05:48:30.564 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.562+0000 7f15d09ed640 1 -- 172.21.15.22:0/3332913192 wait complete. 2024-09-21T05:48:30.565 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.564+0000 7f15d09ed640 1 Processor -- start 2024-09-21T05:48:30.566 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.564+0000 7f15d09ed640 1 -- start start 2024-09-21T05:48:30.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15d09ed640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b8004d40 0x7f15cc160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15d09ed640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 0x7f15cc1f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15d09ed640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15cc15a3e0 0x7f15cc1f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15d09ed640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc0d2a30 con 0x7f15cc159fd0 2024-09-21T05:48:30.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15d09ed640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc0cf770 con 0x7f15b8004d40 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15d09ed640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f15cc0d2bb0 con 0x7f15cc15a3e0 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15c9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 0x7f15cc1f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15c9d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 0x7f15cc1f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55764/0 (socket says 172.21.15.22:55764) 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15c9d9b640 1 -- 172.21.15.22:0/2576017204 learned_addr learned my addr 172.21.15.22:0/2576017204 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15ca59c640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b8004d40 0x7f15cc160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.565+0000 7f15cad9d640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15cc15a3e0 0x7f15cc1f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c9d9b640 1 -- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15cc15a3e0 msgr2=0x7f15cc1f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c9d9b640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15cc15a3e0 0x7f15cc1f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c9d9b640 1 -- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b8004d40 msgr2=0x7f15cc160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c9d9b640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b8004d40 0x7f15cc160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c9d9b640 1 -- 172.21.15.22:0/2576017204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc1f98e0 con 0x7f15cc159fd0 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15ca59c640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b8004d40 0x7f15cc160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c9d9b640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 0x7f15cc1f8d00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f15b4006600 tx=0x7f15b40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:30.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4015070 con 0x7f15cc159fd0 2024-09-21T05:48:30.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15b400f960 con 0x7f15cc159fd0 2024-09-21T05:48:30.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc1f9bc0 con 0x7f15cc159fd0 2024-09-21T05:48:30.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b400fdb0 con 0x7f15cc159fd0 2024-09-21T05:48:30.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.566+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1597d0 con 0x7f15cc159fd0 2024-09-21T05:48:30.570 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.567+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15cc1f9650 con 0x7f15cc159fd0 2024-09-21T05:48:30.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.568+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f15b4011430 con 0x7f15cc159fd0 2024-09-21T05:48:30.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.568+0000 7f15c2ffd640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15ac078b90 0x7f15ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:30.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.568+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6025+0+0 (secure 0 0 0) 0x7f15b409bc00 con 0x7f15cc159fd0 2024-09-21T05:48:30.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.569+0000 7f15ca59c640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15ac078b90 0x7f15ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:30.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.569+0000 7f15ca59c640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15ac078b90 0x7f15ac07b0a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005fd0 tx=0x7f15bc009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:30.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.573+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15b40638a0 con 0x7f15cc159fd0 2024-09-21T05:48:30.800 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:30.798+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f15cc0d24f0 con 0x7f15cc159fd0 2024-09-21T05:48:31.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.425+0000 7f15c2ffd640 1 -- 172.21.15.22:0/2576017204 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v116) ==== 104+0+0 (secure 0 0 0) 0x7f15b40682d0 con 0x7f15cc159fd0 2024-09-21T05:48:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15ac078b90 msgr2=0x7f15ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15ac078b90 0x7f15ac07b0a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005fd0 tx=0x7f15bc009500 comp rx=0 tx=0).stop 2024-09-21T05:48:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 msgr2=0x7f15cc1f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 0x7f15cc1f8d00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f15b4006600 tx=0x7f15b40039c0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 shutdown_connections 2024-09-21T05:48:31.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f15ac078b90 0x7f15ac07b0a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f15cc15a3e0 0x7f15cc1f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f15cc159fd0 0x7f15cc1f8d00 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 --2- 172.21.15.22:0/2576017204 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f15b8004d40 0x7f15cc160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 >> 172.21.15.22:0/2576017204 conn(0x7f15cc0d1220 msgr2=0x7f15cc0b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 shutdown_connections 2024-09-21T05:48:31.432 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.429+0000 7f15d09ed640 1 -- 172.21.15.22:0/2576017204 wait complete. 2024-09-21T05:48:31.446 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8abd1e6640 1 -- 172.21.15.22:0/4290957741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 msgr2=0x7f8aa0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/4290957741 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8aa0004c10 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009b50 tx=0x7f8aa0005830 comp rx=0 tx=0).stop 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8ab759e640 1 -- 172.21.15.22:0/4290957741 reap_dead start 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8abd1e6640 1 -- 172.21.15.22:0/4290957741 shutdown_connections 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8abd1e6640 1 -- 172.21.15.22:0/4290957741 >> 172.21.15.22:0/4290957741 conn(0x7f8ab80d1220 msgr2=0x7f8ab80d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8abd1e6640 1 -- 172.21.15.22:0/4290957741 shutdown_connections 2024-09-21T05:48:31.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.572+0000 7f8abd1e6640 1 -- 172.21.15.22:0/4290957741 wait complete. 2024-09-21T05:48:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.574+0000 7f8abd1e6640 1 Processor -- start 2024-09-21T05:48:31.576 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.574+0000 7f8abd1e6640 1 -- start start 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8abd1e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8ab81f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8abd1e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ab8158b80 0x7f8ab80c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8abd1e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab80c9eb0 0x7f8ab80cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8abd1e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab80d2d80 con 0x7f8aa0004800 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8abd1e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab80d2c00 con 0x7f8ab80c9eb0 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8abd1e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab80d2f00 con 0x7f8ab8158b80 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8ab81f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8ab81f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55778/0 (socket says 172.21.15.22:55778) 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 -- 172.21.15.22:0/2987484011 learned_addr learned my addr 172.21.15.22:0/2987484011 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab659c640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ab8158b80 0x7f8ab80c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 -- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ab8158b80 msgr2=0x7f8ab80c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:31.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ab8158b80 0x7f8ab80c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab759e640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab80c9eb0 0x7f8ab80cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 -- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab80c9eb0 msgr2=0x7f8ab80cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab80c9eb0 0x7f8ab80cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 -- 172.21.15.22:0/2987484011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab80ca2c0 con 0x7f8aa0004800 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.576+0000 7f8ab6d9d640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8ab81f9450 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8003f20 tx=0x7f8aa80017d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.577+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8015070 con 0x7f8aa0004800 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.577+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aa8011900 con 0x7f8aa0004800 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.577+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab81f5950 con 0x7f8aa0004800 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.577+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8011d50 con 0x7f8aa0004800 2024-09-21T05:48:31.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.577+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab81f5f10 con 0x7f8aa0004800 2024-09-21T05:48:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.578+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ab80d3460 con 0x7f8aa0004800 2024-09-21T05:48:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.579+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8aa8018070 con 0x7f8aa0004800 2024-09-21T05:48:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.579+0000 7f8a9f7fe640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8aa4078b90 0x7f8aa407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.579+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 6025+0+0 (secure 0 0 0) 0x7f8aa809bb40 con 0x7f8aa0004800 2024-09-21T05:48:31.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.579+0000 7f8ab659c640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8aa4078b90 0x7f8aa407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:31.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.580+0000 7f8ab659c640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8aa4078b90 0x7f8aa407b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8aac035000 tx=0x7f8aac02d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:31.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.584+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ab80ca2c0 con 0x7f8aa0004800 2024-09-21T05:48:31.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:31.772+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8ab80b6fd0 con 0x7f8aa0004800 2024-09-21T05:48:32.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.426+0000 7f8a9f7fe640 1 -- 172.21.15.22:0/2987484011 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) ==== 181+0+0 (secure 0 0 0) 0x7f8aa8063e50 con 0x7f8aa0004800 2024-09-21T05:48:32.429 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_metadata' removed 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8aa4078b90 msgr2=0x7f8aa407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8aa4078b90 0x7f8aa407b0a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8aac035000 tx=0x7f8aac02d040 comp rx=0 tx=0).stop 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 msgr2=0x7f8ab81f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8ab81f9450 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8003f20 tx=0x7f8aa80017d0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 shutdown_connections 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8ab80c9eb0 0x7f8ab80cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8aa4078b90 0x7f8aa407b0a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8ab8158b80 0x7f8ab80c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 --2- 172.21.15.22:0/2987484011 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8aa0004800 0x7f8ab81f9450 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 >> 172.21.15.22:0/2987484011 conn(0x7f8ab80d1220 msgr2=0x7f8ab8161a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 shutdown_connections 2024-09-21T05:48:32.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.431+0000 7f8abd1e6640 1 -- 172.21.15.22:0/2987484011 wait complete. 2024-09-21T05:48:32.447 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T05:48:32.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.571+0000 7f53e23e4640 1 -- 172.21.15.22:0/2486624140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5130 msgr2=0x7f53dc0cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:32.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.571+0000 7f53e23e4640 1 --2- 172.21.15.22:0/2486624140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5130 0x7f53dc0cfae0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009830 tx=0x7f53c402a510 comp rx=0 tx=0).stop 2024-09-21T05:48:32.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.572+0000 7f53e23e4640 1 -- 172.21.15.22:0/2486624140 shutdown_connections 2024-09-21T05:48:32.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.572+0000 7f53e23e4640 1 --2- 172.21.15.22:0/2486624140 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53dc0d0020 0x7f53dc0d24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.572+0000 7f53e23e4640 1 --2- 172.21.15.22:0/2486624140 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5130 0x7f53dc0cfae0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.572+0000 7f53e23e4640 1 -- 172.21.15.22:0/2486624140 >> 172.21.15.22:0/2486624140 conn(0x7f53dc0d1220 msgr2=0x7f53dc0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:32.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.572+0000 7f53e23e4640 1 -- 172.21.15.22:0/2486624140 shutdown_connections 2024-09-21T05:48:32.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.572+0000 7f53e23e4640 1 -- 172.21.15.22:0/2486624140 wait complete. 2024-09-21T05:48:32.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.573+0000 7f53e23e4640 1 Processor -- start 2024-09-21T05:48:32.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.573+0000 7f53e23e4640 1 -- start start 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e23e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53dc0b5130 0x7f53dc14f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e23e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5d30 0x7f53dc14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e23e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 0x7f53dc14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e23e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc0d30f0 con 0x7f53dc0d0020 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e23e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53dc0d2f70 con 0x7f53dc0b5d30 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e23e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53dc0d3270 con 0x7f53dc0b5130 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 0x7f53dc14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 0x7f53dc14dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55780/0 (socket says 172.21.15.22:55780) 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 -- 172.21.15.22:0/463494503 learned_addr learned my addr 172.21.15.22:0/463494503 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53d3fff640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5d30 0x7f53dc14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:32.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53dbfff640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53dc0b5130 0x7f53dc14f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 -- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53dc0b5130 msgr2=0x7f53dc14f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53dc0b5130 0x7f53dc14f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 -- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5d30 msgr2=0x7f53dc14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5d30 0x7f53dc14fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 -- 172.21.15.22:0/463494503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc1eae10 con 0x7f53dc0d0020 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.575+0000 7f53e0981640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 0x7f53dc14dd80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f53d400ebf0 tx=0x7f53d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.576+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4003b70 con 0x7f53dc0d0020 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.576+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53d4003d10 con 0x7f53dc0d0020 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.576+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc0ca500 con 0x7f53dc0d0020 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.576+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53d4004280 con 0x7f53dc0d0020 2024-09-21T05:48:32.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.576+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53dc0caac0 con 0x7f53dc0d0020 2024-09-21T05:48:32.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.577+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53dc0d3720 con 0x7f53dc0d0020 2024-09-21T05:48:32.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.577+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f53d4004420 con 0x7f53dc0d0020 2024-09-21T05:48:32.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.578+0000 7f53d97fa640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53bc078bc0 0x7f53bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:32.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.578+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5587+0+0 (secure 0 0 0) 0x7f53d409b820 con 0x7f53dc0d0020 2024-09-21T05:48:32.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.578+0000 7f53dbfff640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53bc078bc0 0x7f53bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:32.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.579+0000 7f53dbfff640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53bc078bc0 0x7f53bc07b0d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f53cc007fd0 tx=0x7f53cc001370 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:32.584 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.582+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53d4063690 con 0x7f53dc0d0020 2024-09-21T05:48:32.783 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:32.781+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f53dc1eb390 con 0x7f53dc0d0020 2024-09-21T05:48:33.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.428+0000 7f53d97fa640 1 -- 172.21.15.22:0/463494503 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) ==== 169+0+0 (secure 0 0 0) 0x7f53d40680c0 con 0x7f53dc0d0020 2024-09-21T05:48:33.430 INFO:teuthology.orchestra.run.smithi022.stderr:pool 'cephfs_data' removed 2024-09-21T05:48:33.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.431+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53bc078bc0 msgr2=0x7f53bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.431+0000 7f53e23e4640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53bc078bc0 0x7f53bc07b0d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f53cc007fd0 tx=0x7f53cc001370 comp rx=0 tx=0).stop 2024-09-21T05:48:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.431+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 msgr2=0x7f53dc14dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.434 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.431+0000 7f53e23e4640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 0x7f53dc14dd80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f53d400ebf0 tx=0x7f53d400c6a0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 shutdown_connections 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53dc0d0020 0x7f53dc14dd80 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53dc0b5d30 0x7f53dc14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f53bc078bc0 0x7f53bc07b0d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 --2- 172.21.15.22:0/463494503 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53dc0b5130 0x7f53dc14f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 >> 172.21.15.22:0/463494503 conn(0x7f53dc0d1220 msgr2=0x7f53dc09c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.433+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 shutdown_connections 2024-09-21T05:48:33.436 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.434+0000 7f53e23e4640 1 -- 172.21.15.22:0/463494503 wait complete. 2024-09-21T05:48:33.454 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 -- 172.21.15.22:0/2066925079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0b5d30 msgr2=0x7f0b5c0cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 --2- 172.21.15.22:0/2066925079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0b5d30 0x7f0b5c0cf3d0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c08d900 tx=0x7f0b4c02a540 comp rx=0 tx=0).stop 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 -- 172.21.15.22:0/2066925079 shutdown_connections 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 --2- 172.21.15.22:0/2066925079 >> v2:172.21.15.134:3300/0 conn(0x7f0b5c0b5130 0x7f0b5c0d24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 --2- 172.21.15.22:0/2066925079 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0b5d30 0x7f0b5c0cf3d0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 -- 172.21.15.22:0/2066925079 >> 172.21.15.22:0/2066925079 conn(0x7f0b5c0d1220 msgr2=0x7f0b5c0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:33.574 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 -- 172.21.15.22:0/2066925079 shutdown_connections 2024-09-21T05:48:33.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.572+0000 7f0b62284640 1 -- 172.21.15.22:0/2066925079 wait complete. 2024-09-21T05:48:33.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.573+0000 7f0b62284640 1 Processor -- start 2024-09-21T05:48:33.575 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.574+0000 7f0b62284640 1 -- start start 2024-09-21T05:48:33.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b62284640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c0b5130 0x7f0b5c150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b62284640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b5c0b5d30 0x7f0b5c0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b62284640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 0x7f0b5c0cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b62284640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c0d3030 con 0x7f0b5c0cf910 2024-09-21T05:48:33.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b62284640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c0d2eb0 con 0x7f0b5c0b5130 2024-09-21T05:48:33.578 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b62284640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5c0d31b0 con 0x7f0b5c0b5d30 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 0x7f0b5c0cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 0x7f0b5c0cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55792/0 (socket says 172.21.15.22:55792) 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 -- 172.21.15.22:0/3604460920 learned_addr learned my addr 172.21.15.22:0/3604460920 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 -- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b5c0b5d30 msgr2=0x7f0b5c0ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b5c0b5d30 0x7f0b5c0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 -- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c0b5130 msgr2=0x7f0b5c150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c0b5130 0x7f0b5c150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.575+0000 7f0b60821640 1 -- 172.21.15.22:0/3604460920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c0ca540 con 0x7f0b5c0cf910 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.576+0000 7f0b60821640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 0x7f0b5c0cd480 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400caa0 tx=0x7f0b5400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.576+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400adf0 con 0x7f0b5c0cf910 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.576+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b5400f540 con 0x7f0b5c0cf910 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.576+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c0ca820 con 0x7f0b5c0cf910 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.576+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400f990 con 0x7f0b5c0cf910 2024-09-21T05:48:33.579 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.576+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c0cca20 con 0x7f0b5c0cf910 2024-09-21T05:48:33.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.577+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c0cad40 con 0x7f0b5c0cf910 2024-09-21T05:48:33.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.578+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b5400fb30 con 0x7f0b5c0cf910 2024-09-21T05:48:33.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.578+0000 7f0b58ff9640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0b48078bc0 0x7f0b4807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.580 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.578+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f0b5409b530 con 0x7f0b5c0cf910 2024-09-21T05:48:33.581 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.579+0000 7f0b5bfff640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0b48078bc0 0x7f0b4807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:33.582 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.580+0000 7f0b5bfff640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0b48078bc0 0x7f0b4807b0d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c033000 tx=0x7f0b4c0046c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:33.585 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.583+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b5c0cad40 con 0x7f0b5c0cf910 2024-09-21T05:48:33.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.770+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0b5c0d3730 con 0x7f0b5c0cf910 2024-09-21T05:48:33.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.774+0000 7f0b58ff9640 1 -- 172.21.15.22:0/3604460920 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7f0b54014030 con 0x7f0b5c0cf910 2024-09-21T05:48:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0b48078bc0 msgr2=0x7f0b4807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0b48078bc0 0x7f0b4807b0d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c033000 tx=0x7f0b4c0046c0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 msgr2=0x7f0b5c0cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 0x7f0b5c0cd480 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0b5400caa0 tx=0x7f0b5400cf60 comp rx=0 tx=0).stop 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 shutdown_connections 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0b5c0cf910 0x7f0b5c0cd480 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0b48078bc0 0x7f0b4807b0d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0b5c0b5d30 0x7f0b5c0ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 --2- 172.21.15.22:0/3604460920 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0b5c0b5130 0x7f0b5c150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 >> 172.21.15.22:0/3604460920 conn(0x7f0b5c0d1220 msgr2=0x7f0b5c0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 shutdown_connections 2024-09-21T05:48:33.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.776+0000 7f0b62284640 1 -- 172.21.15.22:0/3604460920 wait complete. 2024-09-21T05:48:33.789 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-09-21T05:48:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.903+0000 7f4af8ed5640 1 -- 172.21.15.22:0/668524136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b6af0 msgr2=0x7f4af40cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.903+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/668524136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b6af0 0x7f4af40cd800 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005fd0 tx=0x7f4aec02a8b0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.903+0000 7f4af8ed5640 1 -- 172.21.15.22:0/668524136 shutdown_connections 2024-09-21T05:48:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.903+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/668524136 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b6af0 0x7f4af40cd800 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.906 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.903+0000 7f4af8ed5640 1 -- 172.21.15.22:0/668524136 >> 172.21.15.22:0/668524136 conn(0x7f4af40d2580 msgr2=0x7f4af40d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:33.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.903+0000 7f4af8ed5640 1 -- 172.21.15.22:0/668524136 shutdown_connections 2024-09-21T05:48:33.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.904+0000 7f4af8ed5640 1 -- 172.21.15.22:0/668524136 wait complete. 2024-09-21T05:48:33.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.905+0000 7f4af8ed5640 1 Processor -- start 2024-09-21T05:48:33.907 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.906+0000 7f4af8ed5640 1 -- start start 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af8ed5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af40b6af0 0x7f4af415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af8ed5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 0x7f4af41f5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af8ed5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4af40b7ab0 0x7f4af41f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af8ed5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40ce320 con 0x7f4af40b7610 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af8ed5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40ce1a0 con 0x7f4af40b6af0 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af8ed5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f4af4168260 con 0x7f4af40b7ab0 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 0x7f4af41f5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 0x7f4af41f5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:55808/0 (socket says 172.21.15.22:55808) 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 -- 172.21.15.22:0/3481792518 learned_addr learned my addr 172.21.15.22:0/3481792518 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:33.910 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af259c640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af40b6af0 0x7f4af415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af2d9d640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4af40b7ab0 0x7f4af41f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 -- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4af40b7ab0 msgr2=0x7f4af41f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4af40b7ab0 0x7f4af41f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 -- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af40b6af0 msgr2=0x7f4af415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af40b6af0 0x7f4af415c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 -- 172.21.15.22:0/3481792518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af415f270 con 0x7f4af40b7610 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.907+0000 7f4af1d9b640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 0x7f4af41f5550 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c3c0 tx=0x7f4ae800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.908+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8017070 con 0x7f4af40b7610 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.908+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ae80035f0 con 0x7f4af40b7610 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.908+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af41f62f0 con 0x7f4af40b7610 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.908+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8003a40 con 0x7f4af40b7610 2024-09-21T05:48:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.908+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af415ef00 con 0x7f4af40b7610 2024-09-21T05:48:33.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.909+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af40cd0f0 con 0x7f4af40b7610 2024-09-21T05:48:33.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.909+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4ae801a070 con 0x7f4af40b7610 2024-09-21T05:48:33.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.910+0000 7f4ae2ffd640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4ad4078bc0 0x7f4ad407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:33.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.910+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f4ae8099de0 con 0x7f4af40b7610 2024-09-21T05:48:33.912 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.910+0000 7f4af259c640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4ad4078bc0 0x7f4ad407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:33.913 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.911+0000 7f4af259c640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4ad4078bc0 0x7f4ad407b0d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4000f60 tx=0x7f4ae4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:33.917 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:33.915+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4af415f270 con 0x7f4af40b7610 2024-09-21T05:48:34.176 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.174+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f4af4160ec0 con 0x7f4af40b7610 2024-09-21T05:48:34.438 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.436+0000 7f4ae2ffd640 1 -- 172.21.15.22:0/3481792518 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v433) ==== 144+0+0 (secure 0 0 0) 0x7f4ae80623e0 con 0x7f4af40b7610 2024-09-21T05:48:34.441 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4ad4078bc0 msgr2=0x7f4ad407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4ad4078bc0 0x7f4ad407b0d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4000f60 tx=0x7f4ae4009040 comp rx=0 tx=0).stop 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 msgr2=0x7f4af41f5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 0x7f4af41f5550 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c3c0 tx=0x7f4ae800c880 comp rx=0 tx=0).stop 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 shutdown_connections 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f4ad4078bc0 0x7f4ad407b0d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f4af40b7ab0 0x7f4af41f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f4af40b7610 0x7f4af41f5550 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 --2- 172.21.15.22:0/3481792518 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f4af40b6af0 0x7f4af415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 >> 172.21.15.22:0/3481792518 conn(0x7f4af40d2580 msgr2=0x7f4af4070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 shutdown_connections 2024-09-21T05:48:34.442 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:34.439+0000 7f4af8ed5640 1 -- 172.21.15.22:0/3481792518 wait complete. 2024-09-21T05:48:34.453 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-21T05:48:34.453 INFO:tasks.cephfs_test_runner: 2024-09-21T05:48:34.454 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T05:48:34.454 INFO:tasks.cephfs_test_runner:Ran 7 tests in 528.349s 2024-09-21T05:48:34.454 INFO:tasks.cephfs_test_runner: 2024-09-21T05:48:34.454 INFO:tasks.cephfs_test_runner:OK 2024-09-21T05:48:34.454 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T05:48:34.466 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T05:48:34.480 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T05:48:34.480 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:34.481 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:34.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:48:34.498 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-09-21T05:48:34.498 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:34.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056795,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19638 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T05:48:34.554 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:48:34.555 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:34.574 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:48:34.574 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-21T05:48:34.574 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T05:48:34.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056030,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=136 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248732k,nr_inodes=812183,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497468k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T05:48:34.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.637 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:48:34.654 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T05:48:34.654 INFO:teuthology.orchestra.run.smithi022.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T05:48:34.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T05:48:34.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.720 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T05:48:34.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.787 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-09-21T05:48:34.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.876 DEBUG:teuthology.orchestra.run.smithi022:> ip netns list 2024-09-21T05:48:34.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.897 DEBUG:teuthology.orchestra.run.smithi022:> sudo ip link delete ceph-brx 2024-09-21T05:48:34.966 INFO:teuthology.orchestra.run.smithi022.stderr:Cannot find device "ceph-brx" 2024-09-21T05:48:34.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:48:34.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.968 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-09-21T05:48:34.987 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T05:48:34.987 INFO:teuthology.orchestra.run.smithi134.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T05:48:34.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:34.988 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T05:48:35.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:35.057 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T05:48:35.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:35.126 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-09-21T05:48:35.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:35.224 DEBUG:teuthology.orchestra.run.smithi134:> ip netns list 2024-09-21T05:48:35.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T05:48:35.241 DEBUG:teuthology.orchestra.run.smithi134:> sudo ip link delete ceph-brx 2024-09-21T05:48:35.311 INFO:teuthology.orchestra.run.smithi134.stderr:Cannot find device "ceph-brx" 2024-09-21T05:48:35.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:48:35.312 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T05:48:35.322 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T05:48:35.322 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:48:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 -- 172.21.15.22:0/2911478403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 msgr2=0x7f61340afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 --2- 172.21.15.22:0/2911478403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340afa00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6134070230 tx=0x7f612402a5f0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 -- 172.21.15.22:0/2911478403 shutdown_connections 2024-09-21T05:48:35.443 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 --2- 172.21.15.22:0/2911478403 >> v2:172.21.15.134:3300/0 conn(0x7f613415f1a0 0x7f613415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 --2- 172.21.15.22:0/2911478403 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340afa00 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 -- 172.21.15.22:0/2911478403 >> 172.21.15.22:0/2911478403 conn(0x7f61340b2320 msgr2=0x7f61340b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 -- 172.21.15.22:0/2911478403 shutdown_connections 2024-09-21T05:48:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.440+0000 7f613952b640 1 -- 172.21.15.22:0/2911478403 wait complete. 2024-09-21T05:48:35.444 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.442+0000 7f613952b640 1 Processor -- start 2024-09-21T05:48:35.445 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.443+0000 7f613952b640 1 -- start start 2024-09-21T05:48:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f613952b640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6134073760 0x7f61341f7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f613952b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f613952b640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61340c5ad0 0x7f61340ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.448 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f613952b640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f613415bae0 con 0x7f613415f710 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f613952b640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f613415b960 con 0x7f61340c5ad0 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f613952b640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f613415bc60 con 0x7f6134073760 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f6132ffd640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6134073760 0x7f61341f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53198/0 (socket says 172.21.15.22:53198) 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 -- 172.21.15.22:0/3772243277 learned_addr learned my addr 172.21.15.22:0/3772243277 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61337fe640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61340c5ad0 0x7f61340ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 -- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6134073760 msgr2=0x7f61341f7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6134073760 0x7f61341f7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 -- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61340c5ad0 msgr2=0x7f61340ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61340c5ad0 0x7f61340ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.444+0000 7f61327fc640 1 -- 172.21.15.22:0/3772243277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613414ab90 con 0x7f613415f710 2024-09-21T05:48:35.449 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.445+0000 7f61327fc640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340c5590 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f612800c3a0 tx=0x7f612800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.445+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128017070 con 0x7f613415f710 2024-09-21T05:48:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.445+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61280053c0 con 0x7f613415f710 2024-09-21T05:48:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.445+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613414ae40 con 0x7f613415f710 2024-09-21T05:48:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.445+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128003850 con 0x7f613415f710 2024-09-21T05:48:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.445+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613414b400 con 0x7f613415f710 2024-09-21T05:48:35.450 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.446+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613415c110 con 0x7f613415f710 2024-09-21T05:48:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.446+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f612801a070 con 0x7f613415f710 2024-09-21T05:48:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.447+0000 7f611b7fe640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6120078b90 0x7f612007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.447+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f6128099de0 con 0x7f613415f710 2024-09-21T05:48:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.447+0000 7f6132ffd640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6120078b90 0x7f612007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.451 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.449+0000 7f6132ffd640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6120078b90 0x7f612007b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6124033000 tx=0x7f6124004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:35.454 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.452+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61340aff40 con 0x7f613415f710 2024-09-21T05:48:35.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.635+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f613406a200 con 0x7f6120078b90 2024-09-21T05:48:35.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.636+0000 7f611b7fe640 1 -- 172.21.15.22:0/3772243277 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13446 (secure 0 0 0) 0x7f6124005630 con 0x7f6120078b90 2024-09-21T05:48:35.639 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:35.639 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6120078b90 msgr2=0x7f612007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6120078b90 0x7f612007b0a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6124033000 tx=0x7f6124004730 comp rx=0 tx=0).stop 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 msgr2=0x7f61340c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340c5590 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f612800c3a0 tx=0x7f612800c860 comp rx=0 tx=0).stop 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 shutdown_connections 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f61340c5ad0 0x7f61340ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f613415f710 0x7f61340c5590 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6120078b90 0x7f612007b0a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 --2- 172.21.15.22:0/3772243277 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6134073760 0x7f61341f7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.641 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.638+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 >> 172.21.15.22:0/3772243277 conn(0x7f61340b2320 msgr2=0x7f613415dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.639+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 shutdown_connections 2024-09-21T05:48:35.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.639+0000 7f613952b640 1 -- 172.21.15.22:0/3772243277 wait complete. 2024-09-21T05:48:35.652 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-21T05:48:33.888289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":298852,"kb_used_data":28568,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754675868,"statfs":{"total":773094113280,"available":772788088832,"internally_reserved":0,"allocated":29253632,"data_stored":22841054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390143,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-256,"num_write":-165,"num_write_kb":-8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001807"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T05:48:31.436412+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:48:31.436412+0000","last_peered":"2024-09-21T05:48:31.436412+0000","last_clean":"2024-09-21T05:48:31.436412+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:48:31.436412+0000","last_undegraded":"2024-09-21T05:48:31.436412+0000","last_fullsized":"2024-09-21T05:48:31.436412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1998848,"data_stored":1199311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":614638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36652,"kb_used_data":5468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335188,"statfs":{"total":96636764160,"available":96599232512,"internally_reserved":0,"allocated":5599232,"data_stored":4802708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542259,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1413120,"data_stored":608943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542258,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40824,"kb_used_data":5480,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331016,"statfs":{"total":96636764160,"available":96594960384,"internally_reserved":0,"allocated":5611520,"data_stored":4803823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36716,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335124,"statfs":{"total":96636764160,"available":96599166976,"internally_reserved":0,"allocated":1400832,"data_stored":608332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542259,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44972,"kb_used_data":5468,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326868,"statfs":{"total":96636764160,"available":96590712832,"internally_reserved":0,"allocated":5599232,"data_stored":4802755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542259,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41424,"kb_used_data":6076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330416,"statfs":{"total":96636764160,"available":96594345984,"internally_reserved":0,"allocated":6221824,"data_stored":5400544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:48:35.652 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:48:35.769 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 -- 172.21.15.22:0/2590879879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b4ed0 msgr2=0x7fa6e4142f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/2590879879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b4ed0 0x7fa6e4142f40 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc009860 tx=0x7fa6dc02a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 -- 172.21.15.22:0/2590879879 shutdown_connections 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/2590879879 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6e40b56e0 0x7fa6e40b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/2590879879 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b4ed0 0x7fa6e4142f40 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 -- 172.21.15.22:0/2590879879 >> 172.21.15.22:0/2590879879 conn(0x7fa6e409ba20 msgr2=0x7fa6e40b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 -- 172.21.15.22:0/2590879879 shutdown_connections 2024-09-21T05:48:35.770 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.767+0000 7fa6eb84d640 1 -- 172.21.15.22:0/2590879879 wait complete. 2024-09-21T05:48:35.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.769+0000 7fa6eb84d640 1 Processor -- start 2024-09-21T05:48:35.771 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.769+0000 7fa6eb84d640 1 -- start start 2024-09-21T05:48:35.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.770+0000 7fa6eb84d640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6e40b4ed0 0x7fa6e413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.770+0000 7fa6eb84d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 0x7fa6e413f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.775 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.770+0000 7fa6eb84d640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6e40b6030 0x7fa6e41437e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.770+0000 7fa6eb84d640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40b1cd0 con 0x7fa6e40b56e0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.770+0000 7fa6eb84d640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e40b1b50 con 0x7fa6e40b4ed0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.770+0000 7fa6eb84d640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6e4071f80 con 0x7fa6e40b6030 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 0x7fa6e413f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 0x7fa6e413f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53216/0 (socket says 172.21.15.22:53216) 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 -- 172.21.15.22:0/492231303 learned_addr learned my addr 172.21.15.22:0/492231303 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e95e9640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6e40b4ed0 0x7fa6e413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e9dea640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6e40b6030 0x7fa6e41437e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 -- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6e40b6030 msgr2=0x7fa6e41437e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6e40b6030 0x7fa6e41437e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 -- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6e40b4ed0 msgr2=0x7fa6e413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6e40b4ed0 0x7fa6e413ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 -- 172.21.15.22:0/492231303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e409ee90 con 0x7fa6e40b56e0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e8de8640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 0x7fa6e413f3a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c970 tx=0x7fa6d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:35.776 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8007bb0 con 0x7fa6e40b56e0 2024-09-21T05:48:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6d8007e40 con 0x7fa6e40b56e0 2024-09-21T05:48:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800f690 con 0x7fa6e40b56e0 2024-09-21T05:48:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.771+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e409f300 con 0x7fa6e40b56e0 2024-09-21T05:48:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.772+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e409f8c0 con 0x7fa6e40b56e0 2024-09-21T05:48:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.772+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e4072400 con 0x7fa6e40b56e0 2024-09-21T05:48:35.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.773+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6d800f830 con 0x7fa6e40b56e0 2024-09-21T05:48:35.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.774+0000 7fa6e1ffb640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6d0078b90 0x7fa6d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:35.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.774+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa6d809b1e0 con 0x7fa6e40b56e0 2024-09-21T05:48:35.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.774+0000 7fa6e95e9640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6d0078b90 0x7fa6d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:35.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.775+0000 7fa6e95e9640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6d0078b90 0x7fa6d007b0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc034000 tx=0x7fa6dc004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:35.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.778+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6d8014030 con 0x7fa6e40b56e0 2024-09-21T05:48:35.964 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.962+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6e409f060 con 0x7fa6d0078b90 2024-09-21T05:48:35.965 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.963+0000 7fa6e1ffb640 1 -- 172.21.15.22:0/492231303 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7fa6e409f060 con 0x7fa6d0078b90 2024-09-21T05:48:35.965 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:35.966 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6d0078b90 msgr2=0x7fa6d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6d0078b90 0x7fa6d007b0a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc034000 tx=0x7fa6dc004730 comp rx=0 tx=0).stop 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 msgr2=0x7fa6e413f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 0x7fa6e413f3a0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800c970 tx=0x7fa6d800ce30 comp rx=0 tx=0).stop 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 shutdown_connections 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa6d0078b90 0x7fa6d007b0a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6e40b6030 0x7fa6e41437e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6e40b56e0 0x7fa6e413f3a0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 --2- 172.21.15.22:0/492231303 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6e40b4ed0 0x7fa6e413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:35.968 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 >> 172.21.15.22:0/492231303 conn(0x7fa6e409ba20 msgr2=0x7fa6e40b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:35.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 shutdown_connections 2024-09-21T05:48:35.969 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:35.965+0000 7fa6eb84d640 1 -- 172.21.15.22:0/492231303 wait complete. 2024-09-21T05:48:35.978 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-21T05:48:35.888870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":286740,"kb_used_data":16428,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687980,"statfs":{"total":773094113280,"available":772800491520,"internally_reserved":0,"allocated":16822272,"data_stored":10346943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390143,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-256,"num_write":-165,"num_write_kb":-8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001849"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T05:48:31.436412+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:48:31.436412+0000","last_peered":"2024-09-21T05:48:31.436412+0000","last_clean":"2024-09-21T05:48:31.436412+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:48:31.436412+0000","last_undegraded":"2024-09-21T05:48:31.436412+0000","last_fullsized":"2024-09-21T05:48:31.436412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1998848,"data_stored":1199311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":614638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":620883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36756,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45000,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326840,"statfs":{"total":96636764160,"available":96590684160,"internally_reserved":0,"allocated":5619712,"data_stored":4815175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37336,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334504,"statfs":{"total":96636764160,"available":96598532096,"internally_reserved":0,"allocated":2035712,"data_stored":1217010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:48:35.979 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T05:48:35.979 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 -- 172.21.15.22:0/2799189255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440cf910 msgr2=0x7fa6440cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 --2- 172.21.15.22:0/2799189255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440cf910 0x7fa6440cfd20 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa638009830 tx=0x7fa63802a510 comp rx=0 tx=0).stop 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 -- 172.21.15.22:0/2799189255 shutdown_connections 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 --2- 172.21.15.22:0/2799189255 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440cf910 0x7fa6440cfd20 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 --2- 172.21.15.22:0/2799189255 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6440b5d30 0x7fa6440cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 -- 172.21.15.22:0/2799189255 >> 172.21.15.22:0/2799189255 conn(0x7fa6440d1220 msgr2=0x7fa6440d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 -- 172.21.15.22:0/2799189255 shutdown_connections 2024-09-21T05:48:36.096 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.093+0000 7fa64b794640 1 -- 172.21.15.22:0/2799189255 wait complete. 2024-09-21T05:48:36.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.095+0000 7fa64b794640 1 Processor -- start 2024-09-21T05:48:36.097 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.095+0000 7fa64b794640 1 -- start start 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.096+0000 7fa64b794640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6440b4120 0x7fa64415ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.096+0000 7fa64b794640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 0x7fa6440cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.096+0000 7fa64b794640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6440cf910 0x7fa6440ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.096+0000 7fa64b794640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440b4c40 con 0x7fa6440b5d30 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.096+0000 7fa64b794640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440d3820 con 0x7fa6440cf910 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.096+0000 7fa64b794640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6440b4dc0 con 0x7fa6440b4120 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 0x7fa6440cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 0x7fa6440cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53228/0 (socket says 172.21.15.22:53228) 2024-09-21T05:48:36.100 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 -- 172.21.15.22:0/1838036177 learned_addr learned my addr 172.21.15.22:0/1838036177 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa649530640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6440b4120 0x7fa64415ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa649d31640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6440cf910 0x7fa6440ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 -- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6440b4120 msgr2=0x7fa64415ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6440b4120 0x7fa64415ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 -- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6440cf910 msgr2=0x7fa6440ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6440cf910 0x7fa6440ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 -- 172.21.15.22:0/1838036177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6441ea380 con 0x7fa6440b5d30 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa648d2f640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 0x7fa6440cdd80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa638000c00 tx=0x7fa638033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638031d60 con 0x7fa6440b5d30 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa638033e10 con 0x7fa6440b5d30 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6440cb880 con 0x7fa6440b5d30 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.097+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638033310 con 0x7fa6440b5d30 2024-09-21T05:48:36.101 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.098+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6440b32e0 con 0x7fa6440b5d30 2024-09-21T05:48:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.099+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6440b52e0 con 0x7fa6440b5d30 2024-09-21T05:48:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.099+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6380334b0 con 0x7fa6440b5d30 2024-09-21T05:48:36.102 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.100+0000 7fa641ffb640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa630078bc0 0x7fa63007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.100+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa6380bbe80 con 0x7fa6440b5d30 2024-09-21T05:48:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.100+0000 7fa649530640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa630078bc0 0x7fa63007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.103 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.101+0000 7fa649530640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa630078bc0 0x7fa63007b0d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa634005fd0 tx=0x7fa634009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:36.107 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.105+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa63803f030 con 0x7fa6440b5d30 2024-09-21T05:48:36.289 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.288+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6441ea130 con 0x7fa630078bc0 2024-09-21T05:48:36.290 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.288+0000 7fa641ffb640 1 -- 172.21.15.22:0/1838036177 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13494 (secure 0 0 0) 0x7fa6441ea130 con 0x7fa630078bc0 2024-09-21T05:48:36.291 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:36.291 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:48:36.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.290+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa630078bc0 msgr2=0x7fa63007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.293 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.290+0000 7fa64b794640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa630078bc0 0x7fa63007b0d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa634005fd0 tx=0x7fa634009500 comp rx=0 tx=0).stop 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.290+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 msgr2=0x7fa6440cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.290+0000 7fa64b794640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 0x7fa6440cdd80 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa638000c00 tx=0x7fa638033c20 comp rx=0 tx=0).stop 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 shutdown_connections 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa6440cf910 0x7fa6440ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa6440b5d30 0x7fa6440cdd80 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa630078bc0 0x7fa63007b0d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 --2- 172.21.15.22:0/1838036177 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6440b4120 0x7fa64415ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.294 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 >> 172.21.15.22:0/1838036177 conn(0x7fa6440d1220 msgr2=0x7fa64406d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:36.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 shutdown_connections 2024-09-21T05:48:36.295 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.291+0000 7fa64b794640 1 -- 172.21.15.22:0/1838036177 wait complete. 2024-09-21T05:48:36.304 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-21T05:48:35.888870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":286740,"kb_used_data":16428,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687980,"statfs":{"total":773094113280,"available":772800491520,"internally_reserved":0,"allocated":16822272,"data_stored":10346943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390143,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-256,"num_write":-165,"num_write_kb":-8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001849"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-21T05:48:31.436412+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:48:31.436412+0000","last_peered":"2024-09-21T05:48:31.436412+0000","last_clean":"2024-09-21T05:48:31.436412+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:48:31.436412+0000","last_undegraded":"2024-09-21T05:48:31.436412+0000","last_fullsized":"2024-09-21T05:48:31.436412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":1998848,"data_stored":1199311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":614638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":620883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36756,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45000,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326840,"statfs":{"total":96636764160,"available":96590684160,"internally_reserved":0,"allocated":5619712,"data_stored":4815175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37336,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334504,"statfs":{"total":96636764160,"available":96598532096,"internally_reserved":0,"allocated":2035712,"data_stored":1217010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:48:36.304 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T05:48:36.421 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 -- 172.21.15.22:0/577725606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b56e0 msgr2=0x7f75ec0b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 --2- 172.21.15.22:0/577725606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b56e0 0x7f75ec0b5af0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0b05f0 tx=0x7f75e002a5f0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 -- 172.21.15.22:0/577725606 shutdown_connections 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 --2- 172.21.15.22:0/577725606 >> v2:172.21.15.134:3300/0 conn(0x7f75ec0b6030 0x7f75ec071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 --2- 172.21.15.22:0/577725606 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b56e0 0x7f75ec0b5af0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 --2- 172.21.15.22:0/577725606 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b4ed0 0x7f75ec142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 -- 172.21.15.22:0/577725606 >> 172.21.15.22:0/577725606 conn(0x7f75ec09ba20 msgr2=0x7f75ec0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 -- 172.21.15.22:0/577725606 shutdown_connections 2024-09-21T05:48:36.422 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.419+0000 7f75f26de640 1 -- 172.21.15.22:0/577725606 wait complete. 2024-09-21T05:48:36.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.421+0000 7f75f26de640 1 Processor -- start 2024-09-21T05:48:36.423 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.421+0000 7f75f26de640 1 -- start start 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.422+0000 7f75f26de640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 0x7f75ec09c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.422+0000 7f75f26de640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b56e0 0x7f75ec15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.422+0000 7f75f26de640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75ec0b6030 0x7f75ec1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75f26de640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec071db0 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75f26de640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f75ec0b1d60 con 0x7f75ec0b6030 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75f26de640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f75ec071f30 con 0x7f75ec0b56e0 2024-09-21T05:48:36.426 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 0x7f75ec09c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 0x7f75ec09c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53248/0 (socket says 172.21.15.22:53248) 2024-09-21T05:48:36.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 -- 172.21.15.22:0/2374897017 learned_addr learned my addr 172.21.15.22:0/2374897017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:36.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75f0c7b640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75ec0b6030 0x7f75ec1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75eb7fe640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b56e0 0x7f75ec15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 -- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b56e0 msgr2=0x7f75ec15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.427 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b56e0 0x7f75ec15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 -- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75ec0b6030 msgr2=0x7f75ec1f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75ec0b6030 0x7f75ec1f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 -- 172.21.15.22:0/2374897017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ec133e80 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75eb7fe640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b56e0 0x7f75ec15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75ebfff640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 0x7f75ec09c970 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00d660 tx=0x7f75dc00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc017070 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75dc005310 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.428 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ec134160 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc009820 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.429 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.423+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ec0cfb10 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.425+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ec0cf900 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.425+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f75dc0099c0 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.426+0000 7f75e8ff9640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75d8078bc0 0x7f75d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:36.430 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.426+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f75dc09aae0 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.426+0000 7f75eb7fe640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75d8078bc0 0x7f75d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:36.431 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.427+0000 7f75eb7fe640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75d8078bc0 0x7f75d807b0d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f75e0033000 tx=0x7f75e0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:36.433 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.431+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f75dc062b20 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.619+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75ec0723e0 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.621 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.619+0000 7f75e8ff9640 1 -- 172.21.15.22:0/2374897017 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+13290 (secure 0 0 0) 0x7f75dc067550 con 0x7f75ec0b4ed0 2024-09-21T05:48:36.622 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:36.622 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":116,"fsid":"cdac58e0-7b6b-4531-9caa-dad309e0d36b","created":"2024-09-21T05:38:57.360847+0000","modified":"2024-09-21T05:48:33.427599+0000","last_up_change":"2024-09-21T05:39:10.745194+0000","last_in_change":"2024-09-21T05:39:00.975574+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T05:39:09.753482+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"92c772d4-9ad3-4bec-9dfc-c5fa87f93276","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6801","nonce":3448237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6803","nonce":3448237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6807","nonce":3448237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3448237758},{"type":"v1","addr":"172.21.15.22:6805","nonce":3448237758}]},"public_addr":"172.21.15.22:6801/3448237758","cluster_addr":"172.21.15.22:6803/3448237758","heartbeat_back_addr":"172.21.15.22:6807/3448237758","heartbeat_front_addr":"172.21.15.22:6805/3448237758","state":["exists","up"]},{"osd":1,"uuid":"6c399baa-b203-49d6-bb41-19e6c7235192","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6824","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6825","nonce":48261475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6826","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6827","nonce":48261475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6830","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6831","nonce":48261475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6828","nonce":48261475},{"type":"v1","addr":"172.21.15.22:6829","nonce":48261475}]},"public_addr":"172.21.15.22:6825/48261475","cluster_addr":"172.21.15.22:6827/48261475","heartbeat_back_addr":"172.21.15.22:6831/48261475","heartbeat_front_addr":"172.21.15.22:6829/48261475","state":["exists","up"]},{"osd":2,"uuid":"acd6191b-3a15-4ec8-a7ce-bf0a5da639d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6816","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6817","nonce":3460873616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6818","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6819","nonce":3460873616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6822","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6823","nonce":3460873616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6820","nonce":3460873616},{"type":"v1","addr":"172.21.15.22:6821","nonce":3460873616}]},"public_addr":"172.21.15.22:6817/3460873616","cluster_addr":"172.21.15.22:6819/3460873616","heartbeat_back_addr":"172.21.15.22:6823/3460873616","heartbeat_front_addr":"172.21.15.22:6821/3460873616","state":["exists","up"]},{"osd":3,"uuid":"f97b07de-93b7-4c01-8ed7-c996a3c64984","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6809","nonce":3633643032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6810","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6811","nonce":3633643032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6814","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6815","nonce":3633643032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6812","nonce":3633643032},{"type":"v1","addr":"172.21.15.22:6813","nonce":3633643032}]},"public_addr":"172.21.15.22:6809/3633643032","cluster_addr":"172.21.15.22:6811/3633643032","heartbeat_back_addr":"172.21.15.22:6815/3633643032","heartbeat_front_addr":"172.21.15.22:6813/3633643032","state":["exists","up"]},{"osd":4,"uuid":"afb9bfb3-4ba6-4180-bc3d-d24b7b92d9a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6827","nonce":3950009634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6829","nonce":3950009634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6833","nonce":3950009634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6830","nonce":3950009634},{"type":"v1","addr":"172.21.15.134:6831","nonce":3950009634}]},"public_addr":"172.21.15.134:6827/3950009634","cluster_addr":"172.21.15.134:6829/3950009634","heartbeat_back_addr":"172.21.15.134:6833/3950009634","heartbeat_front_addr":"172.21.15.134:6831/3950009634","state":["exists","up"]},{"osd":5,"uuid":"2ab1fef7-ea46-4098-bd29-4d998a2ef698","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6803","nonce":1222371224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6805","nonce":1222371224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6809","nonce":1222371224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":1222371224},{"type":"v1","addr":"172.21.15.134:6807","nonce":1222371224}]},"public_addr":"172.21.15.134:6803/1222371224","cluster_addr":"172.21.15.134:6805/1222371224","heartbeat_back_addr":"172.21.15.134:6809/1222371224","heartbeat_front_addr":"172.21.15.134:6807/1222371224","state":["exists","up"]},{"osd":6,"uuid":"7d8b7b3f-e396-4cdf-ada4-ecae5686900c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6811","nonce":640262620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6813","nonce":640262620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6817","nonce":640262620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6814","nonce":640262620},{"type":"v1","addr":"172.21.15.134:6815","nonce":640262620}]},"public_addr":"172.21.15.134:6811/640262620","cluster_addr":"172.21.15.134:6813/640262620","heartbeat_back_addr":"172.21.15.134:6817/640262620","heartbeat_front_addr":"172.21.15.134:6815/640262620","state":["exists","up"]},{"osd":7,"uuid":"5e3b598f-75b2-46e0-9c5b-a3268b93badb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6818","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6819","nonce":1266427391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6820","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6821","nonce":1266427391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6824","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6825","nonce":1266427391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":1266427391},{"type":"v1","addr":"172.21.15.134:6823","nonce":1266427391}]},"public_addr":"172.21.15.134:6819/1266427391","cluster_addr":"172.21.15.134:6821/1266427391","heartbeat_back_addr":"172.21.15.134:6825/1266427391","heartbeat_front_addr":"172.21.15.134:6823/1266427391","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.303889+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.584014+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.370657+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.320927+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.515745+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.292730+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:06.971914+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T05:39:07.857829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:6833/433765027":"2024-09-22T05:48:30.423383+0000","172.21.15.22:6835/3772214991":"2024-09-22T05:48:18.533817+0000","172.21.15.22:6833/3772214991":"2024-09-22T05:48:18.533817+0000","172.21.15.22:6835/433765027":"2024-09-22T05:48:30.423383+0000","172.21.15.134:6837/4249862141":"2024-09-22T05:48:09.072398+0000","172.21.15.134:6835/4249862141":"2024-09-22T05:48:09.072398+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75d8078bc0 msgr2=0x7f75d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75d8078bc0 0x7f75d807b0d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f75e0033000 tx=0x7f75e0004730 comp rx=0 tx=0).stop 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 msgr2=0x7f75ec09c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 0x7f75ec09c970 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00d660 tx=0x7f75dc00db20 comp rx=0 tx=0).stop 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 shutdown_connections 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f75ec0b6030 0x7f75ec1f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f75d8078bc0 0x7f75d807b0d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.624 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f75ec0b56e0 0x7f75ec15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 --2- 172.21.15.22:0/2374897017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f75ec0b4ed0 0x7f75ec09c970 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:36.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 >> 172.21.15.22:0/2374897017 conn(0x7f75ec09ba20 msgr2=0x7f75ec1453c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:36.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 shutdown_connections 2024-09-21T05:48:36.625 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:36.622+0000 7f75f26de640 1 -- 172.21.15.22:0/2374897017 wait complete. 2024-09-21T05:48:37.636 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T05:48:37.636 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:37.772 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 -- 172.21.15.22:0/3342928522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 msgr2=0x7ffb000cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 --2- 172.21.15.22:0/3342928522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb000cf3d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005fd0 tx=0x7ffaf002a5d0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 -- 172.21.15.22:0/3342928522 shutdown_connections 2024-09-21T05:48:37.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 --2- 172.21.15.22:0/3342928522 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb000cf3d0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 -- 172.21.15.22:0/3342928522 >> 172.21.15.22:0/3342928522 conn(0x7ffb000d1220 msgr2=0x7ffb000d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:37.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 -- 172.21.15.22:0/3342928522 shutdown_connections 2024-09-21T05:48:37.773 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.770+0000 7ffb061e6640 1 -- 172.21.15.22:0/3342928522 wait complete. 2024-09-21T05:48:37.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.772+0000 7ffb061e6640 1 Processor -- start 2024-09-21T05:48:37.774 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.772+0000 7ffb061e6640 1 -- start start 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.773+0000 7ffb061e6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b5130 0x7ffb0015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.773+0000 7ffb061e6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb0014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.773+0000 7ffb061e6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb000cf910 0x7ffb000b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.773+0000 7ffb061e6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000d32e0 con 0x7ffb000b5d30 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.773+0000 7ffb061e6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000d3160 con 0x7ffb000b5130 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.773+0000 7ffb061e6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000d3460 con 0x7ffb000cf910 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb0014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb0014e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53258/0 (socket says 172.21.15.22:53258) 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 -- 172.21.15.22:0/3498503575 learned_addr learned my addr 172.21.15.22:0/3498503575 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffaff7fe640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b5130 0x7ffb0015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 -- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb000cf910 msgr2=0x7ffb000b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.777 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb000cf910 0x7ffb000b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 -- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b5130 msgr2=0x7ffb0015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b5130 0x7ffb0015d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 -- 172.21.15.22:0/3498503575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb0014ecb0 con 0x7ffb000b5d30 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffafeffd640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb0014e770 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400c3c0 tx=0x7ffaf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4017070 con 0x7ffb000b5d30 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf40053c0 con 0x7ffb000b5d30 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb001f6cb0 con 0x7ffb000b5d30 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4003870 con 0x7ffb000b5d30 2024-09-21T05:48:37.778 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.774+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb001f61d0 con 0x7ffb000b5d30 2024-09-21T05:48:37.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.775+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb000cf3d0 con 0x7ffb000b5d30 2024-09-21T05:48:37.779 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.776+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffaf401a070 con 0x7ffb000b5d30 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffae7fff640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffaec078b90 0x7ffaec07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7ffaf4099ec0 con 0x7ffb000b5d30 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffae7fff640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7ffaec07ebe0 0x7ffaec081030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffaf4016ec0 con 0x7ffaec07ebe0 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7ffaf409c5c0 con 0x7ffb000b5d30 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffaff7fe640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffaec078b90 0x7ffaec07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffafffff640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7ffaec07ebe0 0x7ffaec081030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.777+0000 7ffafffff640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7ffaec07ebe0 0x7ffaec081030 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.778+0000 7ffaff7fe640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffaec078b90 0x7ffaec07b0a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0033000 tx=0x7ffaf0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:37.780 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.778+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== osd.0 v2:172.21.15.22:6800/3448237758 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffaf4016ec0 con 0x7ffaec07ebe0 2024-09-21T05:48:37.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 --> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffb0015dec0 con 0x7ffaec07ebe0 2024-09-21T05:48:37.793 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffae7fff640 1 -- 172.21.15.22:0/3498503575 <== osd.0 v2:172.21.15.22:6800/3448237758 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffb0015dec0 con 0x7ffaec07ebe0 2024-09-21T05:48:37.793 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:37.793 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:37.793 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7ffaec07ebe0 msgr2=0x7ffaec081030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:6800/3448237758,v1:172.21.15.22:6801/3448237758] conn(0x7ffaec07ebe0 0x7ffaec081030 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffaec078b90 msgr2=0x7ffaec07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffaec078b90 0x7ffaec07b0a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0033000 tx=0x7ffaf0004730 comp rx=0 tx=0).stop 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 msgr2=0x7ffb0014e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 --2- 172.21.15.22:0/3498503575 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b5d30 0x7ffb0014e770 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400c3c0 tx=0x7ffaf400c880 comp rx=0 tx=0).stop 2024-09-21T05:48:37.794 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffafffff640 1 -- 172.21.15.22:0/3498503575 reap_dead start 2024-09-21T05:48:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.791+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 shutdown_connections 2024-09-21T05:48:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.792+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 >> 172.21.15.22:0/3498503575 conn(0x7ffb000d1220 msgr2=0x7ffb00075de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.792+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 shutdown_connections 2024-09-21T05:48:37.795 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.792+0000 7ffb061e6640 1 -- 172.21.15.22:0/3498503575 wait complete. 2024-09-21T05:48:37.805 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/4051313461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680cdc10 msgr2=0x7f6c680ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/4051313461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680cdc10 0x7f6c680ce020 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6c60005fd0 tx=0x7f6c6002a8b0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/4051313461 shutdown_connections 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/4051313461 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680cdc10 0x7f6c680ce020 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/4051313461 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c680b7610 0x7f6c680b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/4051313461 >> 172.21.15.22:0/4051313461 conn(0x7f6c680d2580 msgr2=0x7f6c680d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/4051313461 shutdown_connections 2024-09-21T05:48:37.923 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.920+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/4051313461 wait complete. 2024-09-21T05:48:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.922+0000 7f6c6d7e4640 1 Processor -- start 2024-09-21T05:48:37.924 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.922+0000 7f6c6d7e4640 1 -- start start 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.923+0000 7f6c6d7e4640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 0x7f6c681641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c680cd0f0 0x7f6c680ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c680cdc10 0x7f6c6815d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680b6a70 con 0x7f6c680b7610 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6c680b68f0 con 0x7f6c680cd0f0 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6c680b6bf0 con 0x7f6c680cdc10 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 0x7f6c681641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.927 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 0x7f6c681641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53278/0 (socket says 172.21.15.22:53278) 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 -- 172.21.15.22:0/2393364341 learned_addr learned my addr 172.21.15.22:0/2393364341 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 -- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c680cdc10 msgr2=0x7f6c6815d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c680cdc10 0x7f6c6815d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 -- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c680cd0f0 msgr2=0x7f6c680ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c667fc640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c680cd0f0 0x7f6c680ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c680cd0f0 0x7f6c680ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 -- 172.21.15.22:0/2393364341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c680b0800 con 0x7f6c680b7610 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c667fc640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c680cd0f0 0x7f6c680ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c66ffd640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 0x7f6c681641d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800d770 tx=0x7f6c5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c58017070 con 0x7f6c680b7610 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c58009d20 con 0x7f6c680b7610 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c580106c0 con 0x7f6c680b7610 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c680b0ae0 con 0x7f6c680b7610 2024-09-21T05:48:37.928 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.924+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c680b10a0 con 0x7f6c680b7610 2024-09-21T05:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.925+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c680ce020 con 0x7f6c680b7610 2024-09-21T05:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.926+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c5801a070 con 0x7f6c680b7610 2024-09-21T05:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.927+0000 7f6c4f7fe640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c54078bc0 0x7f6c5407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:37.929 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.927+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f6c5809ac10 con 0x7f6c680b7610 2024-09-21T05:48:37.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.927+0000 7f6c667fc640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c54078bc0 0x7f6c5407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:37.930 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.927+0000 7f6c667fc640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c54078bc0 0x7f6c5407b0d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6c680b2460 tx=0x7f6c5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:37.933 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:37.931+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c580632a0 con 0x7f6c680b7610 2024-09-21T05:48:38.119 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.118+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f6c680d2d10 con 0x7f6c54078bc0 2024-09-21T05:48:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.118+0000 7f6c4f7fe640 1 -- 172.21.15.22:0/2393364341 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6c680d2d10 con 0x7f6c54078bc0 2024-09-21T05:48:38.120 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c54078bc0 msgr2=0x7f6c5407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c54078bc0 0x7f6c5407b0d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6c680b2460 tx=0x7f6c5c009040 comp rx=0 tx=0).stop 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 msgr2=0x7f6c681641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 0x7f6c681641d0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6c5800d770 tx=0x7f6c5800dc30 comp rx=0 tx=0).stop 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 shutdown_connections 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6c54078bc0 0x7f6c5407b0d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6c680cdc10 0x7f6c6815d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6c680cd0f0 0x7f6c680ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 --2- 172.21.15.22:0/2393364341 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6c680b7610 0x7f6c681641d0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 >> 172.21.15.22:0/2393364341 conn(0x7f6c680d2580 msgr2=0x7f6c680b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.123 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 shutdown_connections 2024-09-21T05:48:38.124 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.120+0000 7f6c6d7e4640 1 -- 172.21.15.22:0/2393364341 wait complete. 2024-09-21T05:48:38.134 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T05:48:38.134 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:38.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 -- 172.21.15.22:0/2016185560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5d30 msgr2=0x7ff3ec0cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 --2- 172.21.15.22:0/2016185560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5d30 0x7ff3ec0cf3d0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc005fd0 tx=0x7ff3dc02a940 comp rx=0 tx=0).stop 2024-09-21T05:48:38.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 -- 172.21.15.22:0/2016185560 shutdown_connections 2024-09-21T05:48:38.255 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 --2- 172.21.15.22:0/2016185560 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5d30 0x7ff3ec0cf3d0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 -- 172.21.15.22:0/2016185560 >> 172.21.15.22:0/2016185560 conn(0x7ff3ec0d1220 msgr2=0x7ff3ec0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 -- 172.21.15.22:0/2016185560 shutdown_connections 2024-09-21T05:48:38.256 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.253+0000 7ff3f167e640 1 -- 172.21.15.22:0/2016185560 wait complete. 2024-09-21T05:48:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.255+0000 7ff3f167e640 1 Processor -- start 2024-09-21T05:48:38.257 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.255+0000 7ff3f167e640 1 -- start start 2024-09-21T05:48:38.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3f167e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5130 0x7ff3ec0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3f167e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec0b5d30 0x7ff3ec15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3f167e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3ec0cf910 0x7ff3ec14dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.259 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3f167e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec0d32a0 con 0x7ff3ec0b5130 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3f167e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec0d3120 con 0x7ff3ec0b5d30 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3f167e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ff3ec0d3420 con 0x7ff3ec0cf910 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5130 0x7ff3ec0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5130 0x7ff3ec0cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53302/0 (socket says 172.21.15.22:53302) 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 -- 172.21.15.22:0/2389548595 learned_addr learned my addr 172.21.15.22:0/2389548595 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3ea7fc640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec0b5d30 0x7ff3ec15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eb7fe640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3ec0cf910 0x7ff3ec14dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 -- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3ec0cf910 msgr2=0x7ff3ec14dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ff3ec0cf910 0x7ff3ec14dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 -- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec0b5d30 msgr2=0x7ff3ec15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec0b5d30 0x7ff3ec15fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3eaffd640 1 -- 172.21.15.22:0/2389548595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec14e110 con 0x7ff3ec0b5130 2024-09-21T05:48:38.260 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.256+0000 7ff3ea7fc640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ff3ec0b5d30 0x7ff3ec15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.257+0000 7ff3eaffd640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5130 0x7ff3ec0cd240 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009ca0 tx=0x7ff3dc031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.257+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc03e070 con 0x7ff3ec0b5130 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.257+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3dc004540 con 0x7ff3ec0b5130 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.257+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec15cdd0 con 0x7ff3ec0b5130 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.257+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc043460 con 0x7ff3ec0b5130 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.257+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec0cdbe0 con 0x7ff3ec0b5130 2024-09-21T05:48:38.261 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.258+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff3ec0cf3d0 con 0x7ff3ec0b5130 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff3dc043600 con 0x7ff3ec0b5130 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3d37fe640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff3d8078bc0 0x7ff3d807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7ff3dc0bca70 con 0x7ff3ec0b5130 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3d37fe640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7ff3d807ec10 0x7ff3d8081060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3dc02be90 con 0x7ff3d807ec10 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7ff3dc0bcf00 con 0x7ff3ec0b5130 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3eb7fe640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7ff3d807ec10 0x7ff3d8081060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.262 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.259+0000 7ff3ea7fc640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff3d8078bc0 0x7ff3d807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.260+0000 7ff3eb7fe640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7ff3d807ec10 0x7ff3d8081060 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.260+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== osd.1 v2:172.21.15.22:6824/48261475 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3dc02be90 con 0x7ff3d807ec10 2024-09-21T05:48:38.263 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.260+0000 7ff3ea7fc640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff3d8078bc0 0x7ff3d807b0d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0ce950 tx=0x7ff3e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.273+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 --> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff3ec161ff0 con 0x7ff3d807ec10 2024-09-21T05:48:38.275 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.273+0000 7ff3d37fe640 1 -- 172.21.15.22:0/2389548595 <== osd.1 v2:172.21.15.22:6824/48261475 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff3ec161ff0 con 0x7ff3d807ec10 2024-09-21T05:48:38.276 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:38.276 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:38.276 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:38.276 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7ff3d807ec10 msgr2=0x7ff3d8081060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:6824/48261475,v1:172.21.15.22:6825/48261475] conn(0x7ff3d807ec10 0x7ff3d8081060 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff3d8078bc0 msgr2=0x7ff3d807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ff3d8078bc0 0x7ff3d807b0d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0ce950 tx=0x7ff3e0009040 comp rx=0 tx=0).stop 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5130 msgr2=0x7ff3ec0cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 --2- 172.21.15.22:0/2389548595 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ff3ec0b5130 0x7ff3ec0cd240 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009ca0 tx=0x7ff3dc031e20 comp rx=0 tx=0).stop 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3eb7fe640 1 -- 172.21.15.22:0/2389548595 reap_dead start 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 shutdown_connections 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 >> 172.21.15.22:0/2389548595 conn(0x7ff3ec0d1220 msgr2=0x7ff3ec0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 shutdown_connections 2024-09-21T05:48:38.277 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.274+0000 7ff3f167e640 1 -- 172.21.15.22:0/2389548595 wait complete. 2024-09-21T05:48:38.287 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 -- 172.21.15.22:0/3797292472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b56e0 msgr2=0x7f6ec00b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 --2- 172.21.15.22:0/3797292472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b56e0 0x7f6ec00b5af0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f6ec006dba0 tx=0x7f6eb402a610 comp rx=0 tx=0).stop 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 -- 172.21.15.22:0/3797292472 shutdown_connections 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 --2- 172.21.15.22:0/3797292472 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b56e0 0x7f6ec00b5af0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 --2- 172.21.15.22:0/3797292472 >> v2:172.21.15.134:3300/0 conn(0x7f6ec00b4ed0 0x7f6ec0142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 -- 172.21.15.22:0/3797292472 >> 172.21.15.22:0/3797292472 conn(0x7f6ec009ba20 msgr2=0x7f6ec00b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.401+0000 7f6ec742e640 1 -- 172.21.15.22:0/3797292472 shutdown_connections 2024-09-21T05:48:38.404 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.402+0000 7f6ec742e640 1 -- 172.21.15.22:0/3797292472 wait complete. 2024-09-21T05:48:38.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.403+0000 7f6ec742e640 1 Processor -- start 2024-09-21T05:48:38.405 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.403+0000 7f6ec742e640 1 -- start start 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 0x7f6ec009fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ec00b56e0 0x7f6ec015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec00b6030 0x7f6ec01f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec00b1cd0 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec00b1b50 con 0x7f6ec00b6030 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec0071f80 con 0x7f6ec00b56e0 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 0x7f6ec009fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.408 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 0x7f6ec009fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53328/0 (socket says 172.21.15.22:53328) 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 -- 172.21.15.22:0/199774678 learned_addr learned my addr 172.21.15.22:0/199774678 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec59cb640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec00b6030 0x7f6ec01f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec49c9640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ec00b56e0 0x7f6ec015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 -- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ec00b56e0 msgr2=0x7f6ec015e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ec00b56e0 0x7f6ec015e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 -- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec00b6030 msgr2=0x7f6ec01f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec00b6030 0x7f6ec01f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 -- 172.21.15.22:0/199774678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec00a0320 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec51ca640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 0x7f6ec009fde0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b410 tx=0x7f6ebc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc015020 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ebc011920 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.409 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec015e8f0 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.405+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc011d70 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.406+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec015f050 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.410 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.407+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec0072490 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.407+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ebc018020 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.408+0000 7f6eb1ffb640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ea4078b90 0x7f6ea407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.408+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f6ebc09a9c0 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.408+0000 7f6ec49c9640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ea4078b90 0x7f6ea407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.411 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.409+0000 7f6ec49c9640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ea4078b90 0x7f6ea407b0a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4033000 tx=0x7f6eb4004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.414 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.412+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6ebc09ce20 con 0x7f6ec00b4ed0 2024-09-21T05:48:38.602 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.600+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f6ec015cc80 con 0x7f6ea4078b90 2024-09-21T05:48:38.603 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.601+0000 7f6eb1ffb640 1 -- 172.21.15.22:0/199774678 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ec015cc80 con 0x7f6ea4078b90 2024-09-21T05:48:38.603 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.603+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ea4078b90 msgr2=0x7f6ea407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.603+0000 7f6ec742e640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ea4078b90 0x7f6ea407b0a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4033000 tx=0x7f6eb4004730 comp rx=0 tx=0).stop 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.603+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 msgr2=0x7f6ec009fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.603+0000 7f6ec742e640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 0x7f6ec009fde0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b410 tx=0x7f6ebc00b8d0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.603+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 shutdown_connections 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f6ec00b6030 0x7f6ec01f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f6ea4078b90 0x7f6ea407b0a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f6ec00b56e0 0x7f6ec015e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.606 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 --2- 172.21.15.22:0/199774678 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f6ec00b4ed0 0x7f6ec009fde0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 >> 172.21.15.22:0/199774678 conn(0x7f6ec009ba20 msgr2=0x7f6ec0070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 shutdown_connections 2024-09-21T05:48:38.607 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.604+0000 7f6ec742e640 1 -- 172.21.15.22:0/199774678 wait complete. 2024-09-21T05:48:38.617 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T05:48:38.617 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e6463640 1 -- 172.21.15.22:0/2792766076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 msgr2=0x7f12c8004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.738 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e6463640 1 --2- 172.21.15.22:0/2792766076 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 0x7f12c8004060 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009b50 tx=0x7f12c8005830 comp rx=0 tx=0).stop 2024-09-21T05:48:38.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e4a00640 1 -- 172.21.15.22:0/2792766076 reap_dead start 2024-09-21T05:48:38.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e6463640 1 -- 172.21.15.22:0/2792766076 shutdown_connections 2024-09-21T05:48:38.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e6463640 1 -- 172.21.15.22:0/2792766076 >> 172.21.15.22:0/2792766076 conn(0x7f12e00b2320 msgr2=0x7f12e00b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e6463640 1 -- 172.21.15.22:0/2792766076 shutdown_connections 2024-09-21T05:48:38.739 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.736+0000 7f12e6463640 1 -- 172.21.15.22:0/2792766076 wait complete. 2024-09-21T05:48:38.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.738+0000 7f12e6463640 1 Processor -- start 2024-09-21T05:48:38.740 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.738+0000 7f12e6463640 1 -- start start 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12e6463640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 0x7f12e00cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12e6463640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12e01f5be0 0x7f12e00af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12e6463640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12e015af40 0x7f12e015b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12e6463640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f12e00b4d90 con 0x7f12c8003c10 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12e6463640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f12e00b4c10 con 0x7f12e01f5be0 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12e6463640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f12e00b4f10 con 0x7f12e015af40 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 0x7f12e00cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12dffff640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 0x7f12e00cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53354/0 (socket says 172.21.15.22:53354) 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.739+0000 7f12dffff640 1 -- 172.21.15.22:0/2395057231 learned_addr learned my addr 172.21.15.22:0/2395057231 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12df7fe640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12e01f5be0 0x7f12e00af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dffff640 1 -- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12e015af40 msgr2=0x7f12e015b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dffff640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f12e015af40 0x7f12e015b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.743 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dffff640 1 -- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12e01f5be0 msgr2=0x7f12e00af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dffff640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f12e01f5be0 0x7f12e00af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dffff640 1 -- 172.21.15.22:0/2395057231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e015b890 con 0x7f12c8003c10 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dffff640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 0x7f12e00cc360 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009c70 tx=0x7f12d0033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0038040 con 0x7f12c8003c10 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d0004020 con 0x7f12c8003c10 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e00af550 con 0x7f12c8003c10 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0004470 con 0x7f12c8003c10 2024-09-21T05:48:38.744 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.740+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e00b3da0 con 0x7f12c8003c10 2024-09-21T05:48:38.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.741+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f12e00b54a0 con 0x7f12c8003c10 2024-09-21T05:48:38.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.742+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12d0045020 con 0x7f12c8003c10 2024-09-21T05:48:38.745 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12dcff9640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f12cc078bc0 0x7f12cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f12d00bc0d0 con 0x7f12c8003c10 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12dcff9640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f12cc07ec10 0x7f12cc081060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f12cc081730 con 0x7f12cc07ec10 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f12d00bc480 con 0x7f12c8003c10 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12e4a00640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f12cc07ec10 0x7f12cc081060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12df7fe640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f12cc078bc0 0x7f12cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.743+0000 7f12e4a00640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f12cc07ec10 0x7f12cc081060 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.744+0000 7f12df7fe640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f12cc078bc0 0x7f12cc07b0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000fd0 tx=0x7f12d4001e10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.746 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.744+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== osd.2 v2:172.21.15.22:6816/3460873616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f12cc081730 con 0x7f12cc07ec10 2024-09-21T05:48:38.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 --> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f12e00ccf10 con 0x7f12cc07ec10 2024-09-21T05:48:38.759 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12dcff9640 1 -- 172.21.15.22:0/2395057231 <== osd.2 v2:172.21.15.22:6816/3460873616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f12e00ccf10 con 0x7f12cc07ec10 2024-09-21T05:48:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:38.759 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:38.759 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:38.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f12cc07ec10 msgr2=0x7f12cc081060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:6816/3460873616,v1:172.21.15.22:6817/3460873616] conn(0x7f12cc07ec10 0x7f12cc081060 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f12cc078bc0 msgr2=0x7f12cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f12cc078bc0 0x7f12cc07b0d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f12d4000fd0 tx=0x7f12d4001e10 comp rx=0 tx=0).stop 2024-09-21T05:48:38.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 msgr2=0x7f12e00cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.760 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 --2- 172.21.15.22:0/2395057231 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f12c8003c10 0x7f12e00cc360 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009c70 tx=0x7f12d0033670 comp rx=0 tx=0).stop 2024-09-21T05:48:38.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e4a00640 1 -- 172.21.15.22:0/2395057231 reap_dead start 2024-09-21T05:48:38.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 shutdown_connections 2024-09-21T05:48:38.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.757+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 >> 172.21.15.22:0/2395057231 conn(0x7f12e00b2320 msgr2=0x7f12e0158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.758+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 shutdown_connections 2024-09-21T05:48:38.761 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.758+0000 7f12e6463640 1 -- 172.21.15.22:0/2395057231 wait complete. 2024-09-21T05:48:38.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T05:48:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 -- 172.21.15.22:0/1923321821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900b5fa0 msgr2=0x7f55900b63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1923321821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900b5fa0 0x7f55900b63b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f55900b5c40 tx=0x7f558802aa30 comp rx=0 tx=0).stop 2024-09-21T05:48:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 -- 172.21.15.22:0/1923321821 shutdown_connections 2024-09-21T05:48:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1923321821 >> v2:172.21.15.134:3300/0 conn(0x7f55900d2fc0 0x7f55900d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1923321821 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900b5fa0 0x7f55900b63b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.887 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1923321821 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55900d09a0 0x7f55900b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 -- 172.21.15.22:0/1923321821 >> 172.21.15.22:0/1923321821 conn(0x7f55900d4860 msgr2=0x7f55900d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 -- 172.21.15.22:0/1923321821 shutdown_connections 2024-09-21T05:48:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.885+0000 7f5595bb6640 1 -- 172.21.15.22:0/1923321821 wait complete. 2024-09-21T05:48:38.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.887+0000 7f5595bb6640 1 Processor -- start 2024-09-21T05:48:38.889 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.887+0000 7f5595bb6640 1 -- start start 2024-09-21T05:48:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f5595bb6640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55900b5fa0 0x7f55901e8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f5595bb6640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 0x7f55900c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f5595bb6640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55900d2fc0 0x7f55901571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f5595bb6640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f55900cf350 con 0x7f55900d09a0 2024-09-21T05:48:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f5595bb6640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f55900b3a90 con 0x7f55900d2fc0 2024-09-21T05:48:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f5595bb6640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f55900cf4d0 con 0x7f55900b5fa0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 0x7f55900c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 0x7f55900c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53372/0 (socket says 172.21.15.22:53372) 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 -- 172.21.15.22:0/1934969236 learned_addr learned my addr 172.21.15.22:0/1934969236 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 -- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55900b5fa0 msgr2=0x7f55901e8030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55900b5fa0 0x7f55901e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 -- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55900d2fc0 msgr2=0x7f55901571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55900d2fc0 0x7f55901571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.888+0000 7f558effd640 1 -- 172.21.15.22:0/1934969236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55901f97c0 con 0x7f55900d09a0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.889+0000 7f558effd640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 0x7f55900c8b40 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f558400e7c0 tx=0x7f558400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.889+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558400cc60 con 0x7f55900d09a0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.889+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55901f8f20 con 0x7f55900d09a0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.889+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f558400ee70 con 0x7f55900d09a0 2024-09-21T05:48:38.892 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.889+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584010690 con 0x7f55900d09a0 2024-09-21T05:48:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.889+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55901f94e0 con 0x7f55900d09a0 2024-09-21T05:48:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.890+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55900b5550 con 0x7f55900d09a0 2024-09-21T05:48:38.893 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.891+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5584010830 con 0x7f55900d09a0 2024-09-21T05:48:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.891+0000 7f5577fff640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f557c078b90 0x7f557c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.891+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f558409b4c0 con 0x7f55900d09a0 2024-09-21T05:48:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.891+0000 7f558f7fe640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f557c078b90 0x7f557c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:38.894 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.892+0000 7f558f7fe640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f557c078b90 0x7f557c07b0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f558000b980 tx=0x7f5580005db0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:38.897 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:38.896+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5584063ad0 con 0x7f55900d09a0 2024-09-21T05:48:39.083 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.081+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f55901f8c40 con 0x7f557c078b90 2024-09-21T05:48:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.082+0000 7f5577fff640 1 -- 172.21.15.22:0/1934969236 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f55901f8c40 con 0x7f557c078b90 2024-09-21T05:48:39.084 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f557c078b90 msgr2=0x7f557c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f557c078b90 0x7f557c07b0a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f558000b980 tx=0x7f5580005db0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 msgr2=0x7f55900c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 0x7f55900c8b40 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f558400e7c0 tx=0x7f558400ec80 comp rx=0 tx=0).stop 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 shutdown_connections 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f55900d2fc0 0x7f55901571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f55900d09a0 0x7f55900c8b40 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.086 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f557c078b90 0x7f557c07b0a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 --2- 172.21.15.22:0/1934969236 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f55900b5fa0 0x7f55901e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 >> 172.21.15.22:0/1934969236 conn(0x7f55900d4860 msgr2=0x7f55900d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 shutdown_connections 2024-09-21T05:48:39.087 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.084+0000 7f5595bb6640 1 -- 172.21.15.22:0/1934969236 wait complete. 2024-09-21T05:48:39.097 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T05:48:39.097 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.215+0000 7fa677c7f640 1 -- 172.21.15.22:0/2276718365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 msgr2=0x7fa658004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.215+0000 7fa677c7f640 1 --2- 172.21.15.22:0/2276718365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa658004c10 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa664001690 tx=0x7fa658005830 comp rx=0 tx=0).stop 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.216+0000 7fa677c7f640 1 -- 172.21.15.22:0/2276718365 shutdown_connections 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.216+0000 7fa677c7f640 1 --2- 172.21.15.22:0/2276718365 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa658004c10 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.216+0000 7fa677c7f640 1 -- 172.21.15.22:0/2276718365 >> 172.21.15.22:0/2276718365 conn(0x7fa6700d4860 msgr2=0x7fa6700d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.216+0000 7fa677c7f640 1 -- 172.21.15.22:0/2276718365 shutdown_connections 2024-09-21T05:48:39.218 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.216+0000 7fa677c7f640 1 -- 172.21.15.22:0/2276718365 wait complete. 2024-09-21T05:48:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.217+0000 7fa677c7f640 1 Processor -- start 2024-09-21T05:48:39.219 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.217+0000 7fa677c7f640 1 -- start start 2024-09-21T05:48:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa677c7f640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa670160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa677c7f640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa670161470 0x7fa670157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.222 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa677c7f640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6701f9040 0x7fa670157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa677c7f640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa6700d3760 con 0x7fa658004800 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa677c7f640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa6700d35e0 con 0x7fa670161470 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa677c7f640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa6700ccd50 con 0x7fa6701f9040 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa670160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa670160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53390/0 (socket says 172.21.15.22:53390) 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 -- 172.21.15.22:0/2636808017 learned_addr learned my addr 172.21.15.22:0/2636808017 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa67621c640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6701f9040 0x7fa670157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa67521a640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa670161470 0x7fa670157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 -- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6701f9040 msgr2=0x7fa670157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6701f9040 0x7fa670157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 -- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa670161470 msgr2=0x7fa670157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa670161470 0x7fa670157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.223 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 -- 172.21.15.22:0/2636808017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6701580f0 con 0x7fa658004800 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa67621c640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa6701f9040 0x7fa670157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa675a1b640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa670160f30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa660006630 tx=0x7fa660003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.219+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660015070 con 0x7fa658004800 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.220+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa66000fd50 con 0x7fa658004800 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.220+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6701f5150 con 0x7fa658004800 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.220+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660011690 con 0x7fa658004800 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.220+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6701f5710 con 0x7fa658004800 2024-09-21T05:48:39.224 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.220+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa6701f5dd0 con 0x7fa658004800 2024-09-21T05:48:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.221+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa660011830 con 0x7fa658004800 2024-09-21T05:48:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa66e7fc640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa65c078b90 0x7fa65c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa66009c2b0 con 0x7fa658004800 2024-09-21T05:48:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa66e7fc640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7fa65c07ebe0 0x7fa65c081030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.225 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa65c081700 con 0x7fa65c07ebe0 2024-09-21T05:48:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fa66009c740 con 0x7fa658004800 2024-09-21T05:48:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa67521a640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa65c078b90 0x7fa65c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.222+0000 7fa67621c640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7fa65c07ebe0 0x7fa65c081030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.223+0000 7fa67621c640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7fa65c07ebe0 0x7fa65c081030 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.223+0000 7fa67521a640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa65c078b90 0x7fa65c07b0a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa664032000 tx=0x7fa664001190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.226 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.223+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== osd.3 v2:172.21.15.22:6808/3633643032 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa65c081700 con 0x7fa65c07ebe0 2024-09-21T05:48:39.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 --> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa6700b5930 con 0x7fa65c07ebe0 2024-09-21T05:48:39.238 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa66e7fc640 1 -- 172.21.15.22:0/2636808017 <== osd.3 v2:172.21.15.22:6808/3633643032 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa6700b5930 con 0x7fa65c07ebe0 2024-09-21T05:48:39.238 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:39.238 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:39.238 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:39.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7fa65c07ebe0 msgr2=0x7fa65c081030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:6808/3633643032,v1:172.21.15.22:6809/3633643032] conn(0x7fa65c07ebe0 0x7fa65c081030 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa65c078b90 msgr2=0x7fa65c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa65c078b90 0x7fa65c07b0a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fa664032000 tx=0x7fa664001190 comp rx=0 tx=0).stop 2024-09-21T05:48:39.239 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 msgr2=0x7fa670160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 --2- 172.21.15.22:0/2636808017 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa658004800 0x7fa670160f30 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa660006630 tx=0x7fa660003f60 comp rx=0 tx=0).stop 2024-09-21T05:48:39.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa67621c640 1 -- 172.21.15.22:0/2636808017 reap_dead start 2024-09-21T05:48:39.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 shutdown_connections 2024-09-21T05:48:39.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.236+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 >> 172.21.15.22:0/2636808017 conn(0x7fa6700d4860 msgr2=0x7fa6700d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.237+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 shutdown_connections 2024-09-21T05:48:39.240 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.237+0000 7fa677c7f640 1 -- 172.21.15.22:0/2636808017 wait complete. 2024-09-21T05:48:39.249 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T05:48:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 -- 172.21.15.22:0/2812438672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 msgr2=0x7f0e1c071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 --2- 172.21.15.22:0/2812438672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c071870 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c06d920 tx=0x7f0e1402aac0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 -- 172.21.15.22:0/2812438672 shutdown_connections 2024-09-21T05:48:39.367 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 --2- 172.21.15.22:0/2812438672 >> v2:172.21.15.134:3300/0 conn(0x7f0e1c0b56e0 0x7f0e1c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 --2- 172.21.15.22:0/2812438672 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c071870 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 --2- 172.21.15.22:0/2812438672 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e1c0b4ed0 0x7f0e1c142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 -- 172.21.15.22:0/2812438672 >> 172.21.15.22:0/2812438672 conn(0x7f0e1c09ba20 msgr2=0x7f0e1c0b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 -- 172.21.15.22:0/2812438672 shutdown_connections 2024-09-21T05:48:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.365+0000 7f0e23c38640 1 -- 172.21.15.22:0/2812438672 wait complete. 2024-09-21T05:48:39.368 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.367+0000 7f0e23c38640 1 Processor -- start 2024-09-21T05:48:39.369 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.367+0000 7f0e23c38640 1 -- start start 2024-09-21T05:48:39.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e23c38640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c0b4ed0 0x7f0e1c1f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e23c38640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e1c0b56e0 0x7f0e1c15c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.371 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e23c38640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e23c38640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c071db0 con 0x7f0e1c0b6030 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e23c38640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0b1d60 con 0x7f0e1c0b4ed0 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e23c38640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c071f30 con 0x7f0e1c0b56e0 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c0d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53402/0 (socket says 172.21.15.22:53402) 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 -- 172.21.15.22:0/3433123772 learned_addr learned my addr 172.21.15.22:0/3433123772 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e219d4640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c0b4ed0 0x7f0e1c1f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e211d3640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e1c0b56e0 0x7f0e1c15c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 -- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e1c0b56e0 msgr2=0x7f0e1c15c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e1c0b56e0 0x7f0e1c15c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 -- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c0b4ed0 msgr2=0x7f0e1c1f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c0b4ed0 0x7f0e1c1f64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.372 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.368+0000 7f0e221d5640 1 -- 172.21.15.22:0/3433123772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c1f5430 con 0x7f0e1c0b6030 2024-09-21T05:48:39.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.369+0000 7f0e221d5640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c0d1c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402b0b0 tx=0x7f0e14003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.369+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004250 con 0x7f0e1c0b6030 2024-09-21T05:48:39.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.369+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e140043f0 con 0x7f0e1c0b6030 2024-09-21T05:48:39.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.369+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14004820 con 0x7f0e1c0b6030 2024-09-21T05:48:39.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.369+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c1f5710 con 0x7f0e1c0b6030 2024-09-21T05:48:39.373 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.369+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c1f5cd0 con 0x7f0e1c0b6030 2024-09-21T05:48:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.370+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c0723e0 con 0x7f0e1c0b6030 2024-09-21T05:48:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.371+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0e14036400 con 0x7f0e1c0b6030 2024-09-21T05:48:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.371+0000 7f0e127fc640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0e00078b90 0x7f0e0007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.371+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f0e140baf40 con 0x7f0e1c0b6030 2024-09-21T05:48:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.371+0000 7f0e219d4640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0e00078b90 0x7f0e0007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.374 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.372+0000 7f0e219d4640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0e00078b90 0x7f0e0007b0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0e18005fd0 tx=0x7f0e1800b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.377 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.376+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e14032030 con 0x7f0e1c0b6030 2024-09-21T05:48:39.568 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.566+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f0e1c069d20 con 0x7f0e00078b90 2024-09-21T05:48:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.567+0000 7f0e127fc640 1 -- 172.21.15.22:0/3433123772 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0e1c069d20 con 0x7f0e00078b90 2024-09-21T05:48:39.569 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T05:48:39.571 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0e00078b90 msgr2=0x7f0e0007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0e00078b90 0x7f0e0007b0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0e18005fd0 tx=0x7f0e1800b260 comp rx=0 tx=0).stop 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 msgr2=0x7f0e1c0d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c0d1c40 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402b0b0 tx=0x7f0e14003f10 comp rx=0 tx=0).stop 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 shutdown_connections 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f0e1c0b6030 0x7f0e1c0d1c40 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f0e00078b90 0x7f0e0007b0a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f0e1c0b56e0 0x7f0e1c15c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 --2- 172.21.15.22:0/3433123772 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f0e1c0b4ed0 0x7f0e1c1f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 >> 172.21.15.22:0/3433123772 conn(0x7f0e1c09ba20 msgr2=0x7f0e1c0b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 shutdown_connections 2024-09-21T05:48:39.572 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.569+0000 7f0e23c38640 1 -- 172.21.15.22:0/3433123772 wait complete. 2024-09-21T05:48:39.582 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T05:48:39.583 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:39.703 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 -- 172.21.15.22:0/2893176515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b6920 msgr2=0x7f80340cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 --2- 172.21.15.22:0/2893176515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b6920 0x7f80340cd800 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f80340743c0 tx=0x7f802c02a840 comp rx=0 tx=0).stop 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 -- 172.21.15.22:0/2893176515 shutdown_connections 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 --2- 172.21.15.22:0/2893176515 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b6920 0x7f80340cd800 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 --2- 172.21.15.22:0/2893176515 >> v2:172.21.15.134:3300/0 conn(0x7f80340b7ab0 0x7f80340b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 -- 172.21.15.22:0/2893176515 >> 172.21.15.22:0/2893176515 conn(0x7f80340d2580 msgr2=0x7f80340d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 -- 172.21.15.22:0/2893176515 shutdown_connections 2024-09-21T05:48:39.704 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.701+0000 7f8039994640 1 -- 172.21.15.22:0/2893176515 wait complete. 2024-09-21T05:48:39.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.703+0000 7f8039994640 1 Processor -- start 2024-09-21T05:48:39.705 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.703+0000 7f8039994640 1 -- start start 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.704+0000 7f8039994640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80340b6920 0x7f8034147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.704+0000 7f8039994640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80340b7610 0x7f803415a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.704+0000 7f8039994640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b7ab0 0x7f80341f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.704+0000 7f8039994640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f80340ce0d0 con 0x7f80340b7ab0 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.704+0000 7f8039994640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f80340cdf50 con 0x7f80340b6920 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.704+0000 7f8039994640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f80340ce250 con 0x7f80340b7610 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b7ab0 0x7f80341f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b7ab0 0x7f80341f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53420/0 (socket says 172.21.15.22:53420) 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 -- 172.21.15.22:0/3257437243 learned_addr learned my addr 172.21.15.22:0/3257437243 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 -- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80340b7610 msgr2=0x7f803415a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.708 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f8032ffd640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80340b6920 0x7f8034147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80327fc640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80340b7610 0x7f803415a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f80340b7610 0x7f803415a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 -- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80340b6920 msgr2=0x7f8034147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80340b6920 0x7f8034147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 -- 172.21.15.22:0/3257437243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8034147ba0 con 0x7f80340b7ab0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f8032ffd640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f80340b6920 0x7f8034147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f80337fe640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b7ab0 0x7f80341f7370 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f802c02ae30 tx=0x7f802c032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c037050 con 0x7f80340b7ab0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f802c043070 con 0x7f80340b7ab0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803415a9e0 con 0x7f80340b7ab0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.705+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004550 con 0x7f80340b7ab0 2024-09-21T05:48:39.709 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.706+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8034148300 con 0x7f80340b7ab0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.706+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f80340cd0f0 con 0x7f80340b7ab0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.707+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f802c0046f0 con 0x7f80340b7ab0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f802bfff640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8014078bc0 0x7f801407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f802c0bc4a0 con 0x7f80340b7ab0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f802bfff640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f801407ec10 0x7f8014081060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f8032ffd640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8014078bc0 0x7f801407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f802c032760 con 0x7f801407ec10 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f802c0bc930 con 0x7f80340b7ab0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.708+0000 7f80327fc640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f801407ec10 0x7f8014081060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.709+0000 7f8032ffd640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8014078bc0 0x7f801407b0d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f801c001220 tx=0x7f801c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.711 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.709+0000 7f80327fc640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f801407ec10 0x7f8014081060 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.712 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.710+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== osd.4 v2:172.21.15.134:6826/3950009634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f802c032760 con 0x7f801407ec10 2024-09-21T05:48:39.724 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.722+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 --> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8034147dd0 con 0x7f801407ec10 2024-09-21T05:48:39.725 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f802bfff640 1 -- 172.21.15.22:0/3257437243 <== osd.4 v2:172.21.15.134:6826/3950009634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8034147dd0 con 0x7f801407ec10 2024-09-21T05:48:39.725 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:39.725 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:39.725 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:39.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f801407ec10 msgr2=0x7f8014081060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6826/3950009634,v1:172.21.15.134:6827/3950009634] conn(0x7f801407ec10 0x7f8014081060 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8014078bc0 msgr2=0x7f801407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.726 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8014078bc0 0x7f801407b0d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f801c001220 tx=0x7f801c009040 comp rx=0 tx=0).stop 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b7ab0 msgr2=0x7f80341f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 --2- 172.21.15.22:0/3257437243 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f80340b7ab0 0x7f80341f7370 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f802c02ae30 tx=0x7f802c032bc0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f80337fe640 1 -- 172.21.15.22:0/3257437243 reap_dead start 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 shutdown_connections 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.723+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 >> 172.21.15.22:0/3257437243 conn(0x7f80340d2580 msgr2=0x7f80340b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.724+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 shutdown_connections 2024-09-21T05:48:39.727 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.724+0000 7f8039994640 1 -- 172.21.15.22:0/3257437243 wait complete. 2024-09-21T05:48:39.736 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 -- 172.21.15.22:0/2202164303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7860003a50 msgr2=0x7f7860003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 --2- 172.21.15.22:0/2202164303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7860003a50 0x7f7860003ec0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f78680016c0 tx=0x7f7860005830 comp rx=0 tx=0).stop 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 -- 172.21.15.22:0/2202164303 shutdown_connections 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 --2- 172.21.15.22:0/2202164303 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7860003a50 0x7f7860003ec0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 -- 172.21.15.22:0/2202164303 >> 172.21.15.22:0/2202164303 conn(0x7f787809ba20 msgr2=0x7f78780b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 -- 172.21.15.22:0/2202164303 shutdown_connections 2024-09-21T05:48:39.854 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.852+0000 7f787c932640 1 -- 172.21.15.22:0/2202164303 wait complete. 2024-09-21T05:48:39.855 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.854+0000 7f787c932640 1 Processor -- start 2024-09-21T05:48:39.856 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.854+0000 7f787c932640 1 -- start start 2024-09-21T05:48:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787c932640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860003a50 0x7f787815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787c932640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 0x7f787813b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787c932640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78781f6f20 0x7f78781f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.858 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787c932640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f7878160a90 con 0x7f7878153700 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787c932640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f78780b1d60 con 0x7f7860003a50 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787c932640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f7878160c10 con 0x7f78781f6f20 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 0x7f787813b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 0x7f787813b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53444/0 (socket says 172.21.15.22:53444) 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 -- 172.21.15.22:0/918907846 learned_addr learned my addr 172.21.15.22:0/918907846 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f787659c640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860003a50 0x7f787815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7876d9d640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78781f6f20 0x7f78781f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 -- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78781f6f20 msgr2=0x7f78781f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78781f6f20 0x7f78781f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 -- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860003a50 msgr2=0x7f787815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860003a50 0x7f787815c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.855+0000 7f7875d9b640 1 -- 172.21.15.22:0/918907846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78781f7870 con 0x7f7878153700 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.856+0000 7f7875d9b640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 0x7f787813b790 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f786c011ae0 tx=0x7f786c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.859 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.856+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c00fcb0 con 0x7f7878153700 2024-09-21T05:48:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.856+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f786c00fe50 con 0x7f7878153700 2024-09-21T05:48:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.856+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787813c420 con 0x7f7878153700 2024-09-21T05:48:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.856+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c004280 con 0x7f7878153700 2024-09-21T05:48:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.856+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787813c6c0 con 0x7f7878153700 2024-09-21T05:48:39.860 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.857+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787815cc90 con 0x7f7878153700 2024-09-21T05:48:39.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.858+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f786c0035f0 con 0x7f7878153700 2024-09-21T05:48:39.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.858+0000 7f7866ffd640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7858078bc0 0x7f785807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:39.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.858+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f786c09b460 con 0x7f7878153700 2024-09-21T05:48:39.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.859+0000 7f787659c640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7858078bc0 0x7f785807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:39.861 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.859+0000 7f787659c640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7858078bc0 0x7f785807b0d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7868032000 tx=0x7f786802c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:39.864 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:39.863+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f787815cc90 con 0x7f7878153700 2024-09-21T05:48:40.052 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.050+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f787813f130 con 0x7f7858078bc0 2024-09-21T05:48:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.051+0000 7f7866ffd640 1 -- 172.21.15.22:0/918907846 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f787813f130 con 0x7f7858078bc0 2024-09-21T05:48:40.053 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.053+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7858078bc0 msgr2=0x7f785807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.053+0000 7f787c932640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7858078bc0 0x7f785807b0d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7868032000 tx=0x7f786802c040 comp rx=0 tx=0).stop 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.053+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 msgr2=0x7f787813b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.053+0000 7f787c932640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 0x7f787813b790 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f786c011ae0 tx=0x7f786c009f10 comp rx=0 tx=0).stop 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 shutdown_connections 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f7858078bc0 0x7f785807b0d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f78781f6f20 0x7f78781f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f7878153700 0x7f787813b790 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 --2- 172.21.15.22:0/918907846 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f7860003a50 0x7f787815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.056 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 >> 172.21.15.22:0/918907846 conn(0x7f787809ba20 msgr2=0x7f78780b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 shutdown_connections 2024-09-21T05:48:40.057 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.054+0000 7f787c932640 1 -- 172.21.15.22:0/918907846 wait complete. 2024-09-21T05:48:40.067 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T05:48:40.067 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1800916726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6006f4d0 msgr2=0x7ffb600b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 --2- 172.21.15.22:0/1800916726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6006f4d0 0x7ffb600b3ce0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005fd0 tx=0x7ffb5802a8b0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1800916726 shutdown_connections 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 --2- 172.21.15.22:0/1800916726 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb6006f4d0 0x7ffb600b3ce0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 --2- 172.21.15.22:0/1800916726 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb60160cd0 0x7ffb600cfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1800916726 >> 172.21.15.22:0/1800916726 conn(0x7ffb600b2320 msgr2=0x7ffb600b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1800916726 shutdown_connections 2024-09-21T05:48:40.188 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.186+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1800916726 wait complete. 2024-09-21T05:48:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.188+0000 7ffb64dc3640 1 Processor -- start 2024-09-21T05:48:40.190 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.188+0000 7ffb64dc3640 1 -- start start 2024-09-21T05:48:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb64dc3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6006f4d0 0x7ffb600aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb64dc3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb600afba0 0x7ffb601f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb64dc3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb60160cd0 0x7ffb60159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb64dc3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600b4740 con 0x7ffb6006f4d0 2024-09-21T05:48:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb64dc3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb600b45c0 con 0x7ffb60160cd0 2024-09-21T05:48:40.192 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb64dc3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb600b48c0 con 0x7ffb600afba0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6006f4d0 0x7ffb600aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb5e59c640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6006f4d0 0x7ffb600aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53446/0 (socket says 172.21.15.22:53446) 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb5e59c640 1 -- 172.21.15.22:0/1420720512 learned_addr learned my addr 172.21.15.22:0/1420720512 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb5ed9d640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb60160cd0 0x7ffb60159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.189+0000 7ffb55d9b640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb600afba0 0x7ffb601f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb5e59c640 1 -- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb600afba0 msgr2=0x7ffb601f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb5e59c640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffb600afba0 0x7ffb601f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb5e59c640 1 -- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb60160cd0 msgr2=0x7ffb60159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb5e59c640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb60160cd0 0x7ffb60159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb5e59c640 1 -- 172.21.15.22:0/1420720512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb60154150 con 0x7ffb6006f4d0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb5e59c640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6006f4d0 0x7ffb600aafa0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c460 tx=0x7ffb5000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50016020 con 0x7ffb6006f4d0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb50012940 con 0x7ffb6006f4d0 2024-09-21T05:48:40.193 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb60154430 con 0x7ffb6006f4d0 2024-09-21T05:48:40.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50012d90 con 0x7ffb6006f4d0 2024-09-21T05:48:40.194 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.190+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb601549f0 con 0x7ffb6006f4d0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.191+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb600b4de0 con 0x7ffb6006f4d0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.192+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffb50019020 con 0x7ffb6006f4d0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.192+0000 7ffb577fe640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffb40078b90 0x7ffb4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.192+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7ffb5009a6a0 con 0x7ffb6006f4d0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb577fe640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7ffb4007ebe0 0x7ffb40081030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb50022e50 con 0x7ffb4007ebe0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7ffb500a1050 con 0x7ffb6006f4d0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb55d9b640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffb40078b90 0x7ffb4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.195 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb5ed9d640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7ffb4007ebe0 0x7ffb40081030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb55d9b640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffb40078b90 0x7ffb4007b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffb60155490 tx=0x7ffb48005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.193+0000 7ffb5ed9d640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7ffb4007ebe0 0x7ffb40081030 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.196 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.194+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== osd.5 v2:172.21.15.134:6802/1222371224 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb50022e50 con 0x7ffb4007ebe0 2024-09-21T05:48:40.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.207+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 --> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffb600b3150 con 0x7ffb4007ebe0 2024-09-21T05:48:40.209 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.207+0000 7ffb577fe640 1 -- 172.21.15.22:0/1420720512 <== osd.5 v2:172.21.15.134:6802/1222371224 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffb600b3150 con 0x7ffb4007ebe0 2024-09-21T05:48:40.209 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:40.209 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:40.210 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:40.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7ffb4007ebe0 msgr2=0x7ffb40081030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.210 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6802/1222371224,v1:172.21.15.134:6803/1222371224] conn(0x7ffb4007ebe0 0x7ffb40081030 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffb40078b90 msgr2=0x7ffb4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffb40078b90 0x7ffb4007b0a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffb60155490 tx=0x7ffb48005f90 comp rx=0 tx=0).stop 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6006f4d0 msgr2=0x7ffb600aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 --2- 172.21.15.22:0/1420720512 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb6006f4d0 0x7ffb600aafa0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c460 tx=0x7ffb5000c920 comp rx=0 tx=0).stop 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb5ed9d640 1 -- 172.21.15.22:0/1420720512 reap_dead start 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 shutdown_connections 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 >> 172.21.15.22:0/1420720512 conn(0x7ffb600b2320 msgr2=0x7ffb600cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 shutdown_connections 2024-09-21T05:48:40.211 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.208+0000 7ffb64dc3640 1 -- 172.21.15.22:0/1420720512 wait complete. 2024-09-21T05:48:40.221 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.336+0000 7f8b53630640 1 -- 172.21.15.22:0/1753004617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 msgr2=0x7f8b4c0b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.336+0000 7f8b53630640 1 --2- 172.21.15.22:0/1753004617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c0b7ab0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c005fd0 tx=0x7f8b3c02aaa0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.336+0000 7f8b53630640 1 -- 172.21.15.22:0/1753004617 shutdown_connections 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.336+0000 7f8b53630640 1 --2- 172.21.15.22:0/1753004617 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c0b7ab0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.336+0000 7f8b53630640 1 -- 172.21.15.22:0/1753004617 >> 172.21.15.22:0/1753004617 conn(0x7f8b4c0d2580 msgr2=0x7f8b4c0d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.336+0000 7f8b53630640 1 -- 172.21.15.22:0/1753004617 shutdown_connections 2024-09-21T05:48:40.339 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.337+0000 7f8b53630640 1 -- 172.21.15.22:0/1753004617 wait complete. 2024-09-21T05:48:40.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.338+0000 7f8b53630640 1 Processor -- start 2024-09-21T05:48:40.340 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.338+0000 7f8b53630640 1 -- start start 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.339+0000 7f8b53630640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b53630640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b4c0cd0f0 0x7f8b4c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b53630640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b4c0cdc10 0x7f8b4c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b53630640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c0ce3b0 con 0x7f8b4c0b7610 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b53630640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c0ce230 con 0x7f8b4c0cd0f0 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b53630640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f8b4c168260 con 0x7f8b4c0cdc10 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53460/0 (socket says 172.21.15.22:53460) 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 -- 172.21.15.22:0/1893248647 learned_addr learned my addr 172.21.15.22:0/1893248647 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:40.343 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b50bcb640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b4c0cd0f0 0x7f8b4c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b51bcd640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b4c0cdc10 0x7f8b4c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 -- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b4c0cdc10 msgr2=0x7f8b4c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b4c0cdc10 0x7f8b4c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 -- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b4c0cd0f0 msgr2=0x7f8b4c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b4c0cd0f0 0x7f8b4c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 -- 172.21.15.22:0/1893248647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c1588e0 con 0x7f8b4c0b7610 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b51bcd640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b4c0cdc10 0x7f8b4c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b513cc640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c151530 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000f80 tx=0x7f8b3c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c044070 con 0x7f8b4c0b7610 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b3c004250 con 0x7f8b4c0b7610 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c158bc0 con 0x7f8b4c0b7610 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.340+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c004640 con 0x7f8b4c0b7610 2024-09-21T05:48:40.344 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.341+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c159180 con 0x7f8b4c0b7610 2024-09-21T05:48:40.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.342+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c0b7ab0 con 0x7f8b4c0b7610 2024-09-21T05:48:40.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.343+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b3c047070 con 0x7f8b4c0b7610 2024-09-21T05:48:40.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.343+0000 7f8b49ffb640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b38078bc0 0x7f8b3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.343+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f8b3c0bbaf0 con 0x7f8b4c0b7610 2024-09-21T05:48:40.345 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.343+0000 7f8b50bcb640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b38078bc0 0x7f8b3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.346 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.344+0000 7f8b50bcb640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b38078bc0 0x7f8b3807b0d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c157c90 tx=0x7f8b40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.349 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.348+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b3c032030 con 0x7f8b4c0b7610 2024-09-21T05:48:40.539 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.537+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8b4c0b27c0 con 0x7f8b38078bc0 2024-09-21T05:48:40.540 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.538+0000 7f8b49ffb640 1 -- 172.21.15.22:0/1893248647 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8b4c0b27c0 con 0x7f8b38078bc0 2024-09-21T05:48:40.540 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T05:48:40.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b38078bc0 msgr2=0x7f8b3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b38078bc0 0x7f8b3807b0d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c157c90 tx=0x7f8b40009040 comp rx=0 tx=0).stop 2024-09-21T05:48:40.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 msgr2=0x7f8b4c151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.542 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c151530 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000f80 tx=0x7f8b3c003f10 comp rx=0 tx=0).stop 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 shutdown_connections 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f8b38078bc0 0x7f8b3807b0d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f8b4c0cdc10 0x7f8b4c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f8b4c0cd0f0 0x7f8b4c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 --2- 172.21.15.22:0/1893248647 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f8b4c0b7610 0x7f8b4c151530 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 >> 172.21.15.22:0/1893248647 conn(0x7f8b4c0d2580 msgr2=0x7f8b4c0b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 shutdown_connections 2024-09-21T05:48:40.543 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.540+0000 7f8b53630640 1 -- 172.21.15.22:0/1893248647 wait complete. 2024-09-21T05:48:40.554 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T05:48:40.554 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.672+0000 7f396ece5640 1 -- 172.21.15.22:0/2615869796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3968073c50 msgr2=0x7f396806fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.672+0000 7f396ece5640 1 --2- 172.21.15.22:0/2615869796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3968073c50 0x7f396806fae0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f39680b6ab0 tx=0x7f395802a560 comp rx=0 tx=0).stop 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.672+0000 7f396ece5640 1 -- 172.21.15.22:0/2615869796 shutdown_connections 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.672+0000 7f396ece5640 1 --2- 172.21.15.22:0/2615869796 >> v2:172.21.15.134:3300/0 conn(0x7f3968070020 0x7f39680b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.672+0000 7f396ece5640 1 --2- 172.21.15.22:0/2615869796 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f3968073c50 0x7f396806fae0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.672+0000 7f396ece5640 1 -- 172.21.15.22:0/2615869796 >> 172.21.15.22:0/2615869796 conn(0x7f396806de80 msgr2=0x7f396806e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.673+0000 7f396ece5640 1 -- 172.21.15.22:0/2615869796 shutdown_connections 2024-09-21T05:48:40.675 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.673+0000 7f396ece5640 1 -- 172.21.15.22:0/2615869796 wait complete. 2024-09-21T05:48:40.679 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.674+0000 7f396ece5640 1 Processor -- start 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.674+0000 7f396ece5640 1 -- start start 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ece5640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3968070020 0x7f3968142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ece5640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39681526f0 0x7f396806cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ece5640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f396806d490 0x7f3968148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ece5640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f39680b5500 con 0x7f396806d490 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ece5640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f39680b5380 con 0x7f3968070020 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ece5640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f39680b5680 con 0x7f39681526f0 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f396806d490 0x7f3968148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f396806d490 0x7f3968148210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53486/0 (socket says 172.21.15.22:53486) 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 -- 172.21.15.22:0/1107296681 learned_addr learned my addr 172.21.15.22:0/1107296681 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396ca81640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3968070020 0x7f3968142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f395ffff640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39681526f0 0x7f396806cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.680 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 -- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39681526f0 msgr2=0x7f396806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f39681526f0 0x7f396806cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 -- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3968070020 msgr2=0x7f3968142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f3968070020 0x7f3968142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.676+0000 7f396d282640 1 -- 172.21.15.22:0/1107296681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39681434d0 con 0x7f396806d490 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.677+0000 7f396d282640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f396806d490 0x7f3968148210 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f396000cad0 tx=0x7f396000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.677+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396000ae20 con 0x7f396806d490 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.677+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f396000f450 con 0x7f396806d490 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.677+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39681437b0 con 0x7f396806d490 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.677+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396000f8c0 con 0x7f396806d490 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.677+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3968143d70 con 0x7f396806d490 2024-09-21T05:48:40.681 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.678+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39680b5bd0 con 0x7f396806d490 2024-09-21T05:48:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.679+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f39600213f0 con 0x7f396806d490 2024-09-21T05:48:40.682 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.679+0000 7f395d7fa640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3954078b90 0x7f395407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.679+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f396009b790 con 0x7f396806d490 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.679+0000 7f395d7fa640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f395407ebe0 0x7f3954081030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.679+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3954081700 con 0x7f395407ebe0 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.679+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7f396009bc20 con 0x7f396806d490 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.680+0000 7f396ca81640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3954078b90 0x7f395407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.680+0000 7f395ffff640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f395407ebe0 0x7f3954081030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.680+0000 7f396ca81640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3954078b90 0x7f395407b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3964000f60 tx=0x7f3964009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.681+0000 7f395ffff640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f395407ebe0 0x7f3954081030 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.683 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.681+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== osd.6 v2:172.21.15.134:6810/640262620 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3954081700 con 0x7f395407ebe0 2024-09-21T05:48:40.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.694+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 --> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f396806fb40 con 0x7f395407ebe0 2024-09-21T05:48:40.696 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f395d7fa640 1 -- 172.21.15.22:0/1107296681 <== osd.6 v2:172.21.15.134:6810/640262620 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f396806fb40 con 0x7f395407ebe0 2024-09-21T05:48:40.697 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:40.697 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:40.697 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f395407ebe0 msgr2=0x7f3954081030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6810/640262620,v1:172.21.15.134:6811/640262620] conn(0x7f395407ebe0 0x7f3954081030 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3954078b90 msgr2=0x7f395407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f3954078b90 0x7f395407b0a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3964000f60 tx=0x7f3964009040 comp rx=0 tx=0).stop 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f396806d490 msgr2=0x7f3968148210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 --2- 172.21.15.22:0/1107296681 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f396806d490 0x7f3968148210 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f396000cad0 tx=0x7f396000cf90 comp rx=0 tx=0).stop 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396d282640 1 -- 172.21.15.22:0/1107296681 reap_dead start 2024-09-21T05:48:40.698 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 shutdown_connections 2024-09-21T05:48:40.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 >> 172.21.15.22:0/1107296681 conn(0x7f396806de80 msgr2=0x7f39680b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 shutdown_connections 2024-09-21T05:48:40.699 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.695+0000 7f396ece5640 1 -- 172.21.15.22:0/1107296681 wait complete. 2024-09-21T05:48:40.708 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T05:48:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 -- 172.21.15.22:0/403794566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e015ebd0 msgr2=0x7f42e00d3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 --2- 172.21.15.22:0/403794566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e015ebd0 0x7f42e00d3ea0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009860 tx=0x7f42d002a530 comp rx=0 tx=0).stop 2024-09-21T05:48:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 -- 172.21.15.22:0/403794566 shutdown_connections 2024-09-21T05:48:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 --2- 172.21.15.22:0/403794566 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e015ebd0 0x7f42e00d3ea0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 -- 172.21.15.22:0/403794566 >> 172.21.15.22:0/403794566 conn(0x7f42e00d4820 msgr2=0x7f42e00d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:40.825 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 -- 172.21.15.22:0/403794566 shutdown_connections 2024-09-21T05:48:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.823+0000 7f42e60d3640 1 -- 172.21.15.22:0/403794566 wait complete. 2024-09-21T05:48:40.826 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.825+0000 7f42e60d3640 1 Processor -- start 2024-09-21T05:48:40.827 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.825+0000 7f42e60d3640 1 -- start start 2024-09-21T05:48:40.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42e60d3640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e008d3d0 0x7f42e00ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.829 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42e60d3640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 0x7f42e00cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42e60d3640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e01f8c10 0x7f42e015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42e60d3640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f42e00b85f0 con 0x7f42e015ebd0 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42e60d3640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f42e00b8470 con 0x7f42e01f8c10 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42e60d3640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f42e00b8770 con 0x7f42e008d3d0 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42deffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 0x7f42e00cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42deffd640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 0x7f42e00cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53496/0 (socket says 172.21.15.22:53496) 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.826+0000 7f42deffd640 1 -- 172.21.15.22:0/1564099030 learned_addr learned my addr 172.21.15.22:0/1564099030 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42df7fe640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e008d3d0 0x7f42e00ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42dffff640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e01f8c10 0x7f42e015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42deffd640 1 -- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e008d3d0 msgr2=0x7f42e00ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42deffd640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e008d3d0 0x7f42e00ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.830 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42deffd640 1 -- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e01f8c10 msgr2=0x7f42e015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42deffd640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e01f8c10 0x7f42e015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42deffd640 1 -- 172.21.15.22:0/1564099030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e00cf980 con 0x7f42e015ebd0 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42deffd640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 0x7f42e00cf440 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c4d0 tx=0x7f42d400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4016020 con 0x7f42e015ebd0 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42d4012920 con 0x7f42e015ebd0 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e01f93f0 con 0x7f42e015ebd0 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4012d30 con 0x7f42e015ebd0 2024-09-21T05:48:40.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.827+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e01f99b0 con 0x7f42e015ebd0 2024-09-21T05:48:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.828+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e015ae40 con 0x7f42e015ebd0 2024-09-21T05:48:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.829+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42d4019020 con 0x7f42e015ebd0 2024-09-21T05:48:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.829+0000 7f42c7fff640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f42cc078b90 0x7f42cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.830+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f42d409abe0 con 0x7f42e015ebd0 2024-09-21T05:48:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.830+0000 7f42df7fe640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f42cc078b90 0x7f42cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:40.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.831+0000 7f42df7fe640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f42cc078b90 0x7f42cc07b0a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f42d0034000 tx=0x7f42d0004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:40.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:40.834+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42d4020020 con 0x7f42e015ebd0 2024-09-21T05:48:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.020+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f42e00cae40 con 0x7f42cc078b90 2024-09-21T05:48:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.020+0000 7f42c7fff640 1 -- 172.21.15.22:0/1564099030 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f42d00041c0 con 0x7f42cc078b90 2024-09-21T05:48:41.022 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f42cc078b90 msgr2=0x7f42cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f42cc078b90 0x7f42cc07b0a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f42d0034000 tx=0x7f42d0004730 comp rx=0 tx=0).stop 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 msgr2=0x7f42e00cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 0x7f42e00cf440 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c4d0 tx=0x7f42d400c990 comp rx=0 tx=0).stop 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 shutdown_connections 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f42e01f8c10 0x7f42e015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f42e015ebd0 0x7f42e00cf440 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.022+0000 7f42e60d3640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f42cc078b90 0x7f42cc07b0a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.023+0000 7f42e60d3640 1 --2- 172.21.15.22:0/1564099030 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f42e008d3d0 0x7f42e00ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.023+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 >> 172.21.15.22:0/1564099030 conn(0x7f42e00d4820 msgr2=0x7f42e00cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.023+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 shutdown_connections 2024-09-21T05:48:41.025 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.023+0000 7f42e60d3640 1 -- 172.21.15.22:0/1564099030 wait complete. 2024-09-21T05:48:41.036 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T05:48:41.036 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T05:48:41.156 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 -- 172.21.15.22:0/126376549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0b5130 msgr2=0x7fa8bc0cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 --2- 172.21.15.22:0/126376549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0b5130 0x7fa8bc0cfae0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa8b0005fd0 tx=0x7fa8b002a670 comp rx=0 tx=0).stop 2024-09-21T05:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 -- 172.21.15.22:0/126376549 shutdown_connections 2024-09-21T05:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 --2- 172.21.15.22:0/126376549 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0b5130 0x7fa8bc0cfae0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 -- 172.21.15.22:0/126376549 >> 172.21.15.22:0/126376549 conn(0x7fa8bc0d1220 msgr2=0x7fa8bc0d1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 -- 172.21.15.22:0/126376549 shutdown_connections 2024-09-21T05:48:41.157 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.154+0000 7fa8c2789640 1 -- 172.21.15.22:0/126376549 wait complete. 2024-09-21T05:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.156+0000 7fa8c2789640 1 Processor -- start 2024-09-21T05:48:41.158 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.156+0000 7fa8c2789640 1 -- start start 2024-09-21T05:48:41.160 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.157+0000 7fa8c2789640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0b5130 0x7fa8bc0c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.157+0000 7fa8c2789640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc0b5d30 0x7fa8bc1f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c2789640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0d0020 0x7fa8bc0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c2789640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0d32a0 con 0x7fa8bc0d0020 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c2789640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0d3120 con 0x7fa8bc0b5130 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c2789640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc0d3420 con 0x7fa8bc0b5d30 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0d0020 0x7fa8bc0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0d0020 0x7fa8bc0cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53518/0 (socket says 172.21.15.22:53518) 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 -- 172.21.15.22:0/1477061198 learned_addr learned my addr 172.21.15.22:0/1477061198 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8bbfff640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0b5130 0x7fa8bc0c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8bb7fe640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc0b5d30 0x7fa8bc1f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 -- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc0b5d30 msgr2=0x7fa8bc1f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fa8bc0b5d30 0x7fa8bc1f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 -- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0b5130 msgr2=0x7fa8bc0c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.161 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fa8bc0b5130 0x7fa8bc0c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 -- 172.21.15.22:0/1477061198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc09f400 con 0x7fa8bc0d0020 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c0d26640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0d0020 0x7fa8bc0cad00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c700 tx=0x7fa8b400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4003b70 con 0x7fa8bc0d0020 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8b4003d10 con 0x7fa8bc0d0020 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc0c82a0 con 0x7fa8bc0d0020 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.158+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400aab0 con 0x7fa8bc0d0020 2024-09-21T05:48:41.162 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.159+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc09f6e0 con 0x7fa8bc0d0020 2024-09-21T05:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.159+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8bc0d24f0 con 0x7fa8bc0d0020 2024-09-21T05:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.160+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa8b400ac50 con 0x7fa8bc0d0020 2024-09-21T05:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8b8ff9640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8a8078b90 0x7fa8a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fa8b409b1b0 con 0x7fa8bc0d0020 2024-09-21T05:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8b8ff9640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7fa8a807ebe0 0x7fa8a8081030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.163 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8b4011e50 con 0x7fa8a807ebe0 2024-09-21T05:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_get_version_reply(handle=1 version=116) ==== 24+0+0 (secure 0 0 0) 0x7fa8b409b640 con 0x7fa8bc0d0020 2024-09-21T05:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8bbfff640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8a8078b90 0x7fa8a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.161+0000 7fa8bb7fe640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7fa8a807ebe0 0x7fa8a8081030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.162+0000 7fa8bbfff640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8a8078b90 0x7fa8a807b0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac000f60 tx=0x7fa8ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.162+0000 7fa8bb7fe640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7fa8a807ebe0 0x7fa8a8081030 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.164 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.163+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== osd.7 v2:172.21.15.134:6818/1266427391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8b4011e50 con 0x7fa8a807ebe0 2024-09-21T05:48:41.177 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.175+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 --> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa8bc09f1b0 con 0x7fa8a807ebe0 2024-09-21T05:48:41.178 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8b8ff9640 1 -- 172.21.15.22:0/1477061198 <== osd.7 v2:172.21.15.134:6818/1266427391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa8bc09f1b0 con 0x7fa8a807ebe0 2024-09-21T05:48:41.178 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-09-21T05:48:41.178 INFO:teuthology.orchestra.run.smithi022.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T05:48:41.178 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7fa8a807ebe0 msgr2=0x7fa8a8081030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6818/1266427391,v1:172.21.15.134:6819/1266427391] conn(0x7fa8a807ebe0 0x7fa8a8081030 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8a8078b90 msgr2=0x7fa8a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fa8a8078b90 0x7fa8a807b0a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac000f60 tx=0x7fa8ac009040 comp rx=0 tx=0).stop 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0d0020 msgr2=0x7fa8bc0cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 --2- 172.21.15.22:0/1477061198 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fa8bc0d0020 0x7fa8bc0cad00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c700 tx=0x7fa8b400cbc0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.179 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c0d26640 1 -- 172.21.15.22:0/1477061198 reap_dead start 2024-09-21T05:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 shutdown_connections 2024-09-21T05:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 >> 172.21.15.22:0/1477061198 conn(0x7fa8bc0d1220 msgr2=0x7fa8bc161c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 shutdown_connections 2024-09-21T05:48:41.180 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.176+0000 7fa8c2789640 1 -- 172.21.15.22:0/1477061198 wait complete. 2024-09-21T05:48:41.189 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb0680e640 1 -- 172.21.15.22:0/2196389868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaec004800 msgr2=0x7ffaec004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb0680e640 1 --2- 172.21.15.22:0/2196389868 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffaec004800 0x7ffaec004c10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ffae800c260 tx=0x7ffaec005830 comp rx=0 tx=0).stop 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb04dab640 1 -- 172.21.15.22:0/2196389868 reap_dead start 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb0680e640 1 -- 172.21.15.22:0/2196389868 shutdown_connections 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb0680e640 1 -- 172.21.15.22:0/2196389868 >> 172.21.15.22:0/2196389868 conn(0x7ffb000d4860 msgr2=0x7ffb000d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb0680e640 1 -- 172.21.15.22:0/2196389868 shutdown_connections 2024-09-21T05:48:41.307 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.305+0000 7ffb0680e640 1 -- 172.21.15.22:0/2196389868 wait complete. 2024-09-21T05:48:41.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.306+0000 7ffb0680e640 1 Processor -- start 2024-09-21T05:48:41.308 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.307+0000 7ffb0680e640 1 -- start start 2024-09-21T05:48:41.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb0680e640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaec004800 0x7ffb00151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb0680e640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b9790 0x7ffb000b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb0680e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 0x7ffb00155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb0680e640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000b6830 con 0x7ffb000b8c20 2024-09-21T05:48:41.311 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb0680e640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7ffb000b66b0 con 0x7ffb000b9790 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb0680e640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7ffb000d33d0 con 0x7ffaec004800 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 0x7ffb00155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 0x7ffb00155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53530/0 (socket says 172.21.15.22:53530) 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 -- 172.21.15.22:0/1308312614 learned_addr learned my addr 172.21.15.22:0/1308312614 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffaf7fff640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b9790 0x7ffb000b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffafffff640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaec004800 0x7ffb00151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 -- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaec004800 msgr2=0x7ffb00151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaec004800 0x7ffb00151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 -- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b9790 msgr2=0x7ffb000b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b9790 0x7ffb000b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.308+0000 7ffb04dab640 1 -- 172.21.15.22:0/1308312614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb000b9030 con 0x7ffb000b8c20 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.309+0000 7ffb04dab640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 0x7ffb00155c90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ffaec004780 tx=0x7ffaf800fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.309+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf801b070 con 0x7ffb000b8c20 2024-09-21T05:48:41.312 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.309+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf800de20 con 0x7ffb000b8c20 2024-09-21T05:48:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.309+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb000ceb50 con 0x7ffb000b8c20 2024-09-21T05:48:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.309+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8003bd0 con 0x7ffb000b8c20 2024-09-21T05:48:41.313 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.309+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb000cdd90 con 0x7ffb000b8c20 2024-09-21T05:48:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.310+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb000b5550 con 0x7ffb000b8c20 2024-09-21T05:48:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.311+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffaf801e070 con 0x7ffb000b8c20 2024-09-21T05:48:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.311+0000 7ffafd7fa640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffae0078bc0 0x7ffae007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.311+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7ffaf809a690 con 0x7ffb000b8c20 2024-09-21T05:48:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.311+0000 7ffafffff640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffae0078bc0 0x7ffae007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.314 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.312+0000 7ffafffff640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffae0078bc0 0x7ffae007b0d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0003700 tx=0x7ffaf000b260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.316+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb000b9030 con 0x7ffb000b8c20 2024-09-21T05:48:41.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.506+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7ffb001622b0 con 0x7ffae0078bc0 2024-09-21T05:48:41.508 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.506+0000 7ffafd7fa640 1 -- 172.21.15.22:0/1308312614 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb001622b0 con 0x7ffae0078bc0 2024-09-21T05:48:41.508 INFO:teuthology.orchestra.run.smithi022.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T05:48:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffae0078bc0 msgr2=0x7ffae007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffae0078bc0 0x7ffae007b0d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0003700 tx=0x7ffaf000b260 comp rx=0 tx=0).stop 2024-09-21T05:48:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 msgr2=0x7ffb00155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 0x7ffb00155c90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ffaec004780 tx=0x7ffaf800fde0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 shutdown_connections 2024-09-21T05:48:41.511 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7ffb000b8c20 0x7ffb00155c90 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7ffb000b9790 0x7ffb000b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7ffae0078bc0 0x7ffae007b0d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.508+0000 7ffb0680e640 1 --2- 172.21.15.22:0/1308312614 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7ffaec004800 0x7ffb00151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.509+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 >> 172.21.15.22:0/1308312614 conn(0x7ffb000d4860 msgr2=0x7ffb00161ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.509+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 shutdown_connections 2024-09-21T05:48:41.512 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.509+0000 7ffb0680e640 1 -- 172.21.15.22:0/1308312614 wait complete. 2024-09-21T05:48:41.521 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:48:41.637 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 -- 172.21.15.22:0/2662404113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c8160680 msgr2=0x7f41c8160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 --2- 172.21.15.22:0/2662404113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c8160680 0x7f41c8160a90 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009be0 tx=0x7f41c002a960 comp rx=0 tx=0).stop 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 -- 172.21.15.22:0/2662404113 shutdown_connections 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 --2- 172.21.15.22:0/2662404113 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c8160680 0x7f41c8160a90 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 --2- 172.21.15.22:0/2662404113 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41c8142f40 0x7f41c80b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 -- 172.21.15.22:0/2662404113 >> 172.21.15.22:0/2662404113 conn(0x7f41c809ba20 msgr2=0x7f41c80b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 -- 172.21.15.22:0/2662404113 shutdown_connections 2024-09-21T05:48:41.638 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.635+0000 7f41cedad640 1 -- 172.21.15.22:0/2662404113 wait complete. 2024-09-21T05:48:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.637+0000 7f41cedad640 1 Processor -- start 2024-09-21T05:48:41.639 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.637+0000 7f41cedad640 1 -- start start 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41cedad640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 0x7f41c813d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41cedad640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41c8142f40 0x7f41c815e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41cedad640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c8160680 0x7f41c815e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41cedad640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b1cd0 con 0x7f41c80b4ed0 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41cedad640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80b1b50 con 0x7f41c8160680 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41cedad640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f41c8160fd0 con 0x7f41c8142f40 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.638+0000 7f41ccb49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 0x7f41c813d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 0x7f41c813d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:53548/0 (socket says 172.21.15.22:53548) 2024-09-21T05:48:41.642 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 -- 172.21.15.22:0/2325971162 learned_addr learned my addr 172.21.15.22:0/2325971162 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41cd34a640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c8160680 0x7f41c815e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41c7fff640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41c8142f40 0x7f41c815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 -- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41c8142f40 msgr2=0x7f41c815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41c8142f40 0x7f41c815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 -- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c8160680 msgr2=0x7f41c815e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c8160680 0x7f41c815e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 -- 172.21.15.22:0/2325971162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c80d19f0 con 0x7f41c80b4ed0 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41ccb49640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 0x7f41c813d9a0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00c3c0 tx=0x7f41bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc017070 con 0x7f41c80b4ed0 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41bc0035f0 con 0x7f41c80b4ed0 2024-09-21T05:48:41.643 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c80d1ca0 con 0x7f41c80b4ed0 2024-09-21T05:48:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc003a80 con 0x7f41c80b4ed0 2024-09-21T05:48:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.639+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c813e6c0 con 0x7f41c80b4ed0 2024-09-21T05:48:41.644 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.640+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c8161560 con 0x7f41c80b4ed0 2024-09-21T05:48:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.641+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f41bc01a070 con 0x7f41c80b4ed0 2024-09-21T05:48:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.642+0000 7f41c57fa640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f41ac078b90 0x7f41ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:48:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.642+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f41bc099d50 con 0x7f41c80b4ed0 2024-09-21T05:48:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.642+0000 7f41c7fff640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f41ac078b90 0x7f41ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:48:41.645 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.643+0000 7f41c7fff640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f41ac078b90 0x7f41ac07b0a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f41c8051210 tx=0x7f41b400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:48:41.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.646+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41bc09d030 con 0x7f41c80b4ed0 2024-09-21T05:48:41.831 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.829+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41c815cc80 con 0x7f41ac078b90 2024-09-21T05:48:41.832 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.830+0000 7f41c57fa640 1 -- 172.21.15.22:0/2325971162 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13493 (secure 0 0 0) 0x7f41c815cc80 con 0x7f41ac078b90 2024-09-21T05:48:41.832 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:48:41.832 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f41ac078b90 msgr2=0x7f41ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f41ac078b90 0x7f41ac07b0a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f41c8051210 tx=0x7f41b400b2e0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 msgr2=0x7f41c813d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 0x7f41c813d9a0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00c3c0 tx=0x7f41bc00c880 comp rx=0 tx=0).stop 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 shutdown_connections 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f41c8160680 0x7f41c815e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f41ac078b90 0x7f41ac07b0a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f41c8142f40 0x7f41c815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 --2- 172.21.15.22:0/2325971162 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f41c80b4ed0 0x7f41c813d9a0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 >> 172.21.15.22:0/2325971162 conn(0x7f41c809ba20 msgr2=0x7f41c80b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:48:41.835 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 shutdown_connections 2024-09-21T05:48:41.836 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:48:41.832+0000 7f41cedad640 1 -- 172.21.15.22:0/2325971162 wait complete. 2024-09-21T05:48:41.845 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-21T05:48:39.889997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":286820,"kb_used_data":16496,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687900,"statfs":{"total":773094113280,"available":772800409600,"internally_reserved":0,"allocated":16891904,"data_stored":10376646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390540,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-172,"num_write_kb":-8298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002424"},"pg_stats":[{"pgid":"1.0","version":"16'72","reported_seq":285,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T05:48:34.616575+0000","last_change":"2024-09-21T05:39:12.143435+0000","last_active":"2024-09-21T05:48:34.616575+0000","last_peered":"2024-09-21T05:48:34.616575+0000","last_clean":"2024-09-21T05:48:34.616575+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:48:34.616575+0000","last_undegraded":"2024-09-21T05:48:34.616575+0000","last_fullsized":"2024-09-21T05:48:34.616575+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T05:39:10.745194+0000","last_clean_scrub_stamp":"2024-09-21T05:39:10.745194+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:18:35.113953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33180,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338660,"statfs":{"total":96636764160,"available":96602787840,"internally_reserved":0,"allocated":2035712,"data_stored":1217010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32600,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542260,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32592,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1433600,"data_stored":620883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542259,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36756,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":45000,"kb_used_data":5488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94326840,"statfs":{"total":96636764160,"available":96590684160,"internally_reserved":0,"allocated":5619712,"data_stored":4815175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37336,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334504,"statfs":{"total":96636764160,"available":96598532096,"internally_reserved":0,"allocated":2035712,"data_stored":1217010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:48:41.847 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-21T05:39:10.745194+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=5, tm_min=39, tm_sec=10, tm_wday=5, tm_yday=265, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=21, tm_hour=5, tm_min=48, tm_sec=36, tm_wday=5, tm_yday=265, tm_isdst=0) 2024-09-21T05:48:41.848 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-21T05:48:53.038 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:48:53.040 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:49:01.848 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 -- 172.21.15.22:0/3708394456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea015f340 msgr2=0x7faea006f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 --2- 172.21.15.22:0/3708394456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea015f340 0x7faea006f4d0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fae90005fd0 tx=0x7fae9002aa50 comp rx=0 tx=0).stop 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 -- 172.21.15.22:0/3708394456 shutdown_connections 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 --2- 172.21.15.22:0/3708394456 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea015f340 0x7faea006f4d0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 --2- 172.21.15.22:0/3708394456 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faea015bd10 0x7faea0073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 --2- 172.21.15.22:0/3708394456 >> v2:172.21.15.134:3300/0 conn(0x7faea00cd130 0x7faea00cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 -- 172.21.15.22:0/3708394456 >> 172.21.15.22:0/3708394456 conn(0x7faea00b2320 msgr2=0x7faea00b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 -- 172.21.15.22:0/3708394456 shutdown_connections 2024-09-21T05:49:01.984 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.981+0000 7faea7bda640 1 -- 172.21.15.22:0/3708394456 wait complete. 2024-09-21T05:49:01.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.983+0000 7faea7bda640 1 Processor -- start 2024-09-21T05:49:01.985 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.983+0000 7faea7bda640 1 -- start start 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.984+0000 7faea7bda640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 0x7faea01f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea7bda640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faea015bd10 0x7faea014bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea7bda640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea015f340 0x7faea00ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea7bda640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7faea00b4d90 con 0x7faea00cd130 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea7bda640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7faea00b4c10 con 0x7faea015f340 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea7bda640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7faea00b4f10 con 0x7faea015bd10 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 0x7faea01f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 0x7faea01f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46776/0 (socket says 172.21.15.22:46776) 2024-09-21T05:49:01.988 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 -- 172.21.15.22:0/1518941807 learned_addr learned my addr 172.21.15.22:0/1518941807 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea6177640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea015f340 0x7faea00ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5175640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faea015bd10 0x7faea014bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 -- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faea015bd10 msgr2=0x7faea014bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faea015bd10 0x7faea014bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 -- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea015f340 msgr2=0x7faea00ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea015f340 0x7faea00ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 -- 172.21.15.22:0/1518941807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea00b1c50 con 0x7faea00cd130 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea5976640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 0x7faea01f9c00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d770 tx=0x7fae9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98017070 con 0x7faea00cd130 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae98009d20 con 0x7faea00cd130 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980106c0 con 0x7faea00cd130 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.985+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea00b1f30 con 0x7faea00cd130 2024-09-21T05:49:01.989 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.986+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea0160520 con 0x7faea00cd130 2024-09-21T05:49:01.990 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.986+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea0073740 con 0x7faea00cd130 2024-09-21T05:49:01.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.987+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fae9801a070 con 0x7faea00cd130 2024-09-21T05:49:01.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.988+0000 7fae9e7fc640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae8c078b90 0x7fae8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:01.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.988+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fae9809ab30 con 0x7faea00cd130 2024-09-21T05:49:01.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.988+0000 7faea5175640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae8c078b90 0x7fae8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:01.991 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.989+0000 7faea5175640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae8c078b90 0x7fae8c07b0a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faea00b1990 tx=0x7fae9400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:01.995 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:01.993+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae98062bf0 con 0x7faea00cd130 2024-09-21T05:49:02.182 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.180+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 --> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faea0160780 con 0x7fae8c078b90 2024-09-21T05:49:02.183 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.182+0000 7fae9e7fc640 1 -- 172.21.15.22:0/1518941807 <== mgr.4100 v2:172.21.15.134:6800/3406357127 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7faea0160780 con 0x7fae8c078b90 2024-09-21T05:49:02.184 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:49:02.184 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae8c078b90 msgr2=0x7fae8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae8c078b90 0x7fae8c07b0a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faea00b1990 tx=0x7fae9400b2e0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 msgr2=0x7faea01f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 0x7faea01f9c00 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d770 tx=0x7fae9800dc30 comp rx=0 tx=0).stop 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 shutdown_connections 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7faea015f340 0x7faea00ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fae8c078b90 0x7fae8c07b0a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7faea015bd10 0x7faea014bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 --2- 172.21.15.22:0/1518941807 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7faea00cd130 0x7faea01f9c00 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 >> 172.21.15.22:0/1518941807 conn(0x7faea00b2320 msgr2=0x7faea006e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 shutdown_connections 2024-09-21T05:49:02.187 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.184+0000 7faea7bda640 1 -- 172.21.15.22:0/1518941807 wait complete. 2024-09-21T05:49:02.197 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-21T05:49:01.895031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":282728,"kb_used_data":12424,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754691992,"statfs":{"total":773094113280,"available":772804599808,"internally_reserved":0,"allocated":12722176,"data_stored":6193872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002760"},"pg_stats":[{"pgid":"1.0","version":"116'74","reported_seq":295,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-21T05:48:42.437249+0000","last_change":"2024-09-21T05:48:42.437249+0000","last_active":"2024-09-21T05:48:42.437249+0000","last_peered":"2024-09-21T05:48:42.437249+0000","last_clean":"2024-09-21T05:48:42.437249+0000","last_became_active":"2024-09-21T05:39:12.142262+0000","last_became_peered":"2024-09-21T05:39:12.142262+0000","last_unstale":"2024-09-21T05:48:42.437249+0000","last_undegraded":"2024-09-21T05:48:42.437249+0000","last_fullsized":"2024-09-21T05:48:42.437249+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'74","last_scrub_stamp":"2024-09-21T05:48:42.437114+0000","last_deep_scrub":"116'74","last_deep_scrub_stamp":"2024-09-21T05:48:42.437114+0000","last_clean_scrub_stamp":"2024-09-21T05:48:42.437114+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:15:56.464663+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509562,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33172,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338668,"statfs":{"total":96636764160,"available":96602796032,"internally_reserved":0,"allocated":2035712,"data_stored":1217010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32600,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32596,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36756,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36760,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335080,"statfs":{"total":96636764160,"available":96599121920,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40916,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330924,"statfs":{"total":96636764160,"available":96594866176,"internally_reserved":0,"allocated":1441792,"data_stored":626642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542265,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37332,"kb_used_data":1988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334508,"statfs":{"total":96636764160,"available":96598536192,"internally_reserved":0,"allocated":2035712,"data_stored":1217010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T05:49:02.199 DEBUG:teuthology.orchestra.run.smithi022:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T05:49:02.315 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 -- 172.21.15.22:0/3900710420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 msgr2=0x7f53280b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 --2- 172.21.15.22:0/3900710420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 0x7f53280b5550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f531c009830 tx=0x7f531c02a500 comp rx=0 tx=0).stop 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 -- 172.21.15.22:0/3900710420 shutdown_connections 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 --2- 172.21.15.22:0/3900710420 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53280d2fc0 0x7f53280d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 --2- 172.21.15.22:0/3900710420 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 0x7f53280b5550 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 -- 172.21.15.22:0/3900710420 >> 172.21.15.22:0/3900710420 conn(0x7f53280d4860 msgr2=0x7f53280d4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 -- 172.21.15.22:0/3900710420 shutdown_connections 2024-09-21T05:49:02.316 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.313+0000 7f532eaa2640 1 -- 172.21.15.22:0/3900710420 wait complete. 2024-09-21T05:49:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.315+0000 7f532eaa2640 1 Processor -- start 2024-09-21T05:49:02.317 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.315+0000 7f532eaa2640 1 -- start start 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532eaa2640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 0x7f532815efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532eaa2640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 0x7f53280ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532eaa2640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53280d2fc0 0x7f532806d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532eaa2640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7f53280cf4d0 con 0x7f53280b5fa0 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532eaa2640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7f53280cf350 con 0x7f53280d09a0 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532eaa2640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7f53280cf650 con 0x7f53280d2fc0 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532c83e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 0x7f532815efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532c83e640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 0x7f532815efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46792/0 (socket says 172.21.15.22:46792) 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.316+0000 7f532c83e640 1 -- 172.21.15.22:0/2885004006 learned_addr learned my addr 172.21.15.22:0/2885004006 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532d03f640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53280d2fc0 0x7f532806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f5327fff640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 0x7f53280ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.320 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532c83e640 1 -- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53280d2fc0 msgr2=0x7f532806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532c83e640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53280d2fc0 0x7f532806d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532c83e640 1 -- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 msgr2=0x7f53280ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532c83e640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 0x7f53280ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532c83e640 1 -- 172.21.15.22:0/2885004006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280cbbf0 con 0x7f53280b5fa0 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532c83e640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 0x7f532815efc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f531c009950 tx=0x7f531c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c031d60 con 0x7f53280b5fa0 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f531c033e10 con 0x7f53280b5fa0 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53280cbed0 con 0x7f53280b5fa0 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c005260 con 0x7f53280b5fa0 2024-09-21T05:49:02.321 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.317+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53280cc490 con 0x7f53280b5fa0 2024-09-21T05:49:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.318+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53280b5550 con 0x7f53280b5fa0 2024-09-21T05:49:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.319+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f531c005400 con 0x7f53280b5fa0 2024-09-21T05:49:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.319+0000 7f53257fa640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f530c078b90 0x7f530c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.320+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7f531c0bbcc0 con 0x7f53280b5fa0 2024-09-21T05:49:02.322 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.320+0000 7f5327fff640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f530c078b90 0x7f530c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.323 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.321+0000 7f5327fff640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f530c078b90 0x7f530c07b0a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f53280c98f0 tx=0x7f5314005f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:02.326 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.324+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f531c03f030 con 0x7f53280b5fa0 2024-09-21T05:49:02.513 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.510+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f53280d14c0 con 0x7f53280b5fa0 2024-09-21T05:49:02.517 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.515+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v46)=0 v46) ==== 123+0+0 (secure 0 0 0) 0x7f531c083c80 con 0x7f53280b5fa0 2024-09-21T05:49:02.518 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.516+0000 7f53257fa640 1 -- 172.21.15.22:0/2885004006 <== mon.0 v2:172.21.15.22:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f531c0c0070 con 0x7f53280b5fa0 2024-09-21T05:49:02.520 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f530c078b90 msgr2=0x7f530c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f530c078b90 0x7f530c07b0a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f53280c98f0 tx=0x7f5314005f90 comp rx=0 tx=0).stop 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 msgr2=0x7f532815efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 0x7f532815efc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f531c009950 tx=0x7f531c033c20 comp rx=0 tx=0).stop 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 shutdown_connections 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7f530c078b90 0x7f530c07b0a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7f53280d2fc0 0x7f532806d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7f53280d09a0 0x7f53280ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 --2- 172.21.15.22:0/2885004006 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7f53280b5fa0 0x7f532815efc0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 >> 172.21.15.22:0/2885004006 conn(0x7f53280d4860 msgr2=0x7f53280d1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 shutdown_connections 2024-09-21T05:49:02.521 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.518+0000 7f532eaa2640 1 -- 172.21.15.22:0/2885004006 wait complete. 2024-09-21T05:49:02.531 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T05:49:02.531 DEBUG:teuthology.orchestra.run.smithi022:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 -- 172.21.15.22:0/3678412193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b6200 msgr2=0x7fab64160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 --2- 172.21.15.22:0/3678412193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b6200 0x7fab64160680 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009830 tx=0x7fab4c02a5c0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 -- 172.21.15.22:0/3678412193 shutdown_connections 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 --2- 172.21.15.22:0/3678412193 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b6200 0x7fab64160680 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 --2- 172.21.15.22:0/3678412193 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab640b4ed0 0x7fab64142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 -- 172.21.15.22:0/3678412193 >> 172.21.15.22:0/3678412193 conn(0x7fab6409ba20 msgr2=0x7fab640b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 -- 172.21.15.22:0/3678412193 shutdown_connections 2024-09-21T05:49:02.648 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.645+0000 7fab6afa1640 1 -- 172.21.15.22:0/3678412193 wait complete. 2024-09-21T05:49:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.647+0000 7fab6afa1640 1 Processor -- start 2024-09-21T05:49:02.649 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.647+0000 7fab6afa1640 1 -- start start 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 0x7fab641f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 --2- >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab640b56e0 0x7fab64143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 --2- >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab640b6200 0x7fab64143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 -- --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_getmap magic: 0 -- 0x7fab64160d40 con 0x7fab640b4ed0 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 -- --> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] -- mon_getmap magic: 0 -- 0x7fab64160bc0 con 0x7fab640b56e0 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 -- --> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] -- mon_getmap magic: 0 -- 0x7fab64160ec0 con 0x7fab640b6200 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 0x7fab641f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 --2- >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 0x7fab641f2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.22:3300/0 says I am v2:172.21.15.22:46806/0 (socket says 172.21.15.22:46806) 2024-09-21T05:49:02.652 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 -- 172.21.15.22:0/505694734 learned_addr learned my addr 172.21.15.22:0/505694734 (peer_addr_for_me v2:172.21.15.22:0/0) 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6953e640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab640b6200 0x7fab64143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab63fff640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab640b56e0 0x7fab64143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 -- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab640b6200 msgr2=0x7fab64143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab640b6200 0x7fab64143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 -- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab640b56e0 msgr2=0x7fab64143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab640b56e0 0x7fab64143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 -- 172.21.15.22:0/505694734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab640d2390 con 0x7fab640b4ed0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab63fff640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab640b56e0 0x7fab64143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab68d3d640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 0x7fab641f2b90 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c9a0 tx=0x7fab5c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c007bb0 con 0x7fab640b4ed0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fab5c00f450 con 0x7fab640b4ed0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.649+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab641f30d0 con 0x7fab640b4ed0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.650+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c007e60 con 0x7fab640b4ed0 2024-09-21T05:49:02.653 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.650+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6414f720 con 0x7fab640b4ed0 2024-09-21T05:49:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.651+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab640d0c50 con 0x7fab640b4ed0 2024-09-21T05:49:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.651+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fab5c00f8c0 con 0x7fab640b4ed0 2024-09-21T05:49:02.654 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.652+0000 7fab617fa640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab44078b90 0x7fab4407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T05:49:02.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.652+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5115+0+0 (secure 0 0 0) 0x7fab5c09b290 con 0x7fab640b4ed0 2024-09-21T05:49:02.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.652+0000 7fab63fff640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab44078b90 0x7fab4407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T05:49:02.655 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.653+0000 7fab63fff640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab44078b90 0x7fab4407b0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fab641f7910 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T05:49:02.658 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.657+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fab5c014030 con 0x7fab640b4ed0 2024-09-21T05:49:02.884 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.883+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 --> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab64161470 con 0x7fab640b4ed0 2024-09-21T05:49:02.885 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.883+0000 7fab617fa640 1 -- 172.21.15.22:0/505694734 <== mon.0 v2:172.21.15.22:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) ==== 95+0+3917 (secure 0 0 0) 0x7fab5c0639b0 con 0x7fab640b4ed0 2024-09-21T05:49:02.885 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-09-21T05:49:02.885 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":116,"btime":"2024-09-21T05:48:31:425002+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6834/3282167939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6832","nonce":3282167939},{"type":"v1","addr":"172.21.15.22:6834","nonce":3282167939}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":7414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6836/496932888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":496932888},{"type":"v1","addr":"172.21.15.134:6836","nonce":496932888}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7555,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.134:6837/3257952726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6835","nonce":3257952726},{"type":"v1","addr":"172.21.15.134:6837","nonce":3257952726}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103},{"gid":8571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.22:6835/2396588720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6833","nonce":2396588720},{"type":"v1","addr":"172.21.15.22:6835","nonce":2396588720}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[]} 2024-09-21T05:49:02.886 INFO:teuthology.orchestra.run.smithi022.stderr:dumped fsmap epoch 116 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.885+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab44078b90 msgr2=0x7fab4407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.885+0000 7fab6afa1640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab44078b90 0x7fab4407b0a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fab641f7910 tx=0x7fab54009040 comp rx=0 tx=0).stop 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.885+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 msgr2=0x7fab641f2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.885+0000 7fab6afa1640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 0x7fab641f2b90 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c9a0 tx=0x7fab5c00ce60 comp rx=0 tx=0).stop 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 shutdown_connections 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:6800/3406357127,v1:172.21.15.134:6801/3406357127] conn(0x7fab44078b90 0x7fab4407b0a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3301/0,v1:172.21.15.134:6790/0] conn(0x7fab640b6200 0x7fab64143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] conn(0x7fab640b56e0 0x7fab64143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 --2- 172.21.15.22:0/505694734 >> [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] conn(0x7fab640b4ed0 0x7fab641f2b90 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 >> 172.21.15.22:0/505694734 conn(0x7fab6409ba20 msgr2=0x7fab64071220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 shutdown_connections 2024-09-21T05:49:02.888 INFO:teuthology.orchestra.run.smithi022.stderr:2024-09-21T05:49:02.886+0000 7fab6afa1640 1 -- 172.21.15.22:0/505694734 wait complete. 2024-09-21T05:49:02.899 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T05:49:02.899 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T05:49:02.899 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T05:49:02.899 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:02.911 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T05:49:02.911 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T05:49:02.911 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:02.962 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T05:49:02.962 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T05:49:02.962 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:02.976 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T05:49:02.976 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T05:49:02.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.027 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T05:49:03.027 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T05:49:03.027 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T05:49:03.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.056 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T05:49:03.056 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T05:49:03.056 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.122 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T05:49:03.122 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T05:49:03.123 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.191 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T05:49:03.191 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T05:49:03.191 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.258 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T05:49:03.258 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T05:49:03.258 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.294 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T05:49:03.294 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T05:49:03.294 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.362 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T05:49:03.362 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T05:49:03.362 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.429 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T05:49:03.429 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T05:49:03.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.501 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T05:49:03.502 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T05:49:03.502 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T05:49:03.502 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.530 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T05:49:03.530 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T05:49:03.530 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.572 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T05:49:03.572 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T05:49:03.573 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T05:49:03.573 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.585 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T05:49:03.585 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T05:49:03.585 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.625 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T05:49:03.625 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T05:49:03.625 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T05:49:03.677 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-21T05:49:03.678 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T05:49:03.678 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T05:49:03.678 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | egrep -v 'object missing on disk' | egrep -v 'Invalid tag char' | head -n 1 2024-09-21T05:49:03.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:49:03.717 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T05:49:03.900 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:49:03.901 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T05:49:03.989 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:49:03.990 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T05:49:04.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:49:04.072 DEBUG:teuthology.orchestra.run.smithi022:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T05:49:04.122 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:49:04.122 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T05:49:04.303 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:49:04.303 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T05:49:04.399 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:49:04.399 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T05:49:04.455 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:49:04.455 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T05:49:04.514 INFO:tasks.ceph:Archiving mon data... 2024-09-21T05:49:04.515 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/data/mon.a.tgz 2024-09-21T05:49:04.515 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-09-21T05:49:04.532 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.wUe1jBrHxk 2024-09-21T05:49:04.532 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wUe1jBrHxk 2024-09-21T05:49:04.697 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.wUe1jBrHxk 2024-09-21T05:49:04.782 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.wUe1jBrHxk is 445KB 2024-09-21T05:49:04.881 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.wUe1jBrHxk 2024-09-21T05:49:04.896 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/data/mon.b.tgz 2024-09-21T05:49:04.896 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-21T05:49:04.914 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.h8upkRAsqE 2024-09-21T05:49:04.914 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.h8upkRAsqE 2024-09-21T05:49:05.084 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.h8upkRAsqE 2024-09-21T05:49:05.170 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.h8upkRAsqE is 459KB 2024-09-21T05:49:05.240 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.h8upkRAsqE 2024-09-21T05:49:05.255 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/data/mon.c.tgz 2024-09-21T05:49:05.255 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-21T05:49:05.310 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.5JZz8PP8a3 2024-09-21T05:49:05.311 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5JZz8PP8a3 2024-09-21T05:49:05.460 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.5JZz8PP8a3 2024-09-21T05:49:05.543 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.5JZz8PP8a3 is 459KB 2024-09-21T05:49:05.639 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.5JZz8PP8a3 2024-09-21T05:49:05.654 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T05:49:05.654 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T05:49:05.657 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T05:49:05.737 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T05:49:05.776 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T05:49:05.776 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T05:49:05.778 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/remote/smithi022/crash 2024-09-21T05:49:05.778 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T05:49:05.804 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/remote/smithi134/crash 2024-09-21T05:49:05.805 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T05:49:05.840 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T05:49:05.840 DEBUG:teuthology.orchestra.run.smithi022:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:49:05.849 DEBUG:teuthology.orchestra.run.smithi134:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T05:49:05.928 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T05:49:05.954 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T05:49:05.978 INFO:tasks.ceph:Compressing logs... 2024-09-21T05:49:05.979 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T05:49:05.996 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T05:49:06.019 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T05:49:06.019 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50141.log 2024-09-21T05:49:06.019 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51889.log 2024-09-21T05:49:06.059 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42270.log 2024-09-21T05:49:06.059 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36777.log 2024-09-21T05:49:06.059 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T05:49:06.059 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42270.log.gz 2024-09-21T05:49:06.059 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42671.log 2024-09-21T05:49:06.059 INFO:teuthology.orchestra.run.smithi134.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36777.log.gz 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.3.42671.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42671.log.gz 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36697.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36697.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36697.log.gz 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36577.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.36577.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36577.log.gz 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T05:49:06.060 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36537.log 2024-09-21T05:49:06.061 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T05:49:06.061 INFO:teuthology.orchestra.run.smithi134.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T05:49:06.061 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T05:49:06.061 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36537.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36537.log.gz 2024-09-21T05:49:06.061 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-21T05:49:06.061 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36737.log 2024-09-21T05:49:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T05:49:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36737.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36737.log.gz 2024-09-21T05:49:06.080 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42275.log 2024-09-21T05:49:06.092 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36657.log 2024-09-21T05:49:06.099 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.1.42275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42675.log 2024-09-21T05:49:06.105 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42275.log.gz 2024-09-21T05:49:06.112 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36817.log 2024-09-21T05:49:06.112 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.3.42675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42675.log.gz 2024-09-21T05:49:06.113 INFO:teuthology.orchestra.run.smithi134.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36657.log.gz 2024-09-21T05:49:06.131 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36617.log 2024-09-21T05:49:06.131 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36817.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36817.log.gz 2024-09-21T05:49:06.146 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.36617.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36617.log.gz 2024-09-21T05:49:06.146 INFO:teuthology.orchestra.run.smithi134.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T05:49:06.189 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T05:49:06.259 INFO:teuthology.orchestra.run.smithi134.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T05:49:06.290 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.0.50141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61566.log 2024-09-21T05:49:06.290 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50141.log.gz 2024-09-21T05:49:06.290 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72953.log 2024-09-21T05:49:06.290 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51889.log.gz 2024-09-21T05:49:06.290 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64460.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72953.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61566.log.gz 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72953.log.gz 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.64460.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54411.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44912.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54411.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64460.log.gz 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54411.log.gz 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47146.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.2.44912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44912.log.gz 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36905.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50270.log: /var/log/ceph/ceph-client.admin.47146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-09-21T05:49:06.291 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47146.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72869.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36905.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68346.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72869.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65810.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72869.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68556.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68346.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.65810.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.67269.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70881.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65810.log.gz 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51975.log 2024-09-21T05:49:06.292 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.68556.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67269.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63878.log 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70881.log: /var/log/ceph/ceph-client.admin.51975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48782.log 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51975.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55494.log 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63878.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63878.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60262.log 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.70881.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48782.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54877.log 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55494.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60262.log.gz 2024-09-21T05:49:06.293 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52242.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42901.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54877.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58055.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53099.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58055.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52242.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58157.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58055.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52154.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64274.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58157.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52154.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53099.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43501.log 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52154.log.gz 2024-09-21T05:49:06.294 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.64274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54225.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64274.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60964.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43501.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73571.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54225.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48824.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60964.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45664.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73571.log: /var/log/ceph/ceph-client.admin.48824.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73571.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63836.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48824.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50592.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45664.log.gz 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-09-21T05:49:06.295 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50592.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63836.log.gz 2024-09-21T05:49:06.296 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56188.log 2024-09-21T05:49:06.296 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-09-21T05:49:06.296 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50592.log.gz 2024-09-21T05:49:06.296 INFO:teuthology.orchestra.run.smithi022.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz/var/log/ceph/ceph-client.admin.56188.log: 2024-09-21T05:49:06.296 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63923.log 2024-09-21T05:49:06.297 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56188.log.gz 2024-09-21T05:49:06.297 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70001.log 2024-09-21T05:49:06.297 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63923.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz -5 2024-09-21T05:49:06.297 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63923.log.gz 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61390.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.70001.log.gz 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51026.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42607.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61390.log: /var/log/ceph/ceph-client.admin.51026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62252.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51026.log.gz 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42607.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61390.log.gz 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53639.log 2024-09-21T05:49:06.298 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42607.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.62252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57836.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62252.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69137.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53639.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66933.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57836.log: /var/log/ceph/ceph-client.admin.69137.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57836.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65079.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59426.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69137.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66933.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52494.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65079.log.gz 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59004.log 2024-09-21T05:49:06.299 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59426.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73499.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37054.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59004.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52494.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73499.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62550.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37054.log: /var/log/ceph/ceph-client.admin.52452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70582.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37054.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59768.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70582.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62550.log.gz 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52752.log 2024-09-21T05:49:06.300 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70582.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.59768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73079.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59768.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.52752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61348.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52752.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72425.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73079.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61348.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61348.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56877.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64148.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72425.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65531.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.64148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48192.log 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64148.log.gz 2024-09-21T05:49:06.301 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56877.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58241.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65531.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48192.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58241.log: /var/log/ceph/ceph-client.admin.42691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54038.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58241.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55218.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46528.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54038.log.gz 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59258.log 2024-09-21T05:49:06.302 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55218.log: /var/log/ceph/ceph-client.admin.46528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55218.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46528.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36818.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59258.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51685.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43825.log: /var/log/ceph/ceph-client.admin.36818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56792.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36818.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59510.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51685.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49841.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56792.log: /var/log/ceph/ceph-client.admin.59510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72112.log 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49841.log: gzip -5 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56792.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49841.log.gz 2024-09-21T05:49:06.303 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.69060.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59510.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53954.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72112.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53954.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69060.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44131.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53954.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65984.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.44131.log: /var/log/ceph/ceph-client.admin.49589.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44131.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51254.log 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-09-21T05:49:06.304 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45091.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65984.log: /var/log/ceph/ceph-client.admin.51254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63202.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51254.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45091.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.65984.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50900.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45091.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50806.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63202.log: /var/log/ceph/ceph-client.admin.50900.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63202.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64718.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50900.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50806.log: /var/log/ceph/ceph-client.0.64718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58013.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64718.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50806.log.gz 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36621.log 2024-09-21T05:49:06.305 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55983.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58013.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48553.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36621.log: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.36621.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55983.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47530.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48553.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49087.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51338.log: /var/log/ceph/ceph-client.0.47530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42435.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47530.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-09-21T05:49:06.306 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49087.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44475.log 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42435.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47921.log 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49631.log: /var/log/ceph/ceph-client.0.44475.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58619.log 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44475.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58714.log 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47921.log: /var/log/ceph/ceph-client.admin.58619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71604.log 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47921.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58619.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49167.log 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58714.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.71604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71604.log.gz 2024-09-21T05:49:06.307 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60650.log 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49167.log.gz 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.60650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60650.log.gz 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61998.log 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43741.log: /var/log/ceph/ceph-client.admin.61998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68094.log 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43165.log 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61998.log.gz 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2024-09-21T05:49:06.308 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72911.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68094.log: /var/log/ceph/ceph-client.admin.43165.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68094.log.gz 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64526.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43165.log.gz 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72280.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72911.log: /var/log/ceph/ceph-client.admin.64526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46750.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72911.log.gz 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64526.log.gz 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72280.log.gz 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67395.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46750.log.gz 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55302.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51548.log: /var/log/ceph/ceph-client.admin.67395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49949.log 2024-09-21T05:49:06.309 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67395.log.gz/var/log/ceph/ceph-client.admin.55302.log: 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55260.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55302.log.gz 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49949.log: /var/log/ceph/ceph-client.admin.55260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44470.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49949.log.gz 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55260.log.gz 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66498.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59342.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.44470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44470.log.gz 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71308.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46486.log 2024-09-21T05:49:06.310 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59342.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66498.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58794.log 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.71308.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46486.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61072.log 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64190.log 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58794.log: /var/log/ceph/ceph-client.admin.61072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59576.log 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58794.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61072.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.64190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52368.log 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64190.log.gz 2024-09-21T05:49:06.311 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43993.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59576.log.gz 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70505.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52368.log: /var/log/ceph/ceph-client.admin.43993.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52368.log.gz 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45301.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70505.log: /var/log/ceph/ceph-client.admin.60746.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43993.log.gz 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45301.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70505.log.gz 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51590.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.45301.log.gz 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37035.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60746.log: /var/log/ceph/ceph-client.admin.51590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50942.log 2024-09-21T05:49:06.312 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51590.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67143.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60746.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.37035.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53996.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50942.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52019.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67143.log: /var/log/ceph/ceph-client.admin.53996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63099.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67143.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53996.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63713.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52019.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64316.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63099.log.gz 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67185.log 2024-09-21T05:49:06.313 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63713.log: /var/log/ceph/ceph-client.admin.64316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72070.log 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63713.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64316.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50681.log 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose 87.3% -- /var/log/ceph/ceph-client.admin.58451.log -- replaced with /var/log/ceph/ceph-client.admin.67185.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72070.log: /var/log/ceph/ceph-client.admin.50681.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72070.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58115.log 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50681.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73653.log 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58451.log: /var/log/ceph/ceph-client.admin.58115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47526.log 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58451.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58115.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47104.log 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73653.log.gz 2024-09-21T05:49:06.314 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49673.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.47526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47526.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42523.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1%/var/log/ceph/ceph-client.admin.49673.log: -- replaced with /var/log/ceph/ceph-client.admin.47104.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54537.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49673.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72028.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42523.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70736.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54537.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63538.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72028.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54922.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70736.log: /var/log/ceph/ceph-client.admin.63538.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70736.log.gz 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50137.log 2024-09-21T05:49:06.315 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63538.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36452.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54922.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54922.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.50137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73205.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50137.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61282.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36452.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55134.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73205.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62931.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54712.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61282.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55134.log.gz 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54122.log 2024-09-21T05:49:06.316 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62931.log: /var/log/ceph/ceph-client.admin.54712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58409.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62931.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54712.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46570.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54122.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55008.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58409.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54966.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46570.log: /var/log/ceph/ceph-client.admin.55008.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46570.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46095.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55008.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45801.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54966.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46095.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.46095.log.gz 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51765.log 2024-09-21T05:49:06.317 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45801.log.gz 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70699.log 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.51765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67968.log 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51765.log.gz 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68514.log 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55941.log 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8%/var/log/ceph/ceph-client.admin.67968.log: -- replaced with /var/log/ceph/ceph-client.admin.70699.log.gz 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67968.log.gz 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68810.log 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48343.log 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68514.log.gz 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55941.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55941.log.gz 2024-09-21T05:49:06.318 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.68810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60040.log 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68810.log.gz 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56006.log 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48343.log.gz 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60646.log 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60040.log.gz 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56006.log: /var/log/ceph/ceph-client.admin.49332.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.56006.log.gz 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.70067.log 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-09-21T05:49:06.319 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71031.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.60646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60646.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54369.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70067.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37050.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62973.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54369.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67227.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62973.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.37050.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47983.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.71031.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62973.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60350.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63286.log 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.67227.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47983.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47983.log.gz 2024-09-21T05:49:06.320 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50984.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60350.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56230.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63286.log: /var/log/ceph/ceph-client.admin.50984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61488.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63286.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50984.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49290.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48656.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56230.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61488.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54183.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48656.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49290.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60392.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48656.log.gz 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53870.log 2024-09-21T05:49:06.321 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.60392.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58493.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54183.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43951.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60392.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53870.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47230.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58493.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48301.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43951.log: /var/log/ceph/ceph-client.admin.47230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48427.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43951.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47230.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68388.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48301.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66326.log 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48427.log.gz 2024-09-21T05:49:06.322 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71608.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66326.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.68388.log.gz 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45217.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.71608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72238.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66326.log.gz 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.71608.log.gz 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53399.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45217.log.gz 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65023.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72238.log.gz 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51068.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53399.log: /var/log/ceph/ceph-client.admin.65023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42481.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.65023.log.gz 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61628.log 2024-09-21T05:49:06.323 INFO:teuthology.orchestra.run.smithi022.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53399.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51068.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36509.log 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53033.log 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42481.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36509.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61628.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65267.log 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36509.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65722.log 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.65267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65267.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52200.log 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53033.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56296.log 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65722.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52200.log.gz 2024-09-21T05:49:06.324 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46978.log 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56296.log.gz 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46156.log 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46978.log.gz 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46156.log: /var/log/ceph/ceph-client.admin.52326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63454.log 87.3% 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46156.log.gz 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2024-09-21T05:49:06.325 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60812.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63454.log: /var/log/ceph/ceph-client.admin.41851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48385.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63454.log.gz 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70255.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72341.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48385.log: /var/log/ceph/ceph-client.0.70255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61651.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60812.log.gz -- replaced with /var/log/ceph/ceph-client.admin.48385.log.gz 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70255.log.gz 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73415.log 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72341.log.gz 2024-09-21T05:49:06.326 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43500.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61651.log: /var/log/ceph/ceph-client.admin.73415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72827.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73415.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43159.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43500.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42565.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72827.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.61651.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58283.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43159.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67059.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42565.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59468.log 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58283.log.gz 2024-09-21T05:49:06.327 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71986.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67059.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59216.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59468.log: /var/log/ceph/ceph-client.admin.71986.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59468.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65609.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71986.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56649.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59216.log: /var/log/ceph/ceph-client.admin.65609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71202.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59216.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65609.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72154.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56649.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56985.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71202.log.gz 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-09-21T05:49:06.328 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72154.log: /var/log/ceph/ceph-client.admin.56985.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.72154.log.gz --verbose 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.63328.log 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56488.log 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56985.log.gz 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49799.log 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63328.log.gz 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.56488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56488.log.gz 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46612.log 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49799.log: /var/log/ceph/ceph-client.admin.37015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57971.log 2024-09-21T05:49:06.329 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49799.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45927.log 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46612.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73247.log 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57971.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72743.log 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45927.log: /var/log/ceph/ceph-client.admin.73247.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45927.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69573.log 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73247.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72743.log: /var/log/ceph/ceph-client.admin.69573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48044.log 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72743.log.gz 2024-09-21T05:49:06.330 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69573.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66651.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48044.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66651.log: /var/log/ceph/ceph-client.admin.42985.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66651.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64861.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51847.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.64861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64861.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47188.log 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51847.log.gz 2024-09-21T05:49:06.331 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55686.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67735.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47188.log: /var/log/ceph/ceph-client.0.55686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57727.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47188.log.gz 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55686.log.gz 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63057.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67735.log.gz 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59384.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63057.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57727.log.gz 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53912.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63057.log.gz 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66411.log 2024-09-21T05:49:06.332 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59384.log: /var/log/ceph/ceph-client.admin.53912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67864.log 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59384.log.gz 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53912.log.gz 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63141.log 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.66411.log.gz 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67864.log: /var/log/ceph/ceph-client.admin.63141.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67864.log.gz 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66086.log 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63141.log.gz 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5/var/log/ceph/ceph-client.admin.46011.log: --verbose -- /var/log/ceph/ceph-client.admin.57929.log 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49463.log 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2024-09-21T05:49:06.333 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53291.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66086.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57929.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64376.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49463.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73457.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53291.log: /var/log/ceph/ceph-client.admin.64376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44916.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64376.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.53291.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5/var/log/ceph/ceph-client.2.44916.log: --verbose -- /var/log/ceph/ceph-client.admin.53531.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.73457.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44916.log.gz 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58199.log 2024-09-21T05:49:06.334 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64060.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52748.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53531.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58199.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58199.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.64060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69818.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68052.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64060.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.52748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52748.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71244.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69818.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49421.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68052.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62334.log 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71244.log: /var/log/ceph/ceph-client.admin.49421.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71244.log.gz 2024-09-21T05:49:06.335 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49421.log.gz 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45049.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42110.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49547.log: /var/log/ceph/ceph-client.admin.45049.log: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62753.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62334.log.gz 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45049.log.gz 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69683.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36682.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42110.log.gz 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62753.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62753.log.gz 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58367.log 2024-09-21T05:49:06.336 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.69683.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68178.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36682.log: 78.6% -- replaced with /var/log/ceph/ceph-client.admin.36682.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63015.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57243.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58367.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68178.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68178.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55386.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42300.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63015.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57243.log: /var/log/ceph/ceph-client.admin.55386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54453.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55386.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42300.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57243.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42351.log 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54453.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42300.log.gz 2024-09-21T05:49:06.337 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50421.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54453.log.gz 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56104.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.42351.log.gz -5 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53828.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50421.log: /var/log/ceph/ceph-client.admin.56104.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.50421.log.gz 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56104.log.gz 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65895.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mon.a.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53828.log.gz 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59132.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58577.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57459.log 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65895.log.gz 2024-09-21T05:49:06.338 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58577.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66410.log 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59132.log.gz 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58577.log.gz 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62847.log 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57459.log.gzgzip 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose --/var/log/ceph/ceph-client.admin.66410.log: /var/log/ceph/ceph-client.admin.47338.log 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63633.log 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62847.log.gz 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47338.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.66410.log.gz 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47338.log.gz 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49883.log 2024-09-21T05:49:06.339 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70858.log 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46848.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63633.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49883.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70858.log: /var/log/ceph/ceph-client.admin.73289.log 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46053.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49883.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70858.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51464.log 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72659.log 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73289.log: /var/log/ceph/ceph-client.admin.51464.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73289.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.52410.log 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51464.log.gz 2024-09-21T05:49:06.340 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48992.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72659.log.gz 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48908.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42393.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48908.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48992.log.gz 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.48908.log.gz /var/log/ceph/ceph-client.admin.46442.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53573.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42393.log.gz 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62064.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46442.log: /var/log/ceph/ceph-client.admin.53573.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63244.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61814.log 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62064.log.gz/var/log/ceph/ceph-client.admin.63244.log: 2024-09-21T05:49:06.341 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53573.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48081.log 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63244.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.61814.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51931.log 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2%/var/log/ceph/ceph-client.admin.48081.log: -- replaced with /var/log/ceph/ceph-client.admin.61814.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71416.log 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48081.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51931.log: -- /var/log/ceph/ceph-client.admin.58535.log 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51931.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46358.log 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58535.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71416.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.58535.log.gz 2024-09-21T05:49:06.342 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68220.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46358.log.gz 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.43163.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68220.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43497.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68220.log.gz 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45496.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45343.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43497.log.gz 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56726.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45496.log.gz 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69959.log 2024-09-21T05:49:06.343 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.56726.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45343.log.gz 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62592.log 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69959.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.56726.log.gz 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55344.log 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69959.log.gz 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43164.log 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64418.log 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62592.log.gz 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43164.log: -- replaced with /var/log/ceph/ceph-client.admin.55344.log.gz 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53249.log 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43164.log.gz 2024-09-21T05:49:06.344 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.64418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70662.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64418.log.gz 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53249.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53249.log.gz 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72785.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.2.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.70662.log.gz 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61956.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69395.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6%/var/log/ceph/ceph-client.admin.61956.log: -- replaced with /var/log/ceph/ceph-client.admin.72785.log.gz 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61956.log.gz 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.0.62256.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.69395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69395.log.gz 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.0.62256.log: -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.42649.log 2024-09-21T05:49:06.345 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.0.62256.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42859.log 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45574.log/var/log/ceph/ceph-client.admin.42649.log: 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42859.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42649.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43499.log 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42859.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67777.log 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45574.log: /var/log/ceph/ceph-client.admin.43499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51296.log 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67777.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45574.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67777.log.gz 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52881.log 2024-09-21T05:49:06.346 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51296.log: gzip 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51296.log.gz 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65723.log 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43160.log 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52881.log.gz 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.65723.log.gz 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68430.log/var/log/ceph/ceph-client.admin.43160.log: 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49248.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43160.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56062.log 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1%/var/log/ceph/ceph-client.admin.56062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54080.log 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68430.log.gz 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56062.log.gz 2024-09-21T05:49:06.347 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64938.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43909.log: /var/log/ceph/ceph-client.admin.54080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50420.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz/var/log/ceph/ceph-client.admin.64938.log: 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54080.log.gz 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64938.log.gz 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66284.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68262.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66284.log: gzip 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50420.log.gz -5 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.0.59764.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66284.log.gz 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.68262.log: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.59764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46932.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68262.log.gz 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59764.log.gz 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-09-21T05:49:06.348 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.46932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69391.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43867.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46932.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73373.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.69391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67437.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69391.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46400.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73373.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67437.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49715.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67437.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46400.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46400.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70947.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42775.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5/var/log/ceph/ceph-client.admin.70947.log: --verbose -- /var/log/ceph/ceph-client.admin.48511.log 2024-09-21T05:49:06.349 INFO:teuthology.orchestra.run.smithi022.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49715.log.gz 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61868.log 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48511.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42775.log.gz 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr: 92.2% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48511.log.gz 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70947.log.gz 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73163.log 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.61868.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57417.log 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1%/var/log/ceph/ceph-client.admin.73163.log: -- replaced with /var/log/ceph/ceph-client.admin.61868.log.gz 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73163.log.gz 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.36964.log 2024-09-21T05:49:06.350 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57417.log: gzip -5 --verbose 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57417.log.gz 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.69762.log 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36964.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 86.6% 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36964.log.gz 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48950.log 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69762.log.gz 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66848.log 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.48950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68622.log 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66848.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48950.log.gz 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66848.log.gz 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59090.log 2024-09-21T05:49:06.351 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45385.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6%/var/log/ceph/ceph-client.admin.59090.log: -- replaced with /var/log/ceph/ceph-client.admin.68622.log.gz 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60304.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59090.log.gz 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45385.log: /var/log/ceph/ceph-client.admin.60304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59174.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5%/var/log/ceph/ceph-client.admin.43502.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60304.log.gz 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45385.log.gz 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48866.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59174.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59174.log.gz 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.72383.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48866.log: gzip -5 87.7% --verbose -- /var/log/ceph/ceph-client.admin.66585.log 2024-09-21T05:49:06.352 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48866.log.gz 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36537.log 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72383.log.gz 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69913.log 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66585.log: /var/log/ceph/ceph-client.admin.36537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72995.log 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr: 76.4% -- replaced with /var/log/ceph/ceph-client.admin.36537.log.gz 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61753.log 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66585.log.gz 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69913.log.gz 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73037.log 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72995.log.gz 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61753.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30408.log 2024-09-21T05:49:06.353 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71148.log 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.73037.log.gz 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph.tmp-client.admin.30408.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30408.log.gzgzip -5 --verbose -- 92.3% -- replaced with /var/log/ceph/ceph-client.admin.61753.log.gz 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr: /var/log/ceph/ceph-client.admin.37060.log 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71148.log: gzip -5 --verbose 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71148.log.gz 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.43156.log 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37060.log: gzip -5 --verbose 86.4% -- replaced with /var/log/ceph/ceph-client.admin.37060.log.gz 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.61711.log 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43156.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2024-09-21T05:49:06.354 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61910.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.61711.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47272.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61910.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67654.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47272.log: 92.3% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61711.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45843.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47272.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49757.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67654.log: /var/log/ceph/ceph-client.admin.45843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57093.log 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67654.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45843.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49757.log: gzip -5 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49757.log.gz 2024-09-21T05:49:06.355 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.71893.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.71893.log: 90.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46697.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57093.log.gz 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.71893.log.gz 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52560.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46697.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66142.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46697.log.gz 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49377.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52560.log.gz 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43167.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66142.log.gz 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60209.log 2024-09-21T05:49:06.356 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64018.log 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.49377.log: /var/log/ceph/ceph-client.admin.60209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49377.log.gz 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60209.log.gz/var/log/ceph/ceph-client.admin.64018.log: 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64018.log.gz 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55092.log 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45741.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-09-21T05:49:06.357 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36593.log 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55428.log 87.5% 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55092.log.gz 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36593.log: 77.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36593.log.gz 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55050.log 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67017.log 2024-09-21T05:49:06.358 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43783.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55428.log.gz 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55050.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr:gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55050.log.gz 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51110.log 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67017.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67017.log.gz 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.59300.log 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67908.log 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51110.log.gz 2024-09-21T05:49:06.359 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59300.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59300.log.gz 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.54495.log 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67908.log: gzip -5 --verbose 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67908.log.gz 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.48469.log 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54495.log: gzip -5 --verbose 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54495.log.gz 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.43495.log 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56484.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48469.log.gz 2024-09-21T05:49:06.360 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52284.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43495.log.gz 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.56484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56484.log.gz 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67574.log 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52284.log: gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52284.log.gz 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60458.log 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67574.log.gz 2024-09-21T05:49:06.361 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60458.log: gzip -5 --verbose -- 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60458.log.gz 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr: /var/log/ceph/ceph-client.admin.60898.log 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45133.log: gzip -5 --verbose -- 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr: /var/log/ceph/ceph-client.admin.44608.log 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63496.log 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.44608.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56146.log 2024-09-21T05:49:06.362 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60898.log.gz 2024-09-21T05:49:06.363 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44608.log.gz 2024-09-21T05:49:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66044.log 2024-09-21T05:49:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63496.log: /var/log/ceph/ceph-client.admin.56146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57309.log 2024-09-21T05:49:06.363 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63496.log.gz 2024-09-21T05:49:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66044.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56146.log.gz 2024-09-21T05:49:06.363 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42817.log 2024-09-21T05:49:06.364 INFO:teuthology.orchestra.run.smithi022.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.66044.log.gz 2024-09-21T05:49:06.364 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70625.log 2024-09-21T05:49:06.364 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57309.log.gz 2024-09-21T05:49:06.364 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5/var/log/ceph/ceph-client.admin.42817.log: --verbose -- /var/log/ceph/ceph-client.admin.62658.log 2024-09-21T05:49:06.364 INFO:teuthology.orchestra.run.smithi022.stderr: 87.1%/var/log/ceph/ceph-client.admin.70625.log: -- replaced with /var/log/ceph/ceph-client.admin.42817.log.gz 2024-09-21T05:49:06.364 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62484.log 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70625.log.gz 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62658.log.gz 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37395.log 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62484.log: /var/log/ceph/ceph-client.admin.46198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68814.log 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37395.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2024-09-21T05:49:06.365 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37395.log.gz 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69871.log 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62484.log.gz 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.68814.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.0.68814.log.gz 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55176.log 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53183.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69871.log.gz 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55176.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.58875.log 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55176.log.gz 2024-09-21T05:49:06.366 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72467.log 2024-09-21T05:49:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66806.log 2024-09-21T05:49:06.367 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53183.log.gz 2024-09-21T05:49:06.367 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58875.log: /var/log/ceph/ceph-client.admin.72467.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37048.log 2024-09-21T05:49:06.367 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58875.log.gz 2024-09-21T05:49:06.367 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72467.log.gz/var/log/ceph/ceph-client.admin.66806.log: 2024-09-21T05:49:06.367 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66806.log.gz 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57051.log 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37048.log: /var/log/ceph/ceph-client.0.55682.log 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57051.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.37048.log.gz 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57647.log 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57051.log.gz 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.55682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36564.log 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55682.log.gz 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.57647.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.0.57647.log.gz 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51194.log 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36564.log: 81.5% -- replaced with /var/log/ceph/ceph-client.admin.36564.log.gz 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72701.log 2024-09-21T05:49:06.368 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.51194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60142.log 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70792.log 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51194.log.gz 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61132.log 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72701.log.gz 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60142.log: /var/log/ceph/ceph-client.admin.70792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57351.log 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61132.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.70792.log.gz 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60142.log.gz 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37055.log 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61132.log.gz 2024-09-21T05:49:06.369 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.57351.log: -- /var/log/ceph/ceph-client.admin.58959.log 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37636.log 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.37055.log.gz 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.57351.log.gz 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58959.log: gzip -5 --verbose 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58959.log.gz 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.66196.log 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37636.log: gzip -5 --verbose 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37636.log.gz 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.63794.log 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66196.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66196.log.gz 2024-09-21T05:49:06.370 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42733.log 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47020.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.63794.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.60105.log 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58325.log 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42733.log: /var/log/ceph/ceph-client.admin.60105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66238.log 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42733.log.gz 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58325.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60105.log.gz 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60082.log 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58325.log.gz 2024-09-21T05:49:06.371 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.66238.log: -5 --verbose -- /var/log/ceph/ceph-client.0.70259.log 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.60082.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66238.log.gz 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60082.log.gz 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37435.log 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.70259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70259.log.gz 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72196.log 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62400.log 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37435.log.gz 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.72196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51422.log 2024-09-21T05:49:06.372 INFO:teuthology.orchestra.run.smithi022.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72196.log.gz 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54793.log 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62400.log: /var/log/ceph/ceph-client.admin.51422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68136.log 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51422.log.gz 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54793.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62400.log.gz 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.54793.log.gz 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46797.log 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54309.log 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68136.log.gz 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46797.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46797.log.gz 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41932.log 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37053.log 2024-09-21T05:49:06.373 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54309.log.gz/var/log/ceph/ceph-client.admin.41932.log: 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41932.log.gz 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57651.log/var/log/ceph/ceph-client.admin.37053.log: 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.55864.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.37053.log.gz 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42238.log 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42152.log 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.57651.log: gzip/var/log/ceph/ceph-client.admin.42238.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54267.log 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42238.log.gz 2024-09-21T05:49:06.374 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.0.57651.log.gz 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:gzip/var/log/ceph/ceph-client.admin.42152.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56943.log 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.42152.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.50346.log 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54267.log: /var/log/ceph/ceph-client.admin.56943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43166.log 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50346.log: 87.6% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56943.log.gz -- replaced with /var/log/ceph/ceph-client.admin.54267.log.gz 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43498.log 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50346.log.gz 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.375 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43166.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43166.log.gz 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.43498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68472.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43498.log.gz 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.36731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52977.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.68472.log: -- /var/log/ceph/ceph-client.admin.65961.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52977.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68472.log.gz 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36772.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52977.log.gz 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67479.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65961.log: /var/log/ceph/ceph-client.admin.36772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48118.log 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67479.log: 87.3% 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36772.log.gz -- replaced with /var/log/ceph/ceph-client.admin.65961.log.gz 2024-09-21T05:49:06.376 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67479.log.gz 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58917.log 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.48118.log: gzip -5 --verbose 89.3% -- /var/log/ceph/ceph-client.admin.66674.log 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48118.log.gz 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.58917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73611.log 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58917.log.gz 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61198.log 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.66674.log.gz/var/log/ceph/ceph-client.admin.73611.log: 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73611.log.gz 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.50508.log 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46655.log 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50508.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61198.log.gz 2024-09-21T05:49:06.377 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53734.log 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose 91.0% -- /var/log/ceph/ceph-client.admin.70989.log 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50508.log.gz 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46655.log: /var/log/ceph/ceph-client.admin.53734.log: gzip -5 --verbose 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53734.log.gz 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr: -- /var/log/ceph/ceph-client.admin.63370.log 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.70989.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46655.log.gz 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65271.log 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.63370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50658.log 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63370.log.gz 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.0.65271.log: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.70989.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.0.65271.log.gz 2024-09-21T05:49:06.378 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64714.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.50658.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50658.log.gz/var/log/ceph/ceph-client.0.64714.log: 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64714.log.gz 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52070.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61006.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69203.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52070.log.gz 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.61006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52924.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.69203.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69203.log.gz 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61006.log.gz 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68010.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.52924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63967.log 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52924.log.gz 2024-09-21T05:49:06.379 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64102.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5%/var/log/ceph/ceph-client.admin.63967.log: -- replaced with /var/log/ceph/ceph-client.admin.68010.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63967.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37059.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.64102.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64102.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62889.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.37059.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37059.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73121.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.62889.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62889.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68304.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73121.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73121.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53465.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.68304.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68304.log.gz 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67353.log 2024-09-21T05:49:06.380 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.53465.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53465.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67101.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67353.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67353.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45175.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67101.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67101.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57159.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45175.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45175.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47062.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.57159.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.57159.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59963.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.47062.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47062.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54632.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.59963.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59963.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73331.log 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.54632.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54632.log.gz 2024-09-21T05:49:06.381 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46890.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.73331.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73331.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67311.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.46890.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46890.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67819.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67311.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67311.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.67819.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67819.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42028.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.45969.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66975.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.42028.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42028.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66891.log 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66975.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66975.log.gz 2024-09-21T05:49:06.382 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64232.log 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.66891.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66891.log.gz 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71350.log 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63412.log 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.64232.log: /var/log/ceph/ceph-client.admin.71350.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64232.log.gz 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65000.log 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71350.log.gz/var/log/ceph/ceph-client.admin.63412.log: 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63412.log.gz 2024-09-21T05:49:06.383 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/ceph-client.admin.65000.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65000.log.gz 2024-09-21T05:49:06.384 INFO:teuthology.orchestra.run.smithi134.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T05:49:06.418 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T05:49:06.456 INFO:teuthology.orchestra.run.smithi022.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T05:49:06.530 INFO:teuthology.orchestra.run.smithi134.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T05:49:06.657 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T05:49:06.671 INFO:teuthology.orchestra.run.smithi022.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T05:49:06.671 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-21T05:49:06.677 INFO:teuthology.orchestra.run.smithi134.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T05:49:06.677 INFO:teuthology.orchestra.run.smithi134.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T05:49:06.678 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-21T05:49:06.678 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.645s 2024-09-21T05:49:06.679 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m3.299s 2024-09-21T05:49:06.679 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.129s 2024-09-21T05:49:06.784 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T05:49:06.790 INFO:teuthology.orchestra.run.smithi022.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T05:49:06.885 INFO:teuthology.orchestra.run.smithi022.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T05:49:06.887 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-09-21T05:49:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m0.879s 2024-09-21T05:49:06.887 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m3.764s 2024-09-21T05:49:06.888 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.475s 2024-09-21T05:49:06.888 INFO:tasks.ceph:Archiving logs... 2024-09-21T05:49:06.888 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/remote/smithi022/log 2024-09-21T05:49:06.889 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T05:49:07.825 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/remote/smithi134/log 2024-09-21T05:49:07.826 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T05:49:08.233 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T05:49:08.242 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T05:49:08.242 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T05:49:08.245 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T05:49:08.334 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T05:49:08.342 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T05:49:08.343 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:49:08.345 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T05:49:08.361 INFO:teuthology.orchestra.run.smithi134.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:49:08.361 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-09-21T05:49:08.365 INFO:teuthology.orchestra.run.smithi134.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:49:08.365 INFO:teuthology.orchestra.run.smithi134.stdout:=============================================================================== 2024-09-21T05:49:08.365 INFO:teuthology.orchestra.run.smithi134.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -345us[ -310us] +/- 58ms 2024-09-21T05:49:08.365 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 +774us[ +774us] +/- 64ms 2024-09-21T05:49:08.365 INFO:teuthology.orchestra.run.smithi134.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -561us[ -561us] +/- 90ms 2024-09-21T05:49:08.365 INFO:teuthology.orchestra.run.smithi134.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:49:08.367 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T05:49:08.367 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-09-21T05:49:08.367 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 -337us[ -335us] +/- 58ms 2024-09-21T05:49:08.367 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +771us[ +771us] +/- 64ms 2024-09-21T05:49:08.367 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 42 -542us[ -539us] +/- 90ms 2024-09-21T05:49:08.367 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T05:49:08.367 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T05:49:08.388 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T05:49:08.389 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T05:49:08.427 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T05:49:08.518 DEBUG:teuthology.orchestra.run.smithi134:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T05:49:08.585 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T05:49:08.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:49:08.617 DEBUG:teuthology.orchestra.run.smithi134:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T05:49:08.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:49:08.646 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T05:49:08.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T05:49:08.664 INFO:teuthology.task.internal:Duration was 986.535067 seconds 2024-09-21T05:49:08.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T05:49:08.672 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T05:49:08.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T05:49:08.675 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T05:49:08.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-21T05:49:08.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-21T05:49:08.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-21T05:49:08.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-21T05:49:08.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-21T05:49:08.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-21T05:49:08.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-21T05:49:08.716 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:49:08.727 INFO:teuthology.orchestra.run.smithi134.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T05:49:09.100 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T05:49:09.100 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-09-21T05:49:09.100 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T05:49:09.163 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-21T05:49:09.163 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T05:49:09.193 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T05:49:09.193 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T05:49:09.206 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T05:49:09.260 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T05:49:09.260 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T05:49:09.276 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T05:49:09.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T05:49:09.421 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T05:49:09.421 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T05:49:09.447 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T05:49:09.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T05:49:09.482 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:49:09.489 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:49:09.511 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-09-21T05:49:09.536 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-21T05:49:09.550 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:49:09.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:49:09.580 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T05:49:09.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T05:49:09.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T05:49:09.618 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T05:49:09.619 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/remote/smithi022 2024-09-21T05:49:09.619 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T05:49:09.654 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913499/remote/smithi134 2024-09-21T05:49:09.655 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T05:49:09.689 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T05:49:09.689 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T05:49:09.691 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T05:49:09.739 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T05:49:09.751 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T05:49:09.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T05:49:09.761 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T05:49:09.761 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T05:49:09.764 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T05:49:09.777 INFO:teuthology.orchestra.run.smithi022.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 05:49 /home/ubuntu/cephtest 2024-09-21T05:49:09.793 INFO:teuthology.orchestra.run.smithi134.stdout: 265707 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 05:49 /home/ubuntu/cephtest 2024-09-21T05:49:09.795 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T05:49:09.804 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T05:49:09.826 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} duration: 986.5350668430328 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T05:49:09.826 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T05:49:09.929 INFO:teuthology.run:pass